This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: SimpleXMLRPCServer cannot handle large requests
Type: Stage:
Components: Library (Lib) Versions: Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: akuchling Nosy List: akuchling, georg.brandl, morissette, noplay
Priority: normal Keywords:

Created on 2003-08-21 15:37 by morissette, last changed 2022-04-10 16:10 by admin. This issue is now closed.

Messages (5)
msg17865 - (view) Author: Marc-Andr� Morissette (morissette) Date: 2003-08-21 15:37
SimpleXMLRPCServer throws a WSAEINTR ioerror on 
large XML-RPC requests.

Under Windows, the socket.read() method cannot seem 
to handle large (tens of megabytes) reads (could be a 
Python specific problem). This means that very large 
XML-RPC requests can cause the exception.

Here is a tentative patch against 2.2.3 to fix the 
problem. It should be easy to port it to 2.3

---
 /cygdrive/c/Python22/Lib/SimpleXMLRPCServer.py      
2003-07-09 14:16:52.000000000 -0400
+++ /cygdrive/z/SimpleXMLRPCServer.py   2003-08-21 
11:01:19.000000000 -0400
@@ -73,6 +73,8 @@
 import SocketServer
 import BaseHTTPServer
 import sys
+import cStringIO

 class SimpleXMLRPCRequestHandler
(BaseHTTPServer.BaseHTTPRequestHandler):
     """Simple XML-RPC request handler class.
@@ -95,7 +97,14 @@

         try:
             # get arguments
-            data = self.rfile.read(int(self.headers["content-
length"]))
+            max_chunk_size = 10000000
+            content_length = int(self.headers["content-
length"])
+            buffer = cStringIO.StringIO()
+            for offset in range(0, content_length, 
max_chunk_size):
+                chunk_size = min(content_length - offset, 
max_chunk_size)
+                buffer.write(self.rfile.read(chunk_size))
+            data = buffer.getvalue()
+            buffer.close()
             params, method = xmlrpclib.loads(data)

             # generate response

msg17866 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2005-06-01 12:26
Logged In: YES 
user_id=1188172

Marc-Andre, can you still reproduce this with Python 2.4?
msg17867 - (view) Author: Julien Duponchelle (noplay) * Date: 2005-10-21 09:36
Logged In: YES 
user_id=446148

I have the same problem with Python2.4 with windows and
linux version.

If XML-RPC server reads to large buffer, it returns only a
part of the buffer.
msg17868 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2005-12-04 15:16
Logged In: YES 
user_id=11375

Why does SimpleXMLRPCServer need to be fixed if
socket.read() is choking?  Shouldn't socket.read() be fixed
instead?

noplay: do you mean you're seeing this bug happen on Linux?
 I don't use Windows, so being able to reproduce the problem
on Linux would be very useful.
msg17869 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2005-12-04 15:38
Logged In: YES 
user_id=11375

It turns out that on my Mac socket.read() runs into trouble
around 15Mb, so I could reproduce the problem.  Fix committed
in rev 41586; it's the same principle as the suggested
change, but the code is rewritten for better scalability
(appending to a list instead of using a cStringIO).  Thanks!
History
Date User Action Args
2022-04-10 16:10:45adminsetgithub: 39100
2003-08-21 15:37:31morissettecreate