This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: socket.recv() raises MemoryError exception (WindowsXP ONLY)
Type: Stage:
Components: Windows Versions:
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: tim.peters Nosy List: debatem1, randyw, tim.peters
Priority: normal Keywords:

Created on 2003-12-03 19:16 by randyw, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
sockMemErrWinXP.zip randyw, 2003-12-03 19:16 Client demonstrates bug, with matching server.
Messages (5)
msg19256 - (view) Author: Randy Wiser (randyw) Date: 2003-12-03 19:16
Calling socket.recv(size) with size starting at 3000000 
and decreasing by approximately 1024 each time, quickly 
raises a MemoryError exception, although the process 
memory use does not exceed 30 Mbytes!  Verified on 
Python 2.1, 2.3 and 2.3.2 (under Windows XP 
Professional ONLY version 5.1.2600 hotfix KB824146 if it 
matters :-)

Cannot reproduce (test runs many iterations without fail) 
under Windows 2000 Professional or Server. To reproduce 
on XP, run attached files is separate shell windows:

c:\> python hostMemErrSrv.py
waiting for client on 3021
Connected by ('127.0.0.1', 1217)
. . . . . . . . . . . . . . . .

c:\> python hostMemErr.py
MemoryError, hit return to exit.
Traceback (most recent call last):
  File "hostMemErr.py", line 117, in ?
    main()
  File "hostMemErr.py", line 108, in main
    data = p.dataRecv(sz)
  File "hostMemErr.py", line 63, in dataRecv
    tmp = self.dataSocket.recv (nleft)
  File "<string>", line 1, in recv
MemoryError
msg19257 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2003-12-03 19:28
Logged In: YES 
user_id=31435

Python doesn't do anything differently across different flavors 
of Windows, so you're probably looking at a Microsoft bug, or 
an MS design limitation.  Note that doing recv() with an 
argument in the millions isn't a sane thing to do to begin with.

Unassigned from me, and reduced priority (I'm not going to do 
anything more on this; Randy, try writing equivalent code 
directly in C; if the same thing happens there, as appears 
likely, then your complaint should go to Microsoft).
msg19258 - (view) Author: Randy Wiser (randyw) Date: 2003-12-04 01:16
Logged In: YES 
user_id=215159

A quick test with Python 2.3 -i shows the following, so I'm 
guessing if at first WinXP claims no memory, try try again.  
Thanks for the quick response Tim. We'll try not to ask WinXP 
for 3 Mil. of memory unless we really need it.

python -i hostMemErr.py
MemoryError, hit return to exit.
Traceback (most recent call last):
  File "hostMemErr.py", line 117, in ?
    main()
  File "hostMemErr.py", line 108, in main
    data = p.dataRecv(sz)
  File "hostMemErr.py", line 63, in dataRecv
    tmp = self.dataSocket.recv (nleft)
MemoryError
>>> s= 'z' * 3000000
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
MemoryError
>>> s= 'z' * 3000000
>>> s= 'z' * 3000000
>>> s= 'z' * 3000000
>>> s= 'z' * 30000000
>>> ^Z
msg19259 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2003-12-15 01:47
Logged In: YES 
user_id=31435

It's not crazy to ask WinXP for 3MB of memory in general, it's 
simply weird to pass a number that large to .recv() 
specifically.  Sockets are a low-level facility, working directly 
with system network buffers, whose sizes are measured in KB 
rather than MB.

Since we don't have Microsoft's socket implementation to 
stare at, we can't be sure, but it's possible that XP's 
implementation goes insane when fed unreasonable 
arguments.  If so, I expect they'll eventually fix that.

Regardless, I don't see anything here that's plausibly Python's 
doing (other than reporting back that the system malloc() 
returned NULL), so I'm closing this as 3rd-party, wontfix.
msg114267 - (view) Author: geremy condra (debatem1) Date: 2010-08-18 19:12
I can sporadically reproduce this on Linux as well, repeatedly passing values larger than 2**20 to recv. Doesn't seem to happen with regularity, but I have better than 11Gb of unused RAM on hand. Is there a good reason for this?
History
Date User Action Args
2022-04-11 14:56:01adminsetgithub: 39654
2010-08-18 19:12:02debatem1setnosy: + debatem1
messages: + msg114267
2003-12-03 19:16:41randywcreate