This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: segmentation fault importing huge source
Type: Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jlt63 Nosy List: gersonkurz, jlt63, nnorwitz, tim.peters
Priority: normal Keywords:

Created on 2002-10-07 17:15 by gersonkurz, last changed 2022-04-10 16:05 by admin. This issue is now closed.

Messages (7)
msg12627 - (view) Author: Gerson Kurz (gersonkurz) Date: 2002-10-07 17:15
I posted a problem with importing a very large sourcefile 
in python 2.2.1 to clp, see 
<news:3da070ec.18868296@news.t-online.de>. 
Someone suggested that 2.3 would handle this situation 
much better, so I got the current version from CVS, built 
it with cygwin and tested it. That version raises a 
segmentation fault. The stackdump shows an 
ACCESS_VIOLATION somewhere inside python2.3.exe.

You can download the source that causes this at

http://p-nand-q.com/python/toobigfor2.3.zip

but beware, the zipped source is 613k large (unzipped 
around 4.2 mb).
msg12628 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-10-11 01:43
Logged In: YES 
user_id=33168

Do you know where python crashed?  My guess was after a
failed malloc.  I tried testing this, but I don't have
enough memory/cpu/patience to wait for loading such a large
file.  Can you make the test case smaller?  What is the
minimum size that still causes the seg fault?  What version
of cygwin?  There is a known malloc problem in 1.3.12, see
http://sources.redhat.com/ml/newlib/2002/msg00369.html
msg12629 - (view) Author: Gerson Kurz (gersonkurz) Date: 2002-10-13 15:47
Logged In: YES 
user_id=367261

It is the latest version of cygwin I know of, the cygwin1.dll is 
dated 2002-07-06 02:16. uname -a says

CYGWIN_NT-5.0 DREIZEHN 1.3.12(0.54/3/2) 2002-07-06 
02:16 i686 unknown

so I guess its the 1.3.12 you mentioned.

I can reproduce the crash using the following very simple 
script

b = {}
for i in range(900000):
    b[i] = [0] * 60

when I use range(900000), I get a crash. When I use range
(800000), I see this

Traceback (most recent call last):
  File "test2.py", line 3, in ?
    b[i] = [0] * 60

and then a crash. When I use (500000), it works. It smells 
like some 256mb limit or something; the memory for the 
process goes up to 264.000k which comes close, and then 
crashes. 
msg12630 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-10-18 19:35
Logged In: YES 
user_id=33168

Jason can you shed any more light on this problem?  Is this
the same cygwin malloc problem or a new problem?
msg12631 - (view) Author: Jason Tishler (jlt63) * (Python triager) Date: 2002-10-18 20:14
Logged In: YES 
user_id=86216

Yes, this is the same Cygwin malloc problem.

Gerson, please upgrade to Cygwin 1.3.13.

IMO, the following is the easiest way to verify that the
problem has been corrected:

$ python -c 'import sys; list(xrange(sys.maxint / 4))'
Traceback (most recent call last):
  File "<string>", line 1, in ?
MemoryError

Gerson, please let me know if it's OK to close this
bug report.
msg12632 - (view) Author: Gerson Kurz (gersonkurz) Date: 2002-10-19 05:16
Logged In: YES 
user_id=367261

you are right, it works with cygwin 1.3.13 - the allocation 
succeeds. you can close the bug report. 
msg12633 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2002-10-19 06:24
Logged In: YES 
user_id=31435

Marked 3rdParty and Fixed; was already Closed.  Thanks 
for the followup, Gerson!
History
Date User Action Args
2022-04-10 16:05:43adminsetgithub: 37280
2002-10-07 17:15:24gersonkurzcreate