This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: gzip module cannot write large files
Type: Stage:
Components: Library (Lib) Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: bescoto, brett.cannon, eichin, jlgijsbers, midramas, staschuk
Priority: normal Keywords:

Created on 2003-02-08 19:52 by bescoto, last changed 2022-04-10 16:06 by admin. This issue is now closed.

Messages (8)
msg14515 - (view) Author: Ben Escoto (bescoto) Date: 2003-02-08 19:52
Something like this:

import gzip
fin = open("/dev/urandom", "rb")
fout = gzip.GzipFile("out", "wb")
for i in range(4500): fout.write(fin.read(1024*1024))
fout.close()

will result in an error like:

Exception 'long int too large to convert to int' raised
of class 'exceptions.OverflowError':
...
  File "/usr/local/lib/python2.2/gzip.py", line 253, in
close
    write32(self.fileobj, self.size)
  File "/usr/local/lib/python2.2/gzip.py", line 19, in
write32
    output.write(struct.pack("<l", value))

Tested in python 2.2.1
msg14516 - (view) Author: Steven Taschuk (staschuk) Date: 2003-03-30 18:42
Logged In: YES 
user_id=666873

Current CVS has a fix for this, as of revision 1.37 (part of 2.3a1).

See patch 618135: http://www.python.org/sf/618135.
msg14517 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-05-29 01:20
Logged In: YES 
user_id=357491

From what I can gather from cvs log this was not backported.  Should it be?
msg14518 - (view) Author: Michael Bartl (midramas) Date: 2003-09-24 23:09
Logged In: YES 
user_id=31511

I'd propose to close this bug (fixed in the 2.3 series).

The fix can't be backported, because 2.2.3 was the final
version of the 2.2 series.
msg14519 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-09-24 23:36
Logged In: YES 
user_id=357491

No one ever said 2.2.3 was the final release of the 2.2 branch.  
There was talk just last month about a 2.2.4 release towards the 
end of the year.
msg14520 - (view) Author: Michael Bartl (midramas) Date: 2003-09-25 08:53
Logged In: YES 
user_id=31511

Sorry for the error in my last comment: a backport can still
be done for 2.2.4 or later. It seems the "final" confused
me. I found it out some minutes after I wrote the comment.

As the error is still present in 2.2.3, I will provide a
patch that backports it (you can then vote for it <wink> :)
msg14521 - (view) Author: Mark Eichin (eichin) Date: 2004-11-27 23:00
Logged In: YES 
user_id=79734

It's been a year - can we assume, with 2.4rc1 out, that there really won't 
be a 2.2.4 to put a backport into, and close this?
msg14522 - (view) Author: Johannes Gijsbers (jlgijsbers) * (Python triager) Date: 2004-11-27 23:15
Logged In: YES 
user_id=469548

Yes, I think it can.
History
Date User Action Args
2022-04-10 16:06:38adminsetgithub: 37939
2003-02-08 19:52:33bescotocreate