This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: itertools.count wraps around after maxint
Type: enhancement Stage:
Components: Extension Modules Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: nnorwitz, phr, rhettinger
Priority: normal Keywords:

Created on 2005-10-13 23:27 by phr, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (7)
msg54638 - (view) Author: paul rubin (phr) Date: 2005-10-13 23:27
See below.  This goes against the notion of int/long
unification and can cause weird unexpected behavior,
almost like a buffer overflow.  It should promote to a
long at the appropriate time, or that's not feasible,
then raise an exception, don't wrap around silently. 
Xrange is also not so great about this.  It at least
raises an exception if you give it too large an
endpoint, but promoting to long would be better.

Steven D'Aprano and others on clpy pointed this out.

>>> from itertools import count
>>> b=2**31 - 3
>>> c = count(b)
>>> for i in range(5):
...    print c.next()
...
2147483645
2147483646
2147483647
-2147483648
-2147483647
>>>
msg54639 - (view) Author: paul rubin (phr) Date: 2005-10-13 23:29
Logged In: YES 
user_id=72053

I forgot to say, the test example is Python 2.4.1 on linux.
 I don't know if 2.4.2 has a fix.  I don't see anything
about it in the database.
msg54640 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-10-14 05:18
Logged In: YES 
user_id=33168

I agree something should be done.  Raymond, which behaviour
would you prefer?  I can implement if you want (just let me
know and assign back to me).

BTW, I don't have the same problem.  I need to set b = 2**63
- 3 :-)
(using current CVS).
msg54641 - (view) Author: paul rubin (phr) Date: 2005-10-14 05:53
Logged In: YES 
user_id=72053

If both fixes are feasible then promoting to long is
definitely the correct one.  Raising an exception is just a
kludge to at least not do something horrible silently. 
However, on a fast 32 bit machine, counting past 2**31 or
something is quite realistic.  A web server might send out
that many packets in a few days or weeks.  It shouldn't
crash or go crazy after running for a long time and
overflowing maxint.

It occurs to me, the enumerate iterator should also be
checked and fixed if needed.  It, too, can overflow maxint
if counting something like a network stream.  Maybe there
are some more iterators besides these, which need the same
attention.
msg54642 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2005-10-14 07:12
Logged In: YES 
user_id=80475

It's not a bug -- it is the documented behavior.

The simple work-around is to roll your own generators:

    def count(n):
        while 1:
             yield n
             n += 1

    def enumerate(iterable):
         c = 0
         for elem in iterable:
               yield (c, elem)
               c += 1

Will look at possibly enhancing this for Py2.5.
msg54643 - (view) Author: paul rubin (phr) Date: 2005-10-14 15:11
Logged In: YES 
user_id=72053

You're right, the docs do describe that behavior of
itertools.count (someone on clpy thought otherwise, IIRC). 
I don't see anything in
http://docs.python.org/lib/built-in-funcs.html about what
enumerate does.  It looks my p3-750 can enumerate about 400k
iterations of itertools.repeat(None) per second, so it can
reach maxint in about 1.5 hours, but I don't feel like
running it that long to see what happens. At minimum,
enumerate's doc should be updated to say what it does.

I suppose it's a matter of opinion but I'd take the view
that both of these wraparounds (assuming enumerate wraps
around) are bugs even if they're documented bugs. 
msg54644 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2007-02-08 00:07
Nows raises an OverflowError instead of silently wrapping around.  See revisions 53665 and 53666.
History
Date User Action Args
2022-04-11 14:56:13adminsetgithub: 42481
2005-10-13 23:27:15phrcreate