This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Minor FP bug in object.c
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.0, Python 3.1, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mark.dickinson Nosy List: christian.heimes, mark.dickinson, nmm1, nobody, tim.peters
Priority: normal Keywords: patch

Created on 2003-08-15 15:15 by nmm1, last changed 2022-04-10 16:10 by admin. This issue is now closed.

Messages (10)
msg17786 - (view) Author: Nick Maclaren (nmm1) Date: 2003-08-15 15:15
This is closely related to the one I have reported
in floatobject.c (789159).  VERY closely.  It is
shown up by the following test on a machine with
64-bit longs and floating-point trapping turned on
(though it might show up as bogus results even with
no trapping):

print int( 9223372036854775200.0)
print int( 9223372036854775800.0)
print int(-9223372036854776800.0)
print int(-9223372036854777000.0)

958,959c958,959
<       double intpart, fractpart;
<       int expo;
---
>       double intpart, fractpart, z;
>       int expo, i, j;
978c978,989
<               if (intpart > LONG_MAX || -intpart >
LONG_MAX) {
---
>               /* Remember that (double)LONG_MAX can
round either way. */
>               if (intpart > LONG_MIN/2 && intpart <
LONG_MAX/2)
>                       z = 0.0;
>               else {
>                       z = (intpart >= 0.0 ? intpart :
-intpart);
>                       for (i =
(sizeof(long)*CHAR_BIT-1)/16; i >= 0; --i) {
>                               x = LONG_MAX;
>                               for (j = 0; j < i; ++j)
x >>= 16;
>                               z -=
ldexp(x&0xffff,16*i);
>                       }
>               }
>               if (z > 0.0) {
msg17787 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-02-17 00:22
Logged In: YES 
user_id=31435

Sorry, I can't make for this, so unassigned it.

To help the next person,

1. Please say something about what "the bug" is.  You show 4 
lines printing int(big_integer), but don't show any output.  In 
addition, the patch *appears* to be against _Py_HashDouble
(), but that function isn't involved in int() or in printing.

2. Please generate a context diff for patches, and attach the 
patch to the bug report instead of pasting it into the text 
box.  SourceForge destroys the intended line structure and 
indentation in text boxes, and this non-context diff patch is 
very hard to follow in this mangled form.
msg17788 - (view) Author: Nick Maclaren (nmm1) Date: 2004-03-09 12:20
Logged In: YES 
user_id=652073

The bug is overflow (i.e. undefined behaviour), and so
the symptoms will vary according to the system.  Under
the conditions I was running, there was no output,
because Python terminated with a SIGFPE.  As I said, it
could as easily show up as erroneous results.

In THIS case, the failure occurs in the input processing,
and the purpose of prefixing it by print and int() is if
you try it on a system where it gives wrong answers.  If
you don't have access to a system where overflow can be
turned on or one that uses unusual arithmetic, you will not
be able to repeat it.  That is why I created a fix.

The reason that I didn't attach is is that attachment was
broken; I don't know why, and have neither the time nor the
inclination to debug a Web interface that I do not manage.
I can trivially send such things by Email, which is far
more reliable.  There doesn't appear to be a mechanism to
try again, so here is a context diff:

*** object.c.org        Wed Feb 19 03:21:21 2003
--- object.c    Fri Aug 15 15:22:50 2003
***************
*** 955,962 ****
  long
  _Py_HashDouble(double v)
  {
!       double intpart, fractpart;
!       int expo;
        long hipart;
        long x;         /* the final hash value */
        /* This is designed so that Python numbers of
different types
--- 955,962 ----
  long
  _Py_HashDouble(double v)
  {
!       double intpart, fractpart, z;
!       int expo, i, j;
        long hipart;
        long x;         /* the final hash value */
        /* This is designed so that Python numbers of
different types
***************
*** 975,981 ****
  #endif
        if (fractpart == 0.0) {
                /* This must return the same hash as an
equal int or long. */
!               if (intpart > LONG_MAX || -intpart >
LONG_MAX) {
                        /* Convert to long and use its hash.
*/
                        PyObject *plong;        /* converted
to Python long */
                        if (Py_IS_INFINITY(intpart))
--- 975,992 ----
  #endif
        if (fractpart == 0.0) {
                /* This must return the same hash as an
equal int or long. */
!               /* Remember that (double)LONG_MAX can round
either way. */
!               if (intpart > LONG_MIN/2 && intpart <
LONG_MAX/2)
!                       z = 0.0;
!               else {
!                       z = (intpart >= 0.0 ? intpart :
-intpart);
!                       for (i =
(sizeof(long)*CHAR_BIT-1)/16; i >= 0; --i) {
!                               x = LONG_MAX;
!                               for (j = 0; j < i; ++j) x
>>= 16;
!                               z -= ldexp(x&0xffff,16*i);
!                       }
!               }
!               if (z > 0.0) {
                        /* Convert to long and use its hash.
*/
                        PyObject *plong;        /* converted
to Python long */
                        if (Py_IS_INFINITY(intpart))
msg17789 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-03-15 04:27
Logged In: YES 
user_id=31435

Nick, that helps (thanks!), but I still don't understand the 
connection between your examples and the patch.  The 
patch is indeed to _Py_HashDouble(), but that shouldn't get 
executed when doing, e.g.,

print int(9223372036854775200.0)

Is there a relationship between the examples and the patch, 
or do the examples belong to some other bug report?

Or is another assumption here that you're typing those 
examples in to an interactive Python shell?  *Then* 
_Py_HashDouble() would get called, as part of compilation.
msg17790 - (view) Author: Nick Maclaren (nmm1) Date: 2004-03-15 18:24
Logged In: YES 
user_id=652073

I think that we are at cross-purposes.  Yes, THIS bug occurs
at compilation time, though it makes no difference whether
Python is being used interactively or as a script.  The
overflow occurs during the conversion of the strings to the
doubles.  The examples show it quite clearly if you run
it with overflow trapping set to SIGFPE, which can be done
under Solaris as well as other systems.  I can deduce the
properties of systems that would give wrong answers, but
don't know of any that actually exist.
msg59176 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-03 23:32
Tim, you are the expert in numbers. Is the patch still relevant for 2.5
and newer?
msg59212 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2008-01-04 04:00
Unassigned myself -- I don't care about this 4 years later either ;-)
msg80878 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-31 16:35
I'll take this.

I think there *is* a problem here: on a system with 64-bit long, 
_Py_HashDouble can end up trying to cast the float value 2.**63 to a C 
long.  Since 2.**63 doesn't fit in a long, we get undefined behaviour, 
according to the C standards.

In more detail: when computing the hash of the floating-point value x = 
2.**63 on a 64-bit system, we end up in _Py_HashDouble in 
Objects/object.c.  The intpart of x will be 2.**63, and fractpart is 0.0.  
The comparison:

  if (intpart > LONG_MAX || -intpart > LONG_MAX) {

*fails*, because LONG_MAX (== 2**63-1) is implicitly converted to the 
float value 2.**63 before the comparison.  So we end up at the line:

  x = (long)intpart;

which attempts to convert intpart(== 2.0**63) to a long.  Even if this 
conversion doesn't signal, it could return some random long number, so 
unless we're very lucky, hash(2**63) == hash(2.**63) will fail.

On my system (OS X 10.5.4/x86_64), I *am* lucky:  when 2.**63 is cast to 
long I get LONG_MIN (!).  And it just so happens that hash(LONG_MIN) == 
hash(2**63), so we're okay.
msg81380 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-02-08 14:48
Fixed in the trunk in r69436.  Will merge to 2.6, 3.1 and 3.0.
msg81384 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-02-08 15:14
Fix merged in r69437, r69440, r69441.
History
Date User Action Args
2022-04-10 16:10:39adminsetgithub: 39070
2009-02-08 15:14:40mark.dickinsonsetstatus: open -> closed
resolution: fixed
messages: + msg81384
2009-02-08 14:48:44mark.dickinsonsetmessages: + msg81380
2009-01-31 16:35:29mark.dickinsonsetassignee: nobody -> mark.dickinson
versions: + Python 2.6, Python 3.0, Python 3.1, Python 2.7, - Python 2.5
type: behavior
messages: + msg80878
nosy: + mark.dickinson
2008-01-04 04:00:44tim.peterssetassignee: tim.peters -> nobody
messages: + msg59212
nosy: + nobody
2008-01-03 23:32:36christian.heimessetkeywords: + patch
assignee: tim.peters
versions: + Python 2.5, - Python 2.2
messages: + msg59176
nosy: + christian.heimes
2003-08-15 15:15:56nmm1create