When building python 2.5 with the free Intel compilers for non-commercial use (icc), the build works without optimizations, but fails when optimizations -O2 and -O3 are used.

The compilation proceedes without errors (though there are lot's of warnings and remarks) and makes a python executable, but when the python setup starts, the intepreter cannot import certain packages and cannot add two strings.

case $MAKEFLAGS in \
*-s*)  CC='icc -pthread' LDSHARED='icc -pthread -shared' OPT='-DNDEBUG -g -O2 -Ob2 -w1' ./python -E ./setup.py -q build;; \
*)  CC='icc -pthread' LDSHARED='icc -pthread -shared' OPT='-DNDEBUG -g -O2 -Ob2 -w1' ./python -E ./setup.py build;; \
esac
'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "./setup.py", line 6, in <module>
    import sys, os, imp, re, optparse
  File "/int/apps/mmf/src/Python-2.5_intel/Lib/optparse.py", line 71, in <module>
    import textwrap
  File "/int/apps/mmf/src/Python-2.5_intel/Lib/textwrap.py", line 10, in <module>
    import string, re
  File "/int/apps/mmf/src/Python-2.5_intel/Lib/string.py", line 26, in <module>
    letters = lowercase + uppercase
SystemError: error return without exception set
make: *** [sharedmods] Error 1

For example:
$ ./python
'import site' failed; use -v for traceback
Python 2.5 (r25:51908, Mar 27 2007, 20:10:22) 
[GCC Intel(R) C++ gcc 3.4 mode] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = "123"
>>> a + "12"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: error return without exception set
>>> 

Note that both the import of the site file fails and the addition fails.

Python was configured with the following options:
./configure --with-gcc=icc\
            --with-cxx-main=icc\
            --prefix=/int/apps/mmf/apps/Python-2.5_intel/\
            OPT="-g -O2 -Ob2 -w1"
----------------------------
When compiled with fewer optimiztions, everything seems to work.
./configure --with-gcc=icc\
            --with-cxx-main=icc\
            --prefix=/int/apps/mmf/apps/Python-2.5_intel/\
            OPT="-g -O1 -Ob2 -w1"

Can you debug this further to determine a specific problem? It could be a compiler bug also, for all I can tell.

I have spoken with some people who have had similar issues with the intel compilers in other applications, so it looks like it might be a compiler bug.  I will look into this.

Can anyone confirm this behaviour? (I only have access to one linux box).

My guess is that it relates to strict aliasing (which Python violates).  Can you find the equivalent flag to gcc's -fno-strict-aliasing in icc?

It does not appear to be a problem with aliasing (icc accepts the -fno-strict-aliasing flag and it is set by default using ./configure.  It is another question of whether or not icc behaves this flag!).

This looks similar to a previous problem but I am not using any processor specific optimizations:
http://mail.python.org/pipermail/python-list/2005-March/312145.html

I have narrowed the problem a bit to the file Python/ceval.c.  If you build this manually without optimizations, then everything works fine ("make test" works).
icc -pthread -c -fno-strict-aliasing -DNDEBUG -g -w1 -O1 \
-I. -I./Include -DPy_BUILD_CORE -o Python/ceval.o Python/ceval.c

I have included the warnings the compiler spits out (but it spits out similar warnings everywhere, so I doubt that these are the problem) and attached the "optimization report".  I tried turning off as much as possible (inlining, loop unrolling etc.) but could still not get it to work.  There are some threads about loop unrolling issues with icc, but these were for -O3 optimizations:

http://softwarecommunity.intel.com/isn/community/en-us/forums/thread/321812.aspx

If I compile with -O1 and -Ob1 then it breaks and I get the optimization report attached.  The wierd thing is that if I use -O1 and -finline, I get an empty optimization report and it works.  TFM states that "-finline [is the] Same as -Ob1."  This at least is a compiler problem!  I suspect that -Ob1 is doing more than just inlining (it looks that way from the report) so I am not exactly sure where the problem is here yet.
 
icc -pthread -c -fno-strict-aliasing -DNDEBUG -g -Wall -O1\
 -Ob1 -opt-report -opt-report-level=max -opt-report-file=opt.rep.txt\
  -I. -I./Include   -DPy_BUILD_CORE -o Python/ceval.o Python/ceval.c

Python/ceval.c(198): remark #869: parameter "self" was never referenced
  PyEval_GetCallStats(PyObject *self)
                                ^
Python/ceval.c(2710): remark #981: operands are evaluated in unspecified order
                                            PyString_AsString(keyword));
                                            ^
Python/ceval.c(2706): remark #981: operands are evaluated in unspecified order
                                        PyErr_Format(PyExc_TypeError,
                                        ^
Python/ceval.c(2722): remark #981: operands are evaluated in unspecified order
                                             PyString_AsString(keyword));
                                             ^
Python/ceval.c(2717): remark #981: operands are evaluated in unspecified order
                                        PyErr_Format(PyExc_TypeError,
                                        ^
Python/ceval.c(3690): remark #981: operands are evaluated in unspecified order
                                     PyEval_GetFuncDesc(func),
                                     ^
Python/ceval.c(3686): remark #981: operands are evaluated in unspecified order
                          PyErr_Format(PyExc_TypeError,
                          ^
Python/ceval.c(3800): remark #981: operands are evaluated in unspecified order
                                     PyEval_GetFuncDesc(func));
                                     ^
Python/ceval.c(3796): remark #981: operands are evaluated in unspecified order
                        PyErr_Format(PyExc_TypeError,
                        ^
Python/ceval.c(3815): remark #981: operands are evaluated in unspecified order
                                                     PyEval_GetFuncDesc(func));
                                                     ^
Python/ceval.c(3811): remark #981: operands are evaluated in unspecified order
                                        PyErr_Format(PyExc_TypeError,
                                        ^
File Added: opt.rep.txt

This appears to have been a bug with the intel compilers.  With the latest 
version 10.1 20070913 everything seems to work.

Ok. Closing it as third-party.