I noticed a test_sqlite test failure on my Mac G5 the other day while trying 
to set up a buildbot for sqlalchemy.  Everything runs fine on my G4 
powerbook.  Both machines run Mac OSX 10.4.8 with Apple's gcc 4.0.0, 
build 5026.

I whittled the problem down to just having test_ctypes and test_sqlite 
enabled, then further whittled it down to just having Lib/ctypes/test/
test_find.py available (all other ctypes tests eliminated).  More detailed 
problem descriptions are in these two postings to python-dev:

    http://article.gmane.org/gmane.comp.python.devel/84478
    http://article.gmane.org/gmane.comp.python.devel/84481

Skip

Logged In: YES 
user_id=44345

Another article here:

    http://article.gmane.org/gmane.comp.python.devel/84487

Thomas, I assigned this to you simply so you can cast your eye on the problem and see if you think there might be anything ctypes-related in the problem I've reported.  I've ignored it up 'til now, but I bumped up against it again trying to get the Pybots SQLAlchemy test suite running on my Mac.  During my previous encounter with this problem Ronald Oussouren reported on a possible connection with GLUT:

    http://mail.python.org/pipermail/python-dev/2006-October/069523.html

Assuming there is no obvious ctypes connection you can determine  (it's not clear to me that there is any current directory funny business going on) just assign it back to me and I'll try to find a pysqlite expert to take a look.

Thx,

Skip

Skip,

I do not have a G5 machine available, only a G4 and the problem doesn't appear there.
For someone with a G5 it should not be too difficult to reproduce (or investigate),
I think that running './python Lib/test/regrtest.py test_ctypes test_sqlite' should suffice.

However, if loading the GLUT library changes the cwd, and sqlite fails with a changed working dir,
I'm not sure where the problem really is and how it should be fixed.  It could be said that test_ctypes
should restore the cwd after the tests (but maybe this should be done by the test framework?),
or it could be said that test_sqlite should set the cwd in case the test relies on it.

I'll assign back to you because I have no way to test it on a machine where the problem appears.

FYI, this PyOpenGL bug may be related:

https://sourceforge.net/tracker/?func=detail&atid=105988&aid=1712741&group_id=5988

FWIW, I'm getting an identical error on a MacBook Pro with a Core Duo processor

I've just noticed the same failure on an iBook G4, with OS X 10.4.10, sqlite version 3.4.0, 
and Python built with gcc 4.2.0 (from macports).

(Python 2.6a0 (trunk:56123M, Jun 29 2007, 13:22:18) 
[GCC 4.2.0] on darwin)

And also on Python 2.5, compiled under Apple's GCC. (Still OS X 10.4.10/G4)
The following four lines of Python code are enough to cause the problem for me.

Interestingly, if the second and third lines are exchanged (so that the two import commands come first), then no exception is raised.

Python 2.5.1 (r251:54863, May 10 2007, 20:59:25) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import CDLL
>>> CDLL('/System/Library/Frameworks/GLUT.framework/GLUT').glutWireTetrahedron
<_FuncPtr object at 0x514500>
>>> import sqlite3
>>> sqlite3.connect(":memory:")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.ProgrammingError: library routine called out of sequence

Fails for me as well (Python 2.6a0, PowerBook G4, 10.4.10, Apple's GCC 4.0.1).  Thomas, can you reproduce with the lines marketdickinson used?

I don't see any directory changing going on:

% python
Python 2.6a0 (trunk:54264M, Mar 10 2007, 15:19:48) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getcwd()
'/Users/skip'
>>> from ctypes import CDLL>>> CDLL('/System/Library/Frameworks/GLUT.framework/GLUT').glutWireTetrahedron<_FuncPtr object at 0x106c6c0>
>>> os.getcwd()
'/Users/skip'
>>> import sqlite3
>>> sqlite3.connect(":memory:")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ProgrammingError: library routine called out of sequence
>>> os.getcwd()
'/Users/skip'

Pass it back if you still can't reproduce the problem.  Meanwhile I've subscribed to the pysqlite mailing list and will ask there.

Skip

One other minor(?) data point.  I am using a version of sqlite3 which I compiled and installed, not Apple's older version:

>>> sqlite3.version, sqlite3.sqlite_version
('2.3.3', '3.3.8')
...
% pwd
/Users/skip/local/lib/python2.6/lib-dynload
% otool -L _sqlite3.so 
_sqlite3.so:
        /Users/skip/local/lib/libsqlite3.0.dylib (compatibility version 9.0.0, current version 9.6.0)
        /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 93.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.7)

It appears those of us who can reproduce the problem all have slightly different installs (2.5 v. 2.6, Apple's GCC v. DarwinPorts, Apple's sqlite3 v. DarwinPorts v. homebuilt, G4 v. G5 v. Intel architecture).

I think your minor(?) data point is a major data point.  I'm using the Macports version of sqlite3.  Furthermore, after hiding the /usr/lib version with

fibonacci:/usr/lib dickinsm$ sudo mv libsqlite3.0.dylib libsqlite3.0.dylib.temp

I can't reproduce the problem any more.  I tried sprinkling a few printf statements through the sqlite3 source to see what was going on.  Bizarrely, it looks as though the Apple-supplied sqlite3SafetyCheck() routine is being called from the Macports createCollation(); or at least that's the only way that I can understand the results I'm getting.  Is this possible?

Mark

It looks as though GLUT is a red herring.  I can reproduce the problem with a variety of frameworks in place of GLUT, for example Quartz, Cocoa, AppKit, CoreData, as in the following lines.

import ctypes
ctypes.CDLL('/System/Library/Frameworks/CoreData.framework/CoreData')
import sqlite3
sqlite3.connect(":memory:")

Furthermore, I notice that the GLUT framework has a library dependency on the AppKit framework,  AppKit on CoreData, and CoreData depends on the Apple-supplied libsqlite3, and that Quartz and Cocoa similarly ultimately depend on libsqlite3.  Coincidence?

I can reproduce the problem with a variety of frameworks in place of GLUT, for example Quartz, Cocoa, AppKit, CoreData.  (The .glutWireTetrahedron is unnecessary.)

I'll be damned...  Based on your last response I ran Python
under ktrace then looked at the kdump output.  As it's
marching along it sees a mention of /usr/lib/libsqlite3.so
in /System/Library/Frameworks/CoreData.framework/Versions/A/CoreData
then just a few lines later I see

 20023 python   NAMI  "/usr/lib/libsqlite3.0.dylib"
 20023 python   RET   open 3
 20023 python   CALL  fstat(0x3,0xbfffc770)
 20023 python   RET   fstat 0
 20023 python   CALL  pread(0x3,0xbfffcbd0,0x1000,0)

then a bit later I see what I had wanted to see:

 20023 python   NAMI  "/Users/skip/local/lib/python2.6/lib-dynload/_sqlite3.so"
 20023 python   RET   open 6
 20023 python   CALL  fstat(0x6,0xbfff9c00)
 20023 python   RET   fstat 0
 20023 python   CALL  pread(0x6,0xbfffa060,0x1000,0)
 ...
 20023 python   NAMI  "/Users/skip/local/lib/libsqlite3.0.dylib"
 20023 python   RET   open 6
 20023 python   CALL  fstat(0x6,0xbfff99b0)
 20023 python   RET   fstat 0
 20023 python   CALL  pread(0x6,0xbfff9e10,0x1000,0)

I can't execute the CDLL call if I've renamed /usr/lib/libsqlite.3.0.dylib.  (I suppose that stands to
reason.)

Now the question becomes whether or not the Apple-provided
library can be replaced with something more modern.
Presumably every application that uses CoreData (most
Apple things) gets it if during startup.

Thomas, for the purposes of the Python regression test
suite do you think you can find something other than
GLUT to dynamically load?

Mark, thanks for the sleuthing.

Skip

I submitted a bug report to Apple about the problem, though I don't
really expect a response.  (The other bug I filed in 2004 has never
received any response as far as I can tell.)  Maybe a more recent
version of SQLite will be delivered with Leopard...

Thomas,

Here's a possible modification to ctypes.test.test_find which appears
to be the only place that OpenGL (and thus CoreData on the Mac?) is
accessed.  On Darwin I replace the GL find stuff with Tcl find stuff.
It's a bit ugly, but it might solve the problem.

Skip

File Added: test_find.diff

Skip, thanks for the analysis and test patch.

IMO loading the GLUT library does not add significant value to the ctypes tests, so wouldn't it be best to skip the GLUT lib on OSX?  See the attached patch "test_find without GLUT on OSX" (note that I didn't actually test the patch, and I don't have access to OSX G5 anyway).

Maybe the problem and the analysis should be reported to the PyOpenGL project...
File Added: test_find-no-osx-glut.diff

Certainly the patch is simpler. ;-)  If loading GLUT doesn't add any value on OSX, would it add any value on other platforms?  Do other tests have to find libraries successfully to pass?  If so, maybe test_find should be dispensed with altogether, however if test_find has value we should figure out a library to load on OSX.

This certainly sounds like someone is fooling around with the -force_flat_namespace link flag, which could result in picking up the wrong version of symbols. I haven't checked yet if this is indeed the case here, and if so who is doing this.

Skip,  IMO loading GLUT does not add any value (I just added it because GLUT is related to the OpenGL libraries, and because it worked [That's what I thought at least]).  Finding and loading the other libs adds value because they must be loaded with RTLD_GLOBAL on some platforms, so I would like to keep that part.

So, I suggest to remove the GLUT part from the tests altogether.

Go for it then.  You are, after all, the expert. <wink> I'm crossing
my fingers that this will finally solve the community buildbot
failures I *always* get on mac os x when trying to test out
sqlalchemy.

Skip

Implemented in SVN rev. 56350.

I'm reopening this.  I am seeing the same behavior now on my
MacBook Pro running Mac OS X 10.5.1.  Looking at the crash report
in my ~/Library/Logs/CrashReporter directory I see both /usr/lib
and /opt/local/lib versions of libsqlite3:

 0x1188000 -  0x11defef +libsqlite3.0.dylib ??? (???)
/opt/local/lib/libsqlite3.0.dylib
0x911f6000 - 0x9127dff7  libsqlite3.0.dylib ??? (???)
<273efcb717e89c21207c851d7d33fda4> /usr/lib/libsqlite3.0.dylib

I was able to work around this by disabling the MacPorts version of
sqlite3.  Others might not be so lucky.  I don't see any references
to sqlite in the ctypes test code.  Must be an indirect reference.

Skip

Skip, I had the same problem as you, with the same solution. On a MacBook 
Pro running Mac OS X 10.5.2, running test_ctypes and test_sqlite together 
would kill my machine. Found another installation of sqlite3 on 
/usr/local, removed it and now the test works fine.

So, should this be closed as some kind of 'works for me' + 'wont fix' +
 'third-party-bug' in Apple's sqlite?

Probably.

I do remember investigating this further, but I don't entirely remember 
what I found.  I think Ronald's comment about flat namespaces turned out 
to be highly relevant though.

Refer to msg87277 and msg87278.