This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Unloading extension modules not always safe
Type: Stage:
Components: macOS Versions: Python 2.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jackjansen Nosy List: bob.ippolito, jackjansen, ronaldoussoren
Priority: normal Keywords:

Created on 2003-11-07 20:44 by bob.ippolito, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (9)
msg18936 - (view) Author: Bob Ippolito (bob.ippolito) * (Python committer) Date: 2003-11-07 20:44
I will look into the solution for this, but the "for now" 
solution would be to never try and unlink bundles because 
they may contain ObjC code.  For several reasons, ObjC 
classes can never be unloaded, so bundles containing ObjC 
code can also never be unloaded.  This is more than a 
problem for just PyObjC, because any arbitrary module may 
contain some ObjC code.  We need to detect this before 
running NSUnlinkModule.  I'll try and put together a patch 
sometime soon if nobody else does, but for now see: http://
sourceforge.net/tracker/?
func=detail&aid=832025&group_id=14534&atid=114534
msg18937 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-11-11 10:51
Logged In: YES 
user_id=45365

Bob, I'm confused. As far as I know Python never unloads 
extension modules...
msg18938 - (view) Author: Bob Ippolito (bob.ippolito) * (Python committer) Date: 2003-11-11 14:20
Logged In: YES 
user_id=139309

it does if you del sys.modules['somemodule'] and somemodule's 
reference count goes to zero.
msg18939 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-11-11 15:16
Logged In: YES 
user_id=45365

I'm surprised that it does the unload:-)

I think the correct solution would be for the module itself (or 
someone close to it) to stash away a reference. As this is only a 
problem for some modules (those containing ObjC code) I don't 
think a general change is in order.

The real problem is that the "last reference" as Python sees it isn't 
really the last reference: the ObjC runtime also has references to 
stuff in there.
msg18940 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-11-14 10:40
Logged In: YES 
user_id=45365

I think that either the module itself, or the package responsible for 
the module, should forestall unloading (by tucking away an extra 
reference somewhere).

The root of the problem is that the Python refcount doesn't reflect 
the all references to the module: ObjC keeps references too.

OTOH: if all modules containing ObjC code have this problem, and 
it is easy to detect whether a module contains ObjC code then 
adding a safety net to the import code might be a prudent course 
of action.
msg18941 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2003-11-25 14:18
Logged In: YES 
user_id=580910

Note that the problem doesn't really have anything to do with 
unloading modules, the problem occurs when trying to unload a 
dylib that doesn't contain the proper init function. See bug 
#848907 (sorry about that one, I didn't look for earlier bug 
reports).

What happens:
- pydoc calls imp.load_module('__temp__', open('_objc.so'), ...)
- load_module loads _objc.so
- load_module tries to find init__temp__ in _objc.so
- nothing is found and therefore load_module calls NSUnlinkModule
- NSUnlinkModule aborts the process because _objc.so contains 
Objective-C definitions

No code in the extension module has any change to fix things 
because the init function is never called.

BTW. The problem also exists in a vanilla Python 2.3 installation on 
Panther, I cleared /Library/Python2.3 and ran 'pydoc -k hello' and 
this also crashed.
msg18942 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2003-11-25 14:36
Logged In: YES 
user_id=580910

A quick workaround would be to let pydoc check the type of 
module it tries to load, it currently checks if the file is a binary 
module by checking "if info and 'b' in info[2]" (pydoc.py:171). If it 
would check if the filetype is not C_EXTENSION (3) the problem 
would not arise with pydoc (which probably accounts for >99% of 
the problematic load_module calls).
msg18943 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2003-12-05 09:41
Logged In: YES 
user_id=580910

Another, probably more correct, workaround is to stop alling 
NSUnlinkModule. 

This function is only called when an invalid dynamic object is 
loaded, which should not occur during normal operation. The only 
negative effect of not unloading seems to be that the process 
image is larger than really necessary.
msg18944 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2004-07-15 22:29
Logged In: YES 
user_id=45365

Fixed in dynload_next.c 2.15.
History
Date User Action Args
2022-04-11 14:56:01adminsetgithub: 39520
2003-11-07 20:44:30bob.ippolitocreate