This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: dumbdbm __del__ bug
Type: Stage:
Components: Library (Lib) Versions: Python 2.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: tim.peters Nosy List: brett.cannon, janeaustine50, juneaftn, nnorwitz, rhettinger, tim.peters
Priority: normal Keywords:

Created on 2003-03-13 06:27 by janeaustine50, last changed 2022-04-10 16:07 by admin. This issue is now closed.

Messages (10)
msg15107 - (view) Author: Jane Austine (janeaustine50) Date: 2003-03-13 06:27
I used shelve.py and it falls back on dumbdbm when no
possible alternatives are found on the system.

I found this error, which is recurrent and deterministic:

Exception exceptions.AttributeError: "'NoneType' object 
has no
attribute 'error'" in <bound method _Database.__del__ of
<dumbdbm._Database instance at 0x820c71c>> ignored

The problem seems to reside in the __del__ of 
dumbdbm._Database:

class _Database:
...
    def __del__(self):
        if self._index is not None:
            self._commit()
...
    def _commit(self):
        try: _os.unlink(self._bakfile)
        except _os.error: pass
        try: _os.rename(self._dirfile, self._bakfile)
        except _os.error: pass
        f = _open(self._dirfile, 'w', self._mode)
        for key, (pos, siz) in self._index.items():
            f.write("%s, (%s, %s)\n" % (`key`, `pos`, `siz`))
        f.close()

My investigation showed that the error was from 
_commit. When
it was called, _os or _open was both None. And the 
exception
catch didn't work quite safely cause its in the "except" 
clause.

The reason I suspect is when the time that 
_Database.__del__ was
called the os module(which is imported as _os) is 
already removed out.

I changed the code as:

    def _commit(self):
        global _os
        if _os is None:
            import os as _os
            import __builtin__
            _open = __builtin__.open
        try: _os.unlink(self._bakfile)
        except _os.error: pass
        try: _os.rename(self._dirfile, self._bakfile)
        except _os.error: pass
        ......

Now it works without any problems, AFAIK.

msg15108 - (view) Author: June Kim (juneaftn) Date: 2003-03-13 17:02
Logged In: YES 
user_id=116941

see the thread at http://groups.google.com/groups?
selm=ba1e306f.0303111337.72a696c7%
40posting.google.com , esp. by my name.
msg15109 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2003-03-19 01:06
Logged In: YES 
user_id=33168

Can you provide a test case which triggers this problem?  I
can't see how _os becomes None.  Also I would like to add a
test.  Thanks.
msg15110 - (view) Author: Jane Austine (janeaustine50) Date: 2003-03-26 03:55
Logged In: YES 
user_id=732903

A test case that triggers this problem:

#foobar.py
def open(filename, flag='c'):
    import dumbdbm
    return dumbdbm.open(filename,flag)

c=open('test.dbm')

#main.py
import foobar

$ python main.py
>>> 
Exception exceptions.TypeError: "'NoneType' object is not 
callable" in <bound method _Database.__del__ of 
<dumbdbm._Database instance at 0x401e482c>> ignored
msg15111 - (view) Author: Jane Austine (janeaustine50) Date: 2003-03-26 04:04
Logged In: YES 
user_id=732903

Run the main.py in Python 2.3+ putting the two files in the 
same place.

For Python 2.2+, put the two files in the same package and 
make a new file that imports main.py and run it as follows:

=======
from <packagename> import main
=======
msg15112 - (view) Author: Jane Austine (janeaustine50) Date: 2003-03-26 04:08
Logged In: YES 
user_id=732903

Tested on linux and windows xp with Python2.2.2 and 
Python2.3a2. 
msg15113 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2003-03-26 04:09
Logged In: YES 
user_id=31435

Neal, this is actually a common problem in __del__ 
methods, and the OP's analysis is on target.  When Python 
is shutting down, it tries to tear down module dicts in a 
safe-as-possible order, but modules are full of reference 
cycles and there is no *wholly* safe order.

In general, a __del__ method should never reference 
globals because of this.  The usual solution can be seen in 
tempfile.py:  store the global objects __del__ will need as 
class attributes when the class is created, and refer to 
these bindings in __del__ via self.attrname 
(ClassName.attrname is also no good, because it refers to 
the global ClassName!  that may also become None).

Reimporting a module instead may not be effective (if it's in 
a partially torn-doiwn state but its name is still a key in 
sys.modules, importing again will just retrieve the partially 
torn-down module object).  The class-attr trick is robust.
msg15114 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-06-08 21:45
Logged In: YES 
user_id=357491

OK, so according to Tim all the accessing of _os should be 
removed and instead call instance or class attributes that are 
storing what __del__ is going to need when it is cleaning up by 
calling them off of self since the namespace might be in the 
process of being cleared.  Right?  So 
``_os.unlink(self._bakfile)`` should be something more like 
``self._delattrs['unlink'](self._bakfile)`` where self._delattrs 
stores everything __del__ is going to need.  Or should it just 
store a reference to the os module at self._os since that might be 
a little cleaner looking?
msg15115 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2003-07-14 08:27
Logged In: YES 
user_id=80475

Tim, is this what you just fixed?
msg15116 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2003-07-14 15:15
Logged In: YES 
user_id=31435

Raymond, yes, and thanks for the reminder.  This is fixed in 
current CVS, via the approach I sketched in an comment on 
this report.
History
Date User Action Args
2022-04-10 16:07:38adminsetgithub: 38156
2003-03-13 06:27:19janeaustine50create