Issue451547
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2001-08-16 13:33 by gmcm, last changed 2022-04-10 16:04 by admin. This issue is now closed.
Messages (12) | |||
---|---|---|---|
msg5956 - (view) | Author: Gordon B. McMillan (gmcm) | Date: 2001-08-16 13:33 | |
pickle and cPickle will happily dump a lambda, but on load, both report: SystemError: Failed to import class <lambda> from module __main__ Seen on Py 2.1 & 1.5.2 >>> f = lambda x: x in (1,2,3) >>> o = cPickle.dumps(f) >>> f2 = cPickle.loads(o) Traceback (most recent call last): File "<stdin>", line 1, in ? SystemError: Failed to import class <lambda> from module __main__ >>> o = pickle.dumps(f) >>> f2 = pickle.loads(o) Traceback (most recent call last): File "<stdin>", line 1, in ? File "c:\python21\lib\pickle.py", line 951, in loads return Unpickler(file).load() File "c:\python21\lib\pickle.py", line 567, in load dispatch[key](self) File "c:\python21\lib\pickle.py", line 780, in load_global klass = self.find_class(module, name) File "c:\python21\lib\pickle.py", line 790, in find_class raise SystemError, \ SystemError: Failed to import class <lambda> from module __main__ |
|||
msg5957 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-08-16 15:11 | |
Logged In: YES user_id=6380 I think I'll have to close this with a won't fix, or "then don't do that" resolution. The problem is that whenever a function or class is pickled, pickle must accept on blind faith that it can also be unpickled. How would you check that this is indeed the case? |
|||
msg5958 - (view) | Author: Gordon B. McMillan (gmcm) | Date: 2001-08-16 16:03 | |
Logged In: YES user_id=4923 Hmm. In the (simplistic) case I was trying, I can't see any significant difference between the lambda and the equivalent function (names differ, and the func has 2 appended & apparently unreachable bytecodes, but otherwise the func_* and func_code.co_* attributes match). So what am I missing? Lowering priority - I can live without it easily enough. But if lambda's won't load, they probably shouldn't dump. |
|||
msg5959 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-08-16 16:31 | |
Logged In: YES user_id=6380 I guess what you're missing is that pickling a function doesn't pickle the bytecode! It pickles the name instead. So a reference to "foo.bar" is pickled as "foo.bar", and the unpickler imports bar from foo. |
|||
msg5960 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-08-17 18:51 | |
Logged In: YES user_id=6380 I figured it out. In save_global(), I now try to import the module and then extract the name from the module; this must give us the same object. If it doesn't, raise PicklingError. Checked in as pickle.py rev 1.50. I can't close this yet because cPickle needs a similar patch. |
|||
msg5961 - (view) | Author: Gordon B. McMillan (gmcm) | Date: 2001-08-17 21:45 | |
Logged In: YES user_id=4923 Patch # 452239 is my attempt to patch cPickle.c to match Guido's patch to pickle.py. |
|||
msg5962 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-08-18 21:23 | |
Logged In: YES user_id=6380 I applied Gordon's patch, so this can be closed now. |
|||
msg5963 - (view) | Author: Dave Cole (davecole) | Date: 2002-10-25 13:44 | |
Logged In: YES user_id=28658 I seem to be suffering from some unintentional consequences of this fix. I do not think that the pickle should fail in the follwing case. Wouldn't it be a better idea to just check for a successful of the pickled class instead of requiring the imported class to be stored at the same memory location? >>> import pickle, copy >>> o = copy._EmptyClass() >>> reload(copy) <module 'copy' from '/usr/lib/python2.2/copy.pyc'> >>> pickle.dumps(o, 1) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.2/pickle.py", line 978, in dumps Pickler(file, bin).dump(object) File "/usr/lib/python2.2/pickle.py", line 115, in dump self.save(object) File "/usr/lib/python2.2/pickle.py", line 225, in save f(self, object) File "/usr/lib/python2.2/pickle.py", line 477, in save_inst save(cls) File "/usr/lib/python2.2/pickle.py", line 225, in save f(self, object) File "/usr/lib/python2.2/pickle.py", line 524, in save_global raise PicklingError( pickle.PicklingError: Can't pickle <class copy._EmptyClass at 0x819493c>: it's not the same object as copy._EmptyClass |
|||
msg5964 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2002-10-28 13:26 | |
Logged In: YES user_id=6380 Reopening so I won't forget about this. |
|||
msg5965 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2002-10-29 22:21 | |
Logged In: YES user_id=6380 Well, the problem is that there are lots of other cases that the fix also catches. For example: >>> class C: class C: pass >>> a = C.C() >>> pickle.dumps(a,1) I tend to think that catching this is more important than handling reload() -- since reload() has lots of other problems like this, it's better to blame reload() and not try to fix it at the cost of other situations. |
|||
msg5966 - (view) | Author: Dave Cole (davecole) | Date: 2002-10-29 23:52 | |
Logged In: YES user_id=28658 I realise that there are lots of things that you can do to prevent or confuse introspection working in the pickle module. You can dynamically create classes on the fly too. It just seems a shame that it is possible to prevent the most basic of class definitions from working in the pickler. I can program around the problem but it took the better part of a day to work out why I was experiencing the problem. Maybe the correct bug fix is not to change code but to improve the documentation of the pickle module. |
|||
msg5967 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2002-10-30 23:47 | |
Logged In: YES user_id=6380 The problem really lies with reload(); pickle is just one of the many victims. E.g. this fails too: import foo x = foo.C() assert isinstance(x, foo.C) # OK reload(foo) assert isinstance(x, foo.C) # Fails If you have a suggestion for how to warn about reload(), please submit a *new* bug report with Category set to Documentation. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-10 16:04:19 | admin | set | github: 34974 |
2001-08-16 13:33:45 | gmcm | create |