Pickle falls into infininite recursion, cPickle crashed
(segfault on Linux, bus error on FreeBSD). See the
attached file. There is initial pickle (just a very
simple though somewhat large tree of bookmarks;
unfortunately lesser tree does not trigger the bug);
bkmk_objects.py module that includes class definitions
for the pickled object, and simple test program that
demonstrates the bug.

The program loads initial pickle, runs a function upon
it and saves the pickle back. The function runs through
the entire tree and creates backlinks - every object
got a link to its parent folder.

So it seems the bug is related to pickling objects with
loops.

When I changed builtin list as a parent class of Folder
to UserList (and recreated initial pickle) the bug
disappeared, so I think the bug is related to pickling
new classes.

Logged In: YES 
user_id=6380

Mmm, interesting.

Logged In: YES 
user_id=6380

Oleg, could it be that your data structure is just very
deeply nested? On my machine (RedHat Linux 7.2), your test
program completes without problem, so there doesn't seem to
be something in the data that inherently triggers a bug.
However, when I apply Martin von Loewis's patch from SF bug
576084, cPickle raises RuntimeError: maximum recursion depth
exceeded. I'm guessing that you are building what is in
essence a very long doubly-linked list; the pickle algorithm
necessarily has to enter a recursion level for each list
node in this case.

Switching to UserList probably reduces the number of stack
frames per recursion level and that would explain that it
doesn't fail.

Logged In: YES 
user_id=4799

> could it be that your data structure is just very deeply nested?
[skip]
> I'm guessing that you are building what is in essence a very long doubly-linked list

I thought about it. It is not vrey deeply nested, just very long. So what should I do? Devise a formula that calculates recusion limit depending on the number of objects and call sys.setrecursionlimit()? Or just call sys.setrecursionlimit() in a loop, catching RuntimeError and increasing recusrion limit?

In any case I hope to see cPickle patched so it will not crash with a reasonable recursion limit.

Logged In: YES 
user_id=6380

Well, from the POV of pickle, a long chain is
indistinguishable from a deeply nested structure. cPickle
*will* be fixed (by the patch of bug 576084). So I'm going
to close this as a duplicate.