Issue793826
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2003-08-23 17:24 by arigo, last changed 2022-04-10 16:10 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
iziptest2.py | arigo, 2003-08-23 17:28 | |||
iziptest4.py | arigo, 2003-08-28 13:13 |
Messages (12) | |||
---|---|---|---|
msg17910 - (view) | Author: Armin Rigo (arigo) * | Date: 2003-08-23 17:24 | |
Not sure how crucial this exactly is, but the itertools.izip() can be abused to create a tuple that can be mutated once. I couldn't find a way to crash anything, however, but I wouldn't leave such a possibility open in the Python interpreter nevertheless. . from itertools import imap, izip . . def mutatingtuple(tuple1, f, tuple2): . # this builds a tuple t which is a copy of tuple1, . # then calls f(t), then mutates t to be equal to tuple2 . # (needs len(tuple1) == len(tuple2)). . def g(value, first=[1]): . if first: . del first[:] . f(z.next()) . return value . items = list(tuple2) . items[1:1] = list(tuple1) . gen = imap(g, items) . z = izip(*[gen]*len(tuple1)) . z.next() . . def f(t): . global T . T = t . print T . . mutatingtuple((1,2,3), f, (4,5,6)) # print T -> (1, 2, 3) . print T # print T -> (4, 5, 6) |
|||
msg17911 - (view) | Author: Armin Rigo (arigo) * | Date: 2003-08-23 17:28 | |
Logged In: YES user_id=4771 Assigned to Raymond on the rationale that he's the author of itertoolsmodule.c. I can propose a patch to itertoolsmodule.c if you wish. Also attached above example in a file. |
|||
msg17912 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2003-08-23 23:30 | |
Logged In: YES user_id=80475 I'm inclined to not worry about this one. The tuple re-use technique appears elsewhere in the code base and has not had any problems in practice. The exploitation script is impressive and it makes clear that mutation requires combining a number of rare, deliberate steps that don't occur in practice, i.e. the intensional assignment to a tuple inside a self-referential iterator that refers to itself by the name in an enclosing scope. |
|||
msg17913 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2003-08-24 19:17 | |
Logged In: YES user_id=80475 After more thought, I think this should be closed because the attached tuple mutation script only works when the tuple is not being used (there can be no other references to it) and in that situation, it doesn't matter if a deliberate contrived effort is made to mutate. |
|||
msg17914 - (view) | Author: Armin Rigo (arigo) * | Date: 2003-08-26 16:22 | |
Logged In: YES user_id=4771 I agree with your general analysis : it doesn't seem harmful. I don't understand the bit about "only works when the tuple is not being used", however. In the example script the function 'f' stores a copy of the tuple; any number of references can be hold accross the mutation. Here is another example: g = {} def f(t): global T; T=t; g[T] = 'a' mutatingtuple((1,2,3), f, (4,5,6)) # print T -> (1, 2, 3) g[T] = 'b' print g This prints {(4, 5, 6): 'a', (4, 5, 6): 'b'}. |
|||
msg17915 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2003-08-27 06:00 | |
Logged In: YES user_id=80475 The bit about "only working when the tuple is not being used" relates to the dependency on the C code that checks that the refcount is one: d = {(1,2,3):value} # Create a tuple with one reference # now try to mutate it so that "value" becomes inaccessible I'm curious about how the original idea arose and what was the thought process you used to develop the clever, exploitation script. |
|||
msg17916 - (view) | Author: Armin Rigo (arigo) * | Date: 2003-08-28 13:13 | |
Logged In: YES user_id=4771 Ah ! I knew there was a flaw ! Attached is a script (iziptest4.py) that uses all the above plus yet another minor hole to actually decrement the reference counter of an arbitrary object. Have you already seen Python dying on you with 'Fatal Python error: deallocating None' ? :-) The original idea came from lurking at izip_next(), when I was thinking about using gc.get_referrers() to grab the internally stored tuple (see bug report #793822). Then I realized that the recursion trick would be sufficient for izip_next(). Finally, the attached script exploits the common problem of DECREF'ing an object that is still in the tuple *before* actually storing something new at that position, which is only a problem if one can actually see the tuple from the __del__ method... I can make a patch if you wish. |
|||
msg17917 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2003-08-29 20:22 | |
Logged In: YES user_id=80475 Please do submit a patch (attached to this report) and I'll take a look at it. In your judgement, is this a problem that would negatively impact real code or does it only involve deliberate contortions and mischief? |
|||
msg17918 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2003-08-29 23:14 | |
Logged In: YES user_id=80475 Fixed. See: Modules/itertoolsmodule.c 1.20 and 1.18.6.1 Lib/test/test_itertools.py 1.18 and 1.15.6. |
|||
msg17919 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2003-08-30 00:01 | |
Logged In: YES user_id=80475 Does a further change need to be made or is it sound as it stands now? - Py_DECREF(PyTuple_GET_ITEM(result, i)); + olditem = PyTuple_GET_ITEM(result, i); PyTuple_SET_ITEM(result, i, item); + Py_DECREF(olditem); |
|||
msg17920 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2003-08-30 00:17 | |
Logged In: YES user_id=80475 Applied code to replace the tuple entry before decreffing it. |
|||
msg17921 - (view) | Author: Armin Rigo (arigo) * | Date: 2003-08-30 14:06 | |
Logged In: YES user_id=4771 Moving the Py_INCREF(result) at the beginning seems to fix all issues at once and even probably makes the "olditem=... Py_DECREF(olditem)" dance unnecessary, but it looks fine as it is now. I'd be interested in knowing if you actually measured how much of a performance gain it is to cache the resulting tuple, given that PyTuple_New() is already quite optimized. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-10 16:10:49 | admin | set | github: 39118 |
2003-08-23 17:24:52 | arigo | create |