There's an off-by-one error in PyGen_NeedsFinalizing():
the loop starts by looking at
f->f_blockstack[f->f_iblock], but that's one beyond the
largest legitimate f_blockstack index. As a result,
any generator with an active loop is very likely to
claim it needs finalizing (to the extent that it's
unlikely that reading up trash for b_type will match
SETUP_LOOP).
The attached patch repairs that, but also tries to
remove some of the goofy .close() gimmicks added to old
tests in test_generators to worm around "leaks" caused
by that all generators said they needed finalizers in
2.5 at first. (That's how I bumped into this: I tried
removing the .close() gimmicks first, but -R:: runs
said they still leaked -- that started the trail
leading to PyGen_NeedsFinalizing()).
Alas, after applying the patch, test_generators craps
out in a debug build with a NULL-pointer dereference in
_Py_ForgetReference(), in the guts of the `if` test
deciding whether to produce an "UNREF invalid object"
error. At the time, op_type is _PyGen_Type and
op->_ob_next = _ob_prev = NULL, probably meaning that
op was already passed to _Py_ForgetReference() previously.
We get into _Py_ForgetReference() via a chain starting
at the the decref in ceval.c's
while (STACK_LEVEL() > b->b_level) {
v = POP();
Py_XDECREF(v);
and we got into PyEval_EvalFrameEx via gen_send_ex via
gen_close via gen_dealloc via (eventually) a dict on
some instance going away ... and then I got lost.
Looks like there are several more layers of generator
deallocs on the call stack too.
I'm out of time for looking at this now, so recording
it here.
|