The close() and GC interaction of generators that use yield inside of finally 
blocks doesn't execute correctly when nested. See the attached example.

More information about the issue is in the Mozilla bug tracker (they found 
a similar bug in their implementation for JS 1.7):
https://bugzilla.mozilla.org/show_bug.cgi?id=349012

Logged In: YES 
user_id=6380

I lost my patience halfway through reading the Mozilla bug
tracker, but IMO this works as designed.  The philosophical
question is, would you rather see the outer finally clause
reached in your example, or would you rather see the
following generator terminated upon close? (If you "fix" the
former, the latter ends up in an infinite loop when you
attempt to close() or GC it.)

def gen():
  while True:
    try:
      yield
    except:
      pass

Logged In: YES 
user_id=139309

Uh no, that wouldn't reach an infinite loop because any attempt to yield during 
close will raise RuntimeError and terminate the loop.

The problem is that finally doesn't execute. Finally clauses always must execute. 
If they don't, then they're worthless.

The real strange issue is that if close is called, then GC makes a second attempt, 
and it *does* execute the outer finally clause. There are definitely bugs here.

Logged In: YES 
user_id=56214

Bob, the problem Guido is pointing out is that to run the
finally clauses, we have to resume the generator with the
RuntimeError thus generated.  So it doesn't terminate the
loop, because the RuntimeError is effectively raised at the
point where the yield occurs.  So, in order to run finally
clauses sanely after a close() attempt, we would have to
prevent such loops.  

That doesn't mean Guido's example is valid as such; but I
think it's possible to accidentally create quasi-indefinite
loops using infinite iterators written prior to Python 2.5,
if you had a try/except block that was expecting to catch
something *other* than GeneratorExit or RuntimeError.  So,
the net effect would be an unintended hang, if we tried to
support retrying until the generator can be closed.

Regarding the GC second attempt, this behavior is actually
exactly as-specified by the PEP's pseudo-code explanation of
how close() is handled.  A RuntimeError raised from close()
does *not* close the generator; it specifically indicates
that the generator has not in fact closed.

At this point, after re-reading the PEP and looking at what
the code is doing, it's clear that the behavior is "as
specified", so the title of the bug is incorrect: the
behavior is exactly as specified by PEP 342.  So, the rest
of my comments below are regarding whether PEP 342 should be
changed.

And really, the question there is whether it's sane to
attempt heroic measures in order to run finally clauses in a
generator whose code is *known* to be buggy.  The
RuntimeError basically says, "this generator is broken", and
the GC also tries a second time to close it.  If two close
attempts don't close it, it's a dead duck.

What other measures can we sanely add?  We could try forcing
the RuntimeError into the generator itself, but then we have
to deal with the infinite loop possibility, and the oddities
involved in getting to a language specification that can be
implemented for Jython, IronPython, etc.  On the whole, I
think that we probably reached the right balance the first
time around: a broken generator should not be allowed to
handle the error of its own brokenness.

Logged In: YES 
user_id=139309

It seems that the infinite loop would be the right thing to do, given that's what 
would happen in a similarly broken __del__ (or anywhere else).

Logged In: YES 
user_id=56214

You'll need to propose a patch to PEP 342 to alter the
specification, then, and get it past Python-Dev. 
Personally, I don't think that changing the spec is the
right thing to do for 2.5.

But irrespective of those procedural matters, your __del__
analogy is flawed on two points.  First, we do not re-run a
__del__ method to handle an error that was raised by that
__del__ method!  Second, if a generator contains an
infinite, non-yielding loop, then of course it will loop
forever.  So __del__ does not provide any actually useful
guidance here.

Is it useful to keep this bug open? Does it correspond to a real-world
use case?

Just in case PEP 342 ever changes we might as well target the latest versions.

As an aside I followed the PEP 342 link to here http://legacy.python.org/dev/peps/pep-0342/ and clicked on the "last modified" field.  I was taken here http://hg.python.org/peps/file/tip/pep-0342.txt.  I was rather surprised to see this

view pep-0342.txt @ 5421:6ed8f836c072

PEP 466: finish incomplete sentence
author 	Nick Coghlan <ncoghlan@gmail.com>
date 	Sun, 23 Mar 2014 07:17:32 +1000 (3 hours ago)
parents 	c25fa0913267

Could somebody take a look at this please.