This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Argument genexp corner case
Type: Stage:
Components: Interpreter Core Versions: Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: anthonybaxter, jpe, ncoghlan, nnorwitz
Priority: release blocker Keywords:

Created on 2005-03-21 17:47 by jpe, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (6)
msg24739 - (view) Author: John Ehresman (jpe) * Date: 2005-03-21 17:47
The following raises an unexpected exception; I would
expect it to either work or raise a SyntaxError.  It
seems that the grammar parses it, but the compiler does
not do the right thing.

>>> def foo(a): pass

>>> foo(a = i for i in range(10))

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
NameError: name 'i' is not defined

foo(a = (i for i in range(10)) works.
msg24740 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-10-11 04:10
Logged In: YES 
user_id=33168

I definitely agree this is a big problem.

Here's what the code above generates:
2           0 LOAD_GLOBAL              0 (foo)
              3 LOAD_CONST               1 ('a')
              6 LOAD_GLOBAL              1 (i)
              9 CALL_FUNCTION          256
             12 POP_TOP
             13 LOAD_CONST               0 (None)
             16 RETURN_VALUE

If I put parens around the genexp, I get:
2           0 LOAD_GLOBAL              0 (foo)
              3 LOAD_CONST               1 ('a')
              6 LOAD_CONST               2 (<code object
<generator expression> at 0x2a960baae8, file "<stdin>", line 2>)
              9 MAKE_FUNCTION            0
             12 LOAD_GLOBAL              1 (range)
             15 LOAD_CONST               3 (10)
             18 CALL_FUNCTION            1
             21 GET_ITER
             22 CALL_FUNCTION            1
             25 CALL_FUNCTION          256
             28 POP_TOP
             29 LOAD_CONST               0 (None)
             32 RETURN_VALUE
msg24741 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2005-10-11 13:43
Logged In: YES 
user_id=1038590

The problem is definitely on the parser end though:

Py> compiler.parse("foo(x=i for i in range(10))")
Module(None, Stmt([Discard(CallFunc(Name('foo'),
[Keyword('x', Name('i'))], None, None))]))

It's getting to what looks like a valid keyword argument in
"x=i" and throwing the rest of it away, when it should be
flagging a syntax error (the parser's limited lookahead
should be enough to spot the erroneous 'for' keyword and
bail out).

The problem's actually worse than the OP noted: consider
what will happen if there is a variable "i" visible from the
location of the function call (e.g. from a list
comprehension or for loop in a nested scope). Good luck
tracking that one down. . .
msg24742 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-10-21 06:27
Logged In: YES 
user_id=33168

Checked in as:
 * Python/graminit.c 2.41
 * Lib/test/test_genexps.py 1.10
 * Grammar/Grammar 1.55
 * Misc/NEWS 1.1392 and 1.1391

Leaving it up to Anthony whether this should be backported.
msg24743 - (view) Author: Anthony Baxter (anthonybaxter) (Python triager) Date: 2005-10-21 07:55
Logged In: YES 
user_id=29957

I can't see a problem with backporting this to 2.4
msg24744 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-10-23 00:44
Logged In: YES 
user_id=33168

Ok.  Backported.
History
Date User Action Args
2022-04-11 14:56:10adminsetgithub: 41738
2005-03-21 17:47:59jpecreate