There were some complaints recently on c.l.py about the rather 
non-informative error messages emitted as a result of the tokenizer 
detecting a problem.  In many situations it simply returns 
E_TOKEN which generates a fairly benign, but often unhelpful 
"invalid token" message.

This patch adds several new E_* macrosto Includes/errorcode.h, 
returns them from the appropriate places in Parser/tokenizer.c and 
generates more specific messages in Python/pythonrun.c.  I think the 
error messages are always better, though in some situations they may 
still not be strictly correct.

Assigning to Jeremy since he's the compiler wiz.

Skip

Logged In: YES 
user_id=31392

Is the warning about i vs. j for complex numbers really 
necessary?  It seems like it adds extra, well, complexity 
for a tiny corner case.

Logged In: YES 
user_id=44345

re: i vs. j

Perhaps it's not needed.  The patch was originally designed to address 
the case of runaway triple-quoted strings.  Someone on c.l.py ranted 
about that.  While I was in there, I recalled someone else (perhaps more 
than one person) had berated Python in the past because imaginary 
numbers use 'j' instead of 'i' and decided to stick it in.  It's no big deal to 
take it out.

(When you think about it, they are all corner cases, since most of the 
time the code is syntactically correct. ;-)

S

Logged In: YES 
user_id=31392

The current error message for the complex case seems clear
enough since it identifies exactly the offending character.

>>> 3i+2
  File "<stdin>", line 1
    3i+2
     ^
SyntaxError: invalid syntax

The error message for runaway triple quoted strings is much
more puzzling, since the line of context doesn't have
anything useful on it.

I guess we should think about the others, too:

E_EOLS is marginal, since you do get the line with the error
in the exception.

E_EOFC is a win for the same reason that E_EOFS is, although
I expect it's a less common case.

E_EXP and E_SLASH are borderline -- again because the
current syntax error identifies exactly the line and
character that are causing the problem.

We should get a third opinion, but I'd probably settle for
just E_EOFC and E_EOFS.

Logged In: YES 
user_id=44345

here's a new patch - deletes all but the EOFC & EOFS macros and adds a
test_eof.py test module

Logged In: YES 
user_id=31392

Looks good to me!  Sorry I took so long to review it again.

Logged In: YES 
user_id=44345

implemented in
    Python/pythonrun.c 2.166
    Parser/tokenizer.c 2.64
    Include/errcode.h 2.16