new.code() does not check its arguments properly:

>>> new.code(1,1,1,0,"123",(None,),(5,),(5,),"123","321",2,"")
Fatal Python error: non-string found in code slot

Here the tuple (5,) is used for co_names and co_varnames.

More generally it looks like this part of compile.c could do with quite some cleaning up.

Logged In: YES 
user_id=29957

I thought there was a general concensus that many things in
the 'new' module could be used to make a mess, and that
trying to fix them all was a pretty much open-ended task?

Logged In: YES 
user_id=6656

Maybe new is meant to be scary, but current behaviour is going 
well above and beyond the effort required.  new.code can mutate 
a tuple containing string subclasses into a tuple containing exact 
strings!

Armin, what do you think of the attached?

Logged In: YES 
user_id=4771

Why do you check for argcount<0 but not nlocals<0, which also raises a SystemError?

More to the point, why on earth does PyCode_New() check for negative values of argcount and nlocals and not the other values?  A negative stacksize, for example, is likely to crash the interpreter too.  On the other hand, validating these numbers more precisely is much more complex, and not something we're trying to do here.  So I don't understand why these two values in particular are checked for < 0.  We should as well drop the check altogether.

My position here is that calling new.code() shouldn't do anything bad to the interpreter; but actually executing the resulting code object is allowed to destroy the sun (provided, of course, that CPython runs in an environment that allows that).

Logged In: YES 
user_id=6656

> Why do you check for argcount<0 but not nlocals<0, which
> also raises a SystemError?

Because I didn't read that far...

Obvious extension of patch added.

Agree that current behaviour of PyCode_New is a bit crazy.  
OTOH, not changing PyCode_New itself seems a principle no worse 
than any other for the moment.

Logged In: YES 
user_id=4771

Sounds reasonable.  The patch looks OK.

On a side note, PyCode_New() calls optimize_code(), but
PyCode_New() is also used when un-marshalling .pyc files.  I
wonder how much time is lost re-optimizing in vain the code
loaded from .pyc files, and I wonder why optimization is not
done just as the last step of the compiler instead.  While
I'm wondering, I also wonder if some user bytecode hacks
could be broken by optimize_code().

Logged In: YES 
user_id=6656

OK, checked in as

Lib/test/test_new.py revision 1.18
Python/compile.c revision 2.314

I also rewrote all the tests of new.code() to use realistic 
arguments.

> On a side note, PyCode_New() calls optimize_code(), but
> PyCode_New() is also used when un-marshalling .pyc files.

Yes, I noticed that too, and wondered some of the things you 
wondered.  Ask Raymond?  I think it's his code.

Logged In: YES 
user_id=4771

Raymond said: not worth the burden.  The optimizer is "experimental and not fully developed".  It is also superfast, so it doesn't matter.  The AST branch will change all this anyway.

Logged In: YES 
user_id=80475

For the record, the thought is to leave the optimization
code fully decoupled from the rest of the compiler until the
AST branch goes in.  Also, the optimizer itself needs a bit
more evolution in Py2.5 before it gets moved. For the time
being, it adds more than it costs and should be left alone.
 But, ultimately it should be placed at the tail end of
compilation, before the pyc files are created.