Issue1206077
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2005-05-21 09:23 by sxanth, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
compiler-1.patch | sxanth, 2005-05-21 09:23 | patch-testfile-doc |
Messages (8) | |||
---|---|---|---|
msg48372 - (view) | Author: Stelios (sxanth) | Date: 2005-05-21 09:23 | |
The compiler package produces lower quality bytecode since it does not do any of the peephole optimizations. I tried to fix that to make the package produce bytecode which is as good as the internal compiler. Yeah, probably wasted time but because python is so great it was much easier than I thought and now the package produces rather superior bytecode! The patch implements all the optimizations mentioned in compile.c and additionally: - constant folding (or however this is called). For example, it turns x = -(1<<2) to the constant -4. Also nested tuples of constants are converted to one big tuple constant. - The optimization which checks JUMPs that lead to other JUMPs is extended to re-run the optimization when something is modified. For example, the builtin optimizer can't do: if (a and b) and c where there is a JUMP_IF_FALSE which leads to a JUMP_IF_FALSE which leads to a third JUMP_IF_FALSE. While I was at it, I discovered several "bugs" which I couldn't resist attempting to fix. - Implemented LIST_APPEND for list comprehensions - The docstrings of functions where added in the co_consts of the module - All the names that appeared in a function where added to co_varnames. Not sure about the fix - Made co_filename and co_name interns - In generator expressions the outmost-iterable's names where considered freevars and therefore in: def bar(y): return foo (x for x in y) the generator expression was considered a closure I've used the new compiler for my own benchmark suite and all went well. I've also managed to just *compile* the standard library and with the new compiler it turns out 40167 bytes smaller (!). But really, somebody should run the regression tests with it (I can't due to inefficient hardware). IFF it passes the tests if *could* be used to re-compile the standard library for more efficient code. The patch is vs 2.4.1. However, the patch it's not 100% ready to be applied to mainline; but it's easier to review. The maintainer should tell me how to proceed :-) Further Questions: - generator expressions do not emit SETUP_LOOP/POP_BLOCK. Should they? - What is the status of the package? I discovered all those bugs in a couple of hours. Does this mean that there are many many others? How many new ones has my patch added? Is the package not suitable for serious development? |
|||
msg48373 - (view) | Author: Michael Hudson (mwh) | Date: 2005-05-21 12:19 | |
Logged In: YES user_id=6656 Why do you say this patch isn't ready to apply? Oh, I've read it now :) > What is the status of the package? Not as well looked after as it should be :-/ I'm not sure most of the changes you describe are really bugs, more inefficiencies (e.g. compiler generated list comps still work even if they don't emit LIST_APPEND). Running "./python Lib/test/regrtest.py test_compiler -uall" will compile the entire stdlib with Lib/compiler (this is probably an overnight job) then running just plain "./python Lib/test/regrtest.py" will run the test suite using the compiled copy. It probably makes sense to submit a number of patches, particularly for the optimization versus straight fix divide. |
|||
msg48374 - (view) | Author: Stelios (sxanth) | Date: 2005-05-22 07:55 | |
Logged In: YES user_id=667962 Thanks for the instructions to regtest the compiler. I was hoping for a quick review by someone who knows well the compiler package to confirm that I'm doing the right thing in the places with XXX. It'll save me a lot of time. Hmmm. Since the patch is not ready to be applied, you can close this one if you want and we'll keep the link as a reference for future work. |
|||
msg48375 - (view) | Author: Terry J. Reedy (terry.reedy) * | Date: 2005-05-26 15:02 | |
Logged In: YES user_id=593130 If you think you might split and resubmit within a few months, you could keep this open for further comments and then use it for the optimization upgrades (updates). Put bug fixes in Bug Fixers .... |
|||
msg48376 - (view) | Author: Terry J. Reedy (terry.reedy) * | Date: 2005-05-26 15:04 | |
Logged In: YES user_id=593130 If you think you might split and resubmit within a few months, you could keep this open for further comments and then use it for the optimization upgrades (updates). Put bug fixes in Bug Fixers .... |
|||
msg48377 - (view) | Author: Stelios (sxanth) | Date: 2005-05-27 09:56 | |
Logged In: YES user_id=667962 I'm closing this one. Discovered many more things that can be improved and the patch will eventually get bigger than the entire package. I think it may be the best solution to submit an complete reimplementation of the package. |
|||
msg48378 - (view) | Author: Stelios (sxanth) | Date: 2005-05-29 11:16 | |
Logged In: YES user_id=667962 Some comments after regtesting: * The fix for generator expressions is *wrong* (grep the patch for the string 'genexp-1'). The right way to do this is to make the SymbolVisitor aware of the outmost iterable. * The FoldConst optimizer must be prepared to handle ZeroDivisionError. * It appears that the optimizer exposes some weird unicode bug. I can't find what's wrong so I suppose it's a python bug. Try to compile test_grammar to see it. |
|||
msg48379 - (view) | Author: Stelios (sxanth) | Date: 2005-05-29 12:11 | |
Logged In: YES user_id=667962 Correction: The bug is exposed by test_codecs, not test_grammar. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:56:11 | admin | set | github: 42001 |
2005-05-21 09:23:48 | sxanth | create |