This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Enhancing '-m' to support packages (PEP 338)
Type: Stage:
Components: None Versions: Python 2.5
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ncoghlan, rhettinger
Priority: normal Keywords: patch

Created on 2004-10-08 21:44 by ncoghlan, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
execmodule.py ncoghlan, 2004-10-09 15:57 Python code to execute a module as a script
PEP_338.diff ncoghlan, 2004-12-12 01:38 C source diff for PEP 338
pep338.zip ncoghlan, 2004-12-12 01:41 Simple package for feature demonstration
Messages (9)
msg47031 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2004-10-08 21:44
Thomas Heller posted "python -m pychecker.checker
<script>' as an example where it is worthwhile to allow
execution of a script inside a package. This seems like
a compelling use case to me (making pychecker easy to
run is enough of a PITA that I confess I often don't
run it), and I'm sure there are other examples.

The only semantics that I can see working for packages
are the following:

python -m pychecker.checker

is equivalent to:

python execchecker.py

where execchecker.py is:

import sys, imp
import pychecker
info = imp.find_module("checker", pychecker.__path__)
sys.argv[0] = info[1]
execfile(info[0])

We *have* to do the import of the requested script's
containing package - otherwise, we can't obey Python's
packaging rules with respect to the use of __path__.

I will attach two versions.

The first will, when it fails to find a module for
itself, switch to executing something like "-m
execmodule". 'execmodule.py' would be a new stdlib
module to handle the above behaviour (initially not
providing any useful interfaces when directly
imported). Guido was certainly right when he said this
will be easier to enhance if written in Python - I
think this way (going to Python when it isn't a
top-level module) gives us the best of both worlds.

The second version will be pure C. The reason I'm going
to do that, is I expect the idea of adding a standard
library module for this to be controversial, especially
given the proximity to 2.4beta1. (I hope I'm wrong,
since I think the Python-C hybrid is a more flexible
solution, which can also be leveraged by other
interpreter implementations)
msg47032 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2004-10-08 22:51
Logged In: YES 
user_id=1038590

execmodule.py attached.

You can experiment using "python -m execmodule <script>",
since I haven't written the C code to automatically hook
into this yet.

"from execmodule import execmodule" also has intriguing
possibilities. . .
msg47033 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2004-10-08 23:53
Logged In: YES 
user_id=1038590

run_package_Py.diff attached, and a new version of execmodule.py

If it fails to find the module requested with '-m', it
reinterprets the command line from "python -m <script>" to
"python -m execmodule <script>".

One obscure issue is that a command line like "python
-mpychecker.checker" will overwrite the C-level argv[0] with
the path to execmodule.py, whereas "python -m
pychecker.checker" will leave it alone (the path to
execmodule.py is stored in argv[1] in the second case).

This may interact oddly with anyone using Py_GetArgcArgv to
get the *original* command line arguments.

One possibility is to just leave junk in sys.argv[0] when
running execmodule automatically (execmodule just deletes it
anyway).

I'm out of time for today, so I doubt I'll be able to get to
a C version before the beta release.
msg47034 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2004-10-09 00:44
Logged In: YES 
user_id=80475

This should be taken back to python-dev.  Guido had strong 
reservations about even the simple version of the patch.  It 
is rapidly losing its innocence.  

Also, it's a little late in the game. The beta is due out on 
Tuesday (leaving no time for thorough review or opportunity 
for fixups if necessary).

The mention of "intriguing possibilities" will likely not help 
rally support.
msg47035 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2004-10-09 14:17
Logged In: YES 
user_id=1038590

I agreee with Raymond - bumping the group to Python 2.5.

I'll put execmodule.py in the cookbook as a useful 2.4
recipe (getting the failure modes right was surprisingly
tricky, and I can see people wanting this capability)

At the moment, making this part of the standard library
would be understandably controversial. It will probably be
better to wait and see what the wider (i.e. non-python-dev)
community thinks of the basic version if '-m' before making
things too complex.

The intriguing possibilities of execmodule are simply that
it makes "execmodule('some_pkg.some_module') and
"execmodule('somemodule') available at the Python level. The
notion of using the Python namespace instead of the
filesystem (via execfile) to execute subscripts is an
interesting one.
msg47036 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2004-10-09 15:57
Logged In: YES 
user_id=1038590

execmodule has been posted to the cookbook:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307772

I've also updated the attached version of execmodule.py to
match what I posted to the cookbook. The main difference is
to increase the similarity of the interface to that for
execfile - the dict arguments are called globals and locals,
and the module is correctly executed in the caller's
namespace if no globals are provided.
msg47037 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2004-11-24 13:01
Logged In: YES 
user_id=1038590

I've put a PEP together for adding this in 2.5 - a draft was
posted to python-dev, but I'm still waiting for the PEP number.

Two points from the PEP draft:
1. The implementation will stay as a Python/C hybrid. It's
more useful that way (e.g. "scripts that run scripts" like
pychecker.checker or profile can easily be given '-m' style
arguments of their own)

2. The C portion of the implementation needs to change so
that argv[0] is never altered. The easiest way to do that is
to leave junk in sys.argv[0] when execmodule.py gets invoked
implicitly. I don't have a functional build machine at the
moment, which is why I haven't updated the patch.

The draft can be found here:
http://mail.python.org/pipermail/python-dev/2004-November/049837.html
msg47038 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2004-12-12 01:38
Logged In: YES 
user_id=1038590

The PEP now has a number (PEP 338) which I've added to the 
title of the tracker item. 
 
I've also attached an updated patch which matches the 
behaviour described in the PEP. 
 
Finally, I added a small archive with a simple package structure 
to demonstrate the feature. 
 
To try it out: 
  1. Download execmodule.py to your build environment's Lib 
directory 
  2. Apply the patch from the src directory with "patch -p0 -i 
PEP_338.diff" & rebuild 
  3. Extract the archive so it makes a "pep338" package inside 
the Lib directory. 
 
Then, "./python -m pep338.hi" and "./python -m 
pep338.subpkg.hi" should print out short messages along with 
the contents of sys.argv. 
msg47039 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2006-02-11 08:51
Logged In: YES 
user_id=1038590

Closing this patch and submitting a new one after major
update to the PEP.
History
Date User Action Args
2022-04-11 14:56:07adminsetgithub: 40995
2004-10-08 21:44:01ncoghlancreate