This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: inspect.py imports local "tokenize.py" file
Type: Stage:
Components: Library (Lib) Versions: Python 2.4
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: drfarina, georg.brandl, loewis
Priority: normal Keywords:

Created on 2006-11-02 20:10 by drfarina, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
inspect_bug.tar.gz drfarina, 2006-11-04 10:09
Messages (8)
msg30430 - (view) Author: Daniel Farina (drfarina) Date: 2006-11-02 20:10
urllib2 may do a relative import of tokenize.py, which
can cause it to function abnormally when the user has a
file named "tokenizer.py" in the directory as a script
that utilizes urllib2.

The attached tarball has a shell script called
"showme.sh" that will give standard input to afile.py,
which contains two import statements and nothing else.
Code in the neighboring tokenize.py will be executed,
resulting in printing those lines to standard output.

Expected behavior:
no code in tokenize.py should be executed.

Reproducible on Ubuntu 6.10
msg30431 - (view) Author: Daniel Farina (drfarina) Date: 2006-11-02 20:11
Logged In: YES 
user_id=425987

Typo in the above:

"tokenizer.py" should just be "tokenize.py"
msg30432 - (view) Author: Daniel Farina (drfarina) Date: 2006-11-02 20:16
Logged In: YES 
user_id=425987

Yet another typo: the script is called "show.sh"

It's the only shell script in there, so no fear.
msg30433 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-11-03 16:20
Logged In: YES 
user_id=849994

I can't reproduce that here with your example code, and
there's also no mention of "tokenize" in urllib2.py.

In any case, "import tokenize" is not a relative import, and
it's the only way a standard library module can import
another standard library module. That this can interfere
with user-defined modules is known and must be worked around
by not naming them like builtin modules.
msg30434 - (view) Author: Daniel Farina (drfarina) Date: 2006-11-03 18:26
Logged In: YES 
user_id=425987

I have a fresh Ubuntu Edgy install (although my home
directory is quite old and persistent). I have installed the
Python 2.5 package, but am not using it in this case.

You are correct, urllib2 doesn't contain the import. I still
get the behavior on my machine. 

On my machine, the following interaction takes place:
fdr@Tenacity:~/urllib2_bug$ ls
afile.py  show.sh  tokenize.py  tokenize.pyc
fdr@Tenacity:~/urllib2_bug$ ./show.sh
import sys

import urllib2

fdr@Tenacity:~/urllib2_bug$ more show.sh
#!/bin/sh
cat afile.py | python afile.py
fdr@Tenacity:~/urllib2_bug$ more afile.py
import sys
import urllib2
fdr@Tenacity:~/urllib2_bug$

As we can see from the contents of afile.py, it shouldn't be
printing anything.
msg30435 - (view) Author: Daniel Farina (drfarina) Date: 2006-11-04 10:09
Logged In: YES 
user_id=425987

I have done something slightly less lazy and had the
tokenize.py file throw an exception. The result was not in
fact stemming from urllib2, but the inspect.py file. I have
duplicated this issue on a fresh install of Edgy on a VM at
work (from scratch, no home directory preservation or
anything). I'm perfectly willing to accept that I should
file this in Ubuntu's launchpad instead. I pray I am not a
complete crackpot.

Here is a new transcription of my interactions (slightly
edited for brevity):

[fdr@Tenacity ~/inspect_bug]$ more *
::::::::::::::
afile.py
::::::::::::::
import inspect

#some random text.
#more text.
#maybe even some more.
::::::::::::::
show.sh
::::::::::::::
#!/bin/sh
cat afile.py | python afile.py
::::::::::::::
tokenize.py
::::::::::::::
import sys

for line in sys.stdin:
    print line

raise Exception, 'Shouldn\'t be here'
[fdr@Tenacity ~/inspect_bug]$ ./show.sh 
import inspect



#some random text.

#more text.

#maybe even some more.

Traceback (most recent call last):
  File "afile.py", line 1, in ?
    import inspect
  File "/usr/lib/python2.4/inspect.py", line 31, in ?
    import sys, os, types, string, re, dis, imp, tokenize,
linecache
  File "/home/fdr/inspect_bug/tokenize.py", line 6, in ?
    raise Exception, 'Shouldn\'t be here'
Exception: Shouldn't be here
[fdr@Tenacity ~/inspect_bug]$ 

msg30436 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-11-04 15:27
Logged In: YES 
user_id=21627

As gbrandl explains, it is an error to name your own modules
with the names of the modules of the standard library. As
you have a module named tokenize, and as tokenize is also a
module of the standard library, the bug is in your code.
Closing this report as invalid.
msg30437 - (view) Author: Daniel Farina (drfarina) Date: 2006-11-04 20:40
Logged In: YES 
user_id=425987

Oops. Sorry about that...
History
Date User Action Args
2022-04-11 14:56:21adminsetgithub: 44191
2006-11-02 20:10:08drfarinacreate