This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python compiler encodes path to source in .pyc
Type: Stage:
Components: IDLE Versions: Python 2.3
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: jwpolley, loewis, nbastin
Priority: normal Keywords:

Created on 2004-06-23 02:17 by jwpolley, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (7)
msg21247 - (view) Author: Jonathan Polley (jwpolley) Date: 2004-06-23 02:17
This bug was notices in version 2.0, but I was able to
reproduce it in version 2.3.4.  When a python module is
compiled, the path to that file is encoded in the .pyc
file.  This causes problems when a multi-platform
development environment is used, in my case it is a
hybrid UNIX/Windows platform.

To reproduce the problem, perform the following steps
from within IDLE:

1)  run python
2)  import crlf
3)  exit python
4)  copy the crlf.py and crlf.pyc files from
Tools/Scripts to another directory
5)  run python
6)  add the path to the copies of crlf.py* to the start
of the system path.
7)  import crlf
8)  print crlf.__file__ (this will yield the proper
path to the files)
9)  using the Open Module..., try to open the crlf
module.  You will get an error saying that the module
was not found.

This is a major problem when doing multi-platform
debugging.  If an exception is raised in a module that
was compiled on another system (with a different path
or OS), the debugger can not find the file to open it
so it can be debugged.  It also makes the "Open
Module..." menu option unreliable.

If you look in the .pyc file you will find the path to
the location where the file was originally generated.
msg21248 - (view) Author: Nick Bastin (nbastin) * (Python committer) Date: 2004-06-25 14:47
Logged In: YES 
user_id=430343

Changing this to be filed against IDLE, not the parser/compiler (should 
also fix the debugger as well).  There is no magic value that the compiler 
could put in  there that would make this right, so external tools will just 
have to deal with that.
msg21249 - (view) Author: Nick Bastin (nbastin) * (Python committer) Date: 2004-06-25 14:48
Logged In: YES 
user_id=430343

Changing this to be filed against IDLE, not the parser/compiler (should 
also fix the debugger as well).  There is no magic value that the compiler 
could put in  there that would make this right, so external tools will just 
have to deal with that.
msg21250 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2004-06-27 21:06
Logged In: YES 
user_id=21627

I fail to see any kind of bug in this report. Yes, the path
to the source code is hard-coded in the .pyc file, but that
is intentional, and not a bug. If you want the tracebacks to
be correct on multiple installations, you have to recreate
the .pyc files every time.
msg21251 - (view) Author: Jonathan Polley (jwpolley) Date: 2004-06-28 04:34
Logged In: YES 
user_id=391068

This bug was reported because the encoding of the path in the 
.pyc generates errors in some tools (namely IDLE) as well as 
generates erroneous tracebacks when exceptions are raised.  
When some of the not very software savvy script writers see "/rfs/
proj/cse/scripts" in their traceback, instead of the "P:/scripts" that 
they expect, it leads to confusion.

I'm not sure why the __file__ and exception mechanism use 
different methods for determining the path to the file, but they 
do.  The same can be said for the value of having the fully 
qualified path encoded in the .pyc.
msg21252 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2004-06-28 04:52
Logged In: YES 
user_id=21627

You will have to report errors on tools one by one, and some
of them may not be fixable. From this report, it is not
clear what precisely the error in IDLE is.

In any case, the behaviour that the .pyc compiler embeds the
source file path cannot and will not change. The mechanism
for __file__ is different because __file__, for a .pyc file,
refers to that very .pyc file, not to the source file. OTOH,
the debug information refers to the source file.

If the .pyc file says the source is in
/rfs/proj/cse/scripts, you should arrange that the source
file really is in that location. If you move the files to a
different machine, you will need to delete the .pyc files.

I do not completely agree that the traceback is erroneous.
If your complaint is that it refers to a non-existing file:
this could be fixed by not printing the name of the file if
it doesn't exist. However, I would not think that this would
be an improvement.

If you think that the traceback should magically guess where
the source file is: this is not implementable in a reliable way.
msg21253 - (view) Author: Jonathan Polley (jwpolley) Date: 2004-06-28 12:15
Logged In: YES 
user_id=391068

>If you move the files to a different machine, you will need to 
delete the .pyc files.

In my case, the files are not moved between machines.  They are 
referenced by both Windows and UNIX systems simultaneously.  
The platform that references the .py file first will generate the 
.pyc, so we have a mixture of UNIX and Windows paths embedded 
in the .pyc files.  In the case when we deliver python scripts, we 
only deliver compiled python files.
History
Date User Action Args
2022-04-11 14:56:05adminsetgithub: 40436
2004-06-23 02:17:20jwpolleycreate