This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Absolute/relative import not working?
Type: behavior Stage:
Components: Interpreter Core Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ncoghlan Nosy List: mitchchapman, mnot, ncoghlan, nnorwitz, twouters, zseil
Priority: normal Keywords:

Created on 2006-06-21 19:35 by mitchchapman, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
package.tgz mitchchapman, 2006-06-21 19:35 Re-creation of PEP 328 example package structure
main_relative_imports.diff ncoghlan, 2006-06-30 16:51 Allow relative imports from main when -m is used
Messages (11)
msg28851 - (view) Author: Mitch Chapman (mitchchapman) Date: 2006-06-21 19:35
Trying to import from a module using dotted import syntax produces 
this exception:

ValueError: Relative importpath too deep

This behavior has been confirmed on Mac OS X 10.4 using the Python 
2.5b1 disk image; and on CentOS 4 using the Python 2.5b1 source 
tarball.

The exception is raised regardless of whether the PYTHONPATH 
environment variable can see the toplevel directory of the package 
being tested; regardless of whether the import is performed from an 
interactive Python session or from a script invoked from the command 
line; and regardless of whether the main script starts with

from __future__ import absolute_import

To test, I tried to re-create the package structure used as an example 
in PEP 328.  (See attachments.)

Most of the files were empty, except moduleX.py and moduleY.py:

#moduleX.py:
from __future__ import absolute_import

from .moduleY import spam

#moduleY.py:
spam = "spam"

According to the PEP, if should be possible to import moduleX without 
error.  But I get the above exception whenever I try to import moduleX 
or to run it from the command line.

$ python2.5 moduleX.py 
Traceback (most recent call last):
  File "moduleX.py", line 3, in <module>
    from .moduleY import spam
ValueError: Relative importpath too deep


Is this a usage/documentation error?
msg28852 - (view) Author: Mark Nottingham (mnot) Date: 2006-06-21 21:16
Logged In: YES 
user_id=21868

Seeing the same behaviour; OSX with the installer.
msg28853 - (view) Author: Ziga Seilnacht (zseil) * (Python committer) Date: 2006-06-21 22:59
Logged In: YES 
user_id=1326842

I think this is a usage error. The problem is that
you run moduleX as a script. This puts the module's
directory as the first entry in sys.path (see
http://docs.python.org/dev/lib/module-sys.html#l2h-5058
for detais).
As a consequence, moduleX is recognised as a top
level module, not as part of a package.

If you want to test relative import, try opening an
interactive shell in the directory where `package`
resides, and type:

>>> from package.subpackage1 import moduleX
>>> moduleX.spam
'spam'
msg28854 - (view) Author: Mitch Chapman (mitchchapman) Date: 2006-06-21 23:57
Logged In: YES 
user_id=348188

Hm... but the same error occurs if one tries to import moduleX from an 
interactive Python session, or from a sibling module.

In other words, in 2.5b1 any module which uses relative imports can be used 
only as a fully-qualified member of a package.  It cannot be imported directly 
by a sibling module, and it cannot be used as a main module at all:

$ python2.5 -m package.subpackage1.moduleX
...
    from .moduleY import spam
ValueError: Relative importpath too deep

Given other efforts (PEP 299; PEP 338) to make it easier to use modules both 
as mainlines and as imports, I still think this is a bug.
msg28855 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2006-06-23 13:52
Logged In: YES 
user_id=34209

See the discussion started at:
http://mail.python.org/pipermail/python-dev/2006-June/066161.html

It's not a bug in 328 or 338 (the PEP that adds the -m
switch for packages), but in the interaction between the
two. I don't think this will be fixed for 2.5, since there
is no obvious fix. If it hurts when you press there, don't
press there. Either don't use -m for packaged modules, or
have the packaged module only use absolute imports. (But
don't be surprised if the script-module is imported twice,
once as __main__ and once as the module itself. That's a
whole other bug/feature.)



msg28856 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2006-06-27 04:41
Logged In: YES 
user_id=33168

http://mail.python.org/pipermail/python-dev/2006-June/066197.html

Nick you made mention of changing the docs, but I'm not sure
if you did or not.  Could you address this bug?  Thanks.
msg28857 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2006-06-27 11:34
Logged In: YES 
user_id=1038590

The issue isn't actually unique to the -m switch - the
problem is that relative imports are based on __name__, and
in the main module, __name__ always has the value
'__main__'. Hence, relative imports currently can't work
properly from the main module of an application, because the
main module doesn't know where it really fits in the Python
module namespace (this is at least fixable in theory for the
 main modules executed through the -m switch, but directly
executed files and the interactive interpreter are
completely out of luck).

With the old implicit relative imports this behaviour is
masked by the fact that executing a module puts that
module's directory on sys.path. When you execute a module in
a package directly, it actually imports its sibling modules
as top-level modules. The fact that the -m switch doesn't
allow this to occur is a deliberate design decision (putting
package directories on the system level path is a bad idea
because you can get multiple copies of the same module under
different names).

You should be able get something similar to the old implicit
relative import behaviour by sticking the following at the
top of your module (before doing any relative imports):

if __name__ = '__main__':
   from os.path import dirname
   __path__ = [dirname(__file__)]
   del dirname

This makes the relative import machinery treat your main
module as a package. The problem with this workaround is
that, just like the old situation with implicit relative
imports from the main module, you may end up with two
different copies of the sibling modules in sys.modules. One
copy will be '__main__.foo' while the other will be
'package.foo' (with implicit relative imports, the first
copy would have been a top level module called 'foo').

When I went to document this, I got stuck on the fact that
section 6 of the tutorial hasn't been updated for PEP 328 at
*all*. And the language ref palms the details off on to
Guido's packages essay. So describing the limitation in the
real documentation entails documenting PEP 328 properly in
the first place (which I'm *not* volunteering to do :).

However, I'll add a section to PEP 328 about '__main__' and
relative imports (including the workaround to get something
similar to the old behaviour back).
msg28858 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2006-06-27 11:46
Logged In: YES 
user_id=1038590

All that said, PEP 328 currently says that "from ...sys
import path" should be legal from moduleX in the example.

I tried it, and it currently fails - the ValueError gets
raised as soon as the number of dots in the relative path
exceeds the number of dots in __name__, instead of treating
a single excess dot as requesting an absolute import
instead. (All of the other examples in the PEP work as
specified when moduleX and subpackage1 are imported rather
than executed directly)

I believe fixing this would also fix Mitch's problem - an
explicit relative import from __main__ would be treated as a
top level import.

I also have an idea regarding the -m switch that I will
raise on python-dev.
msg28859 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2006-06-30 16:51
Logged In: YES 
user_id=1038590

Patch attached that allows relative imports from a main
module to work so long as:
  a. the top level package is either in the current
directory or somewhere else on sys.path; and
  b. the module is executed using -m so Python knows where
it fits in the package hierarchy

So to get the PEP 328 example to work, you'd have to run it as:

$python2.5 -m package.subpackage1.moduleX

The patch relies on a feature added to runpy in rev 47142
(post beta 1). I've added a question to PEP 356 as to how
this should be handled, since we're technically in feature
freeze.

Patch attached directly to the bug report because it's
stupidly early in the morning and I don't feel like dealing
with SF and then copying the patch tracker number here :)
msg28860 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2006-07-10 11:08
Logged In: YES 
user_id=1038590

Punting on this until 2.6. See updated PEP 338 for the gory
details.
msg58121 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2007-12-03 13:03
PEP 366 has been implemented for 2.6, which fixes this bug. The fix
turned out to be quite invasive (hence the PEP), so it won't be
backported to 2.5.
History
Date User Action Args
2022-04-11 14:56:18adminsetgithub: 43535
2007-12-03 13:03:08ncoghlansetstatus: open -> closed
type: behavior
resolution: fixed
messages: + msg58121
2006-06-21 19:35:52mitchchapmancreate