This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: os.path.expandvars deletes things on w32
Type: Stage:
Components: Library (Lib) Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: sjoerd Nosy List: bdadsetan, gvanrossum, mikemccand, sjoerd, tim.peters
Priority: normal Keywords:

Created on 2001-12-18 14:29 by mikemccand, last changed 2022-04-10 16:04 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
ntpath.patch sjoerd, 2007-01-16 11:42
Messages (10)
msg8304 - (view) Author: Michael McCandless (mikemccand) Date: 2001-12-18 14:29
Try this:

  import os.path
  print os.path.expandvars('foo$doesnotexist')

On FreeBSD, Python 2.1, I get:

  'foo$doesnotexist'

But on WIN32, Python 2.1, I get:

  'foo'

The docs explicitly states that variables that are not 
found will be left in place ... but on win32 that 
appears to not be the case.
msg8305 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-12-18 14:35
Logged In: YES 
user_id=6380

Confirmed, also in 2.2. I don't understand it, the code
looks OK.
msg8306 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-12-18 14:43
Logged In: YES 
user_id=6380

Hm, I do understand it, the code is broken (compared to the
spec).

No time to fix it.
msg8307 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2001-12-19 06:56
Logged In: YES 
user_id=31435

Another bug:  with two adjacent envars that do exist, only 
the first is expanded (on Windows):

>>> os.path.expandvars('$TMP$TMP')
'c:\\windows\\TEMP$TMP'
>>>

Another bug:  the Windows expandvars doesn't expand envars 
in single quotes, but the posixpath flavor does:

>>> ntpath.expandvars("'$TMP'")
"'$TMP'"
>>> posixpath.expandvars("'$TMP'")
"'c:\\windows\\TEMP'"
>>>

Another bug:  $$ is an escape sequence (meaning a single $) 
on Windows but not on Unix:

>>> ntpath.expandvars('$$')
'$'
>>> posixpath.expandvars('$$')
'$$'
>>>

Unassigning from me, as this is a bottomless pit spanning 
platforms and bristling with backward-compatibility traps 
no matter what's done about it.  Somebody who cares enough 
should write a PEPlet to sort out the mess, else I'd just 
leave it alone.
msg8308 - (view) Author: Behrang Dadsetan (bdadsetan) Date: 2003-06-22 13:45
Logged In: YES 
user_id=806514

tim_one is right. There is plenty of dodgy things hiding
behind the os.path world, especially when it comes to
os.path.expandvars()

There are two problems here. 
- Mismatch in between the doc strings of the different
implementation of expandvars and the "official"
os.path.expandvars documentation.
- the ntpath and dospath implementations are buggy when
compared to their comments/docstrings.

About the first problem, the inconsistency created some time
ago in between the different implementations tasks makes it
difficult to choose a solution. Everyone will probably agree
that all the platform specific implementations of expandvars
should have the same functionality. The one that should be
taken over will probably need to be announced by the BDFL.

Some rule which should not have let this here happen, and on
which I believe we all will agree on:
Same interface=same documentation->same functionality

To implement either copy paste exactly the same expandvars
definition from one platform to another (NT, DOS, POSIX), or
somehow rather arrange that when there is no specific
implementation for the platform, a "default" python
implementation is used on the os.path level. To maximize the
fruits of my small work, I would of course prefer that the
version below becomes the standard and that the
documentation get updated.

To be complete, shall the documentation remain unchanged and
the implementation of dos and nt gets adapted (copied from
posix), the mac implementation could remain unchanged. But I
feel its docstring and its documentation should be in line
with the rest of the implementations.

So my view point-> same interface, same documentation

For the second problem - as of now a real bug whatever we
decide, I wrote within this comment (hereafter) a new
expandvars version which fits the docstring documentation of
dospath.py and the comments of ntpath.py. Sorry you will be
getting no patch from me at the moment since sourceforge's
anonymous CVS access does not like me. Please note that my
version borrows alot from the posixpath.py implementation
and my changes are the ones of a python amateur who is open
to critic.

#expandvars() implementation
_varprog = None
_findquotes = None
def expandvars(path):
    """Expand paths containing shell variable substitutions.
    The following rules apply:
        - no expansion within single quotes
        - no escape character, except for '$$' which is
translated into '$'
        - ${varname} is accepted.
        - varnames can be made out of letters, digits and
the character '_'"""
    global _varprog, _findquotes
    if '$' not in path:
        return path
    if not _varprog:
        import re
        _varprog = re.compile(r'\$(\w+|\{[^}]*\}|\$)')
        _findquotes = re.compile("'.*?'")
    quoteareas = []
    i = 0
    while 1:
        quotearea = _findquotes.search(path, i)
        if not quotearea:
            break
        (i, j) = quotearea.span(0)
        quoteareas.append((i, j))
        i = j
    i = 0
    while 1:
        m = _varprog.search(path, i)
        if not m:
            break
        i, j = m.span(0)
        insidequotes=None
        for (quotebegin, quoteend) in quoteareas:
            if quotebegin < i and quoteend > i:
                insidequotes=1
                break
        if insidequotes:
            i = j
            continue
        name = m.group(1)
        if name[:1] == '$':
            path = path[:i] + '$' + path[j:]
            i = i + 1
        else:
            if name[:1] == '{' and name[-1:] == '}':
                name = name[1:-1]
            if os.environ.has_key(name):
                tail = path[j:]
                path = path[:i] + os.environ[name]
                i = len(path)
                path = path + tail
            else:
                i = j
    return path
msg8309 - (view) Author: Sjoerd Mullender (sjoerd) * (Python committer) Date: 2007-01-16 11:42
I got bit by this today and saw there was a bug report of over 6 years old.
The patch is trivial, though.
The attached patch may not solve the problem that the various implementations of expandvars are made exactly the same again, but it does solve the problem that this implementation doesn't do what it promises in the doc string.  It also solves the problem noted by Tim of two consecutive non-existing variables being treated differently.
File Added: ntpath.patch
msg8310 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-01-16 15:50
Looks good. Sjoerd, can you check that in yourself or did you give up your privileges?
msg8311 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-01-16 15:52
Oh, I forgot. It needs a unit test (preferably one that tests each xxpath module on each platform).
msg8312 - (view) Author: Sjoerd Mullender (sjoerd) * (Python committer) Date: 2007-01-16 16:03
I can check this in.  I'll try to create some tests.
msg8313 - (view) Author: Sjoerd Mullender (sjoerd) * (Python committer) Date: 2007-01-16 16:44
Committed as rev. 53460.
History
Date User Action Args
2022-04-10 16:04:47adminsetgithub: 35775
2001-12-18 14:29:51mikemccandcreate