This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: os.path.exists returns false negatives in MAC environments.
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: cda32, josiahcarlson, pitrou, sbennett, stefano-m, terry.reedy, vdupras
Priority: normal Keywords:

Created on 2005-02-07 00:57 by sbennett, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (11)
msg60645 - (view) Author: Stephen Bennett (sbennett) Date: 2005-02-07 00:57
In Mandatory Access Control environments (such as
SELinux), it's quite possible for stat to fail with
permission denied. In this case, os.path.exists will
return False incorrectly. The simple(ish) fix is to
check for an access denied error (which would indicate
present, but not readable) when using stat to check for
existence of files.
msg60646 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2005-02-16 20:26
Logged In: YES 
user_id=593130

Does 'access denied' always mean 'present but not readable' 
in every environment that gives such messages?  I ask 
because I have vague memories of wasting time trying to 
get access to something that did not exist, because access 
denied (or something like that) meant that I was denied 
access even to info about whether it existed or not.

In any case, a reproducible example would help someone to 
verify, fix, and write a test case for this if it is deemed to be 
a fixable bug.
msg60647 - (view) Author: Stephen Bennett (sbennett) Date: 2005-02-16 22:46
Logged In: YES 
user_id=817465

As far as I know (at least for SELinux), permission denied
on stat() always means that the file exists, but getattr
isn't allowed. As for a reproducible test case, probably the
simplest example is a vanilla Fedora Core 3 system with
SELinux enabled and strict policy. From a regular user
account, call os.path.exists("/etc/shadow"). It will return
False even though the file exists. For comparison, an `ls -l
/etc/shadow` from the command line will simply print
'Permission Denied'.
msg60648 - (view) Author: Josiah Carlson (josiahcarlson) * (Python triager) Date: 2005-05-31 07:09
Logged In: YES 
user_id=341410

I believe Terry was curious about something like
os.path.exists("/etc/shadow/abc123") vs `ls -l
/etc/shadow/abc123`.  If not, I know I am curious, and I
believe it may help with a corner case.
msg60649 - (view) Author: Stephen Bennett (sbennett) Date: 2005-05-31 12:56
Logged In: YES 
user_id=817465

In the case of /etc/shadow/abc123, the stat will fail with
"Not a directory". However, attempting to stat /root/abc123
as a limited account will return permission denied unless
you have the search permission to the parent directory.
However, this is an issue with regular Unix permissions too
-- try to stat() a file that's inside a directory with 000
permissions.

One possible way around this is to attempt to get a listing
of the parent dir if the stat fails with permission denied
-- if it succeeds then the file exists but can't be statted;
if it fails then (at least for the purposes of the library
functions) if doesn't.
msg70891 - (view) Author: Virgil Dupras (vdupras) (Python triager) Date: 2008-08-08 09:26
hsoft-dev:~ hsoft$ mkdir foobar
hsoft-dev:~ hsoft$ echo "baz" > foobar/baz
hsoft-dev:~ hsoft$ chmod 000 foobar/baz
hsoft-dev:~ hsoft$ python
Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os.path
>>> os.path.exists('foobar/baz')
True
>>> 
hsoft-dev:~ hsoft$ chmod 000 foobar
hsoft-dev:~ hsoft$ python
Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os.path
>>> os.path.exists('foobar/baz')
False
>>> os.path.exists('foobar')
True
>>> 

This seems like the correct behavior to me.
msg70893 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-08-08 09:41
The only sane alternative to the current behaviour would be to raise an
Exception from os.path.exists rather than returning False. But it would
also break a lot of code, and complexify code using os.path.exists which
currently doesn't need to check for exceptions. Returning True sounds
completely incorrect on the other hand.

If there is no other straightforward method than stat() to know if a
path exists, I suggest closing this bug as wontfix.
msg70895 - (view) Author: Virgil Dupras (vdupras) (Python triager) Date: 2008-08-08 09:48
Antoine, I think that when this bug was filed, the first 
"os.path.exists('foobar/baz')" in my example would return False.

So it's not that this bug shouldn't be fixed, but that it is already 
fixed.
msg91909 - (view) Author: Colin Alston (cda32) Date: 2009-08-24 08:22
I also hit upon this issue and IMHO returning False in a "permission
denied" scenario is less than obvious behaviour. 

It also means I have no way to catch this edge case in my own code
immediately, I have to implement far larger logic to check now if False
means the file doesn't exist, or if the user doesn't have access to the
file. 

os.path.exists should absolutely raise a Permission exception.
msg114406 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-08-19 19:11
Previous comments suggest there is no agreement as to whether or not this is an oustanding bug.
msg253564 - (view) Author: Stefano Mazzucco (stefano-m) Date: 2015-10-27 21:26
FWIW, I have just been experiencing this on CentOS 6.5 with Python 2.7.5 (sorry).

I could make the Python code happy by running chcon[1] with the correct context type, otherwise os.path.exists would happily return False for a file that actually existed.


[1] https://fedoraproject.org/wiki/SELinux/chcon
History
Date User Action Args
2022-04-11 14:56:09adminsetgithub: 41538
2017-03-24 16:34:15serhiy.storchakasetstatus: open -> closed
resolution: wont fix
stage: resolved
2015-10-27 21:26:22stefano-msetnosy: + stefano-m
messages: + msg253564
2014-02-03 19:48:53BreamoreBoysetnosy: - BreamoreBoy
2010-08-19 19:11:40BreamoreBoysetversions: + Python 3.1, Python 2.7, Python 3.2
nosy: + BreamoreBoy

messages: + msg114406

type: behavior
2009-08-24 08:22:13cda32setnosy: + cda32
messages: + msg91909
2008-08-08 09:48:38vduprassetmessages: + msg70895
2008-08-08 09:41:17pitrousetnosy: + pitrou
messages: + msg70893
2008-08-08 09:26:54vduprassetnosy: + vdupras
messages: + msg70891
2005-02-07 00:57:06sbennettcreate