This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Prevent race condition in os.makedirs
Type: Stage:
Components: None Versions:
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: georg.brandl, georg.brandl, nirs, rbarran
Priority: normal Keywords: patch

Created on 2005-07-17 19:42 by nirs, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
makedirs.diff nirs, 2005-12-05 12:46 prevent race conditions in os.makedirs, new tests
Messages (11)
msg48583 - (view) Author: Nir Soffer (nirs) * Date: 2005-07-17 19:42
makedirs check if a directory exists, and call mkdir to create it. In 
rare cases, the directory might have been created by another 
process or thread in the time passed from the exists() call to the 
mkdir() call.

To prevent this rare race condition, use try: expect: to create the 
directory, then ignore "File exists" errors, which mean the directory 
was there.
msg48584 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2005-07-17 19:57
Logged In: YES 
user_id=1188172

See bug #1223238.
msg48585 - (view) Author: Nir Soffer (nirs) * Date: 2005-07-17 20:06
Logged In: YES 
user_id=832344

I can't find such bug or patch.
msg48586 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2005-07-17 20:13
Logged In: YES 
user_id=1188172

http://sourceforge.net/tracker/index.php?func=detail&aid=1223238&group_id=5470&atid=105470
msg48587 - (view) Author: Nir Soffer (nirs) * Date: 2005-07-17 21:13
Logged In: YES 
user_id=832344

Here is another patch, using simpler design, maybe for 2.5?

The attached patch will try once to create each leaf in the path, ignoring 
existing leafs. 

This should work for most cases. If one run application that can delete its 
own directories, he should use proper locking.
msg48588 - (view) Author: Richard Barran (rbarran) Date: 2005-12-02 16:07
Logged In: YES 
user_id=1207189

Hi,
Your patch "makedirs.py" for python 2.5 will fail under
Windows if a drive letter is specified:

>>> os.getcwd()
'C:\\Temp\\makedirs'
>>> os.listdir('.')
['makedirs.py', 'makedirs.pyc', 'test.py']
>>> makedirs.makedirs('c:/temp/makedirs/a/b')
>>> os.listdir('.')
['makedirs.py', 'makedirs.pyc', 'temp', 'test.py']

It will create the following path:
C:\Temp\makedirs\temp\makedirs\a\b

Also I ran it through the test suite (lib\test\test_os.py)
and it failed one test:

======================================================================
FAIL: test_makedir (__main__.MakedirTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_os.py", line 327, in test_makedir
    self.failUnlessRaises(OSError, os.makedirs, os.curdir)
AssertionError: OSError not raised

----------------------------------------------------------------------

I haven't looked into *why* this happens - I'll dig a bit
deeper into this subject sometime next week.

HTH,
Richard
msg48589 - (view) Author: Nir Soffer (nirs) * Date: 2005-12-02 18:11
Logged In: YES 
user_id=832344

The 2.5 code is only a proof of concept, don't use it as is :-)

All the directories are handled in the same way - ignore OSError for 
existing directories, which is not compatible with the docs, that say it should 
raise an OSError for the last directory.  This has to be fixed of course.

I'll get 2.5 development code and submit a real working patch.




msg48590 - (view) Author: Nir Soffer (nirs) * Date: 2005-12-05 01:08
Logged In: YES 
user_id=832344

I deleted both patches as they are both wrong:

The patch against 2.4.1 will not raise OSError when trying to create existing 
directories.

The simpler code for 2.5 will not work because os.path.normpath is not safe 
to use if the path contain symbolic links.
msg48591 - (view) Author: Nir Soffer (nirs) * Date: 2005-12-05 01:21
Logged In: YES 
user_id=832344

Please review this new patch, tested with current 2.5 code.

I factored duplicate code in makedirs and removedirs into private _splits 
function, kind of the "super" version of os.path.split for the "super" directory 
utilities.

I also simplified the test code for makedirs, and added more test cases.
msg48592 - (view) Author: Nir Soffer (nirs) * Date: 2005-12-05 12:46
Logged In: YES 
user_id=832344

Sorry, here is the patch.
msg48593 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-12-09 09:24
Something similar has been committed as patch #1608267.
History
Date User Action Args
2022-04-11 14:56:12adminsetgithub: 42197
2005-07-17 19:42:09nirscreate