This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: os.listdir doesn't release GIL
Type: Stage:
Components: Extension Modules Versions: Python 2.4
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: georg.brandl Nosy List: ellisj, georg.brandl, georg.brandl, gvanrossum, jimjjewett, loewis
Priority: normal Keywords:

Created on 2006-02-15 22:45 by ellisj, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
listdir-threads.diff georg.brandl, 2006-02-18 11:13
Messages (10)
msg27531 - (view) Author: Jonathan Ellis (ellisj) Date: 2006-02-15 22:45
posix_listdir in posixmodule.c does not release the
global interpreter lock, blocking all other threads for
the duration of the call.
msg27532 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-02-18 11:13
Logged In: YES 
user_id=1188172

Attaching a patch. Please check.
msg27533 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-03-03 17:48
Logged In: YES 
user_id=21627

The patch looks fine. Please apply.
msg27534 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-03-07 12:48
Logged In: YES 
user_id=849994

Committed as rev. 42884.
msg27535 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2006-03-07 21:09
Logged In: YES 
user_id=6380

So the patch is broken because there's a continue in the
do-while-loop that jumps to the "while (result)" test
without setting result.

Fixing this by adding a label and a goto (in two places!)
creates awful spaghetti code.

Before somebody spends more time refactoring such hairy
code, I'd like to see a better motivation; the motivation
currently provided is a bit thin.  Are there use cases where
this call takes a long time?
msg27536 - (view) Author: Jonathan Ellis (ellisj) Date: 2006-03-07 22:36
Logged In: YES 
user_id=657828

My original use case is on a server back end with fairly
busy disks.  It's not unusual for listdir to take 100ms. 
That's a relatively long time to shut down the rest of the
system.
msg27537 - (view) Author: Jim Jewett (jimjjewett) Date: 2006-03-07 23:48
Logged In: YES 
user_id=764593

I think the spaghetti was already there.  

The gotos can be removed by just reversing the logic (if ! 
(...))

The XXX comment complains about four versions, but I count 
either three (windows without opendir, OS2, everything 
else) or five (the non-OS2 versions are split for unicode).

The code getting skipped is repeated because the logic for 
skipping "." and ".." is repeated four times.  (There would 
be another goto, if the OS2 case allowed threads.)

msg27538 - (view) Author: Jim Jewett (jimjjewett) Date: 2006-03-07 23:52
Logged In: YES 
user_id=764593

I think the spaghetti was already there.  

The gotos can be removed by just reversing the logic (if ! 
(...))

The XXX comment complains about four versions, but I count 
either three (windows without opendir, OS2, everything 
else) or five (the non-OS2 versions are split for unicode).

The code getting skipped is repeated because the logic for 
skipping "." and ".." is repeated four times.  (There would 
be another goto, if the OS2 case allowed threads.)

msg27539 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2006-03-08 04:07
Logged In: YES 
user_id=6380

Well, the difference is that before the patch was applied it
worked, and now it's broken.  So somebody please roll it
back; *then* we can think of a proper fix and review it
properly.
msg27540 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-03-08 07:47
Logged In: YES 
user_id=849994

I had already fixed my broken patch with "spaghetti" shortly
after Thomas' comment.

Martin has now refactored without goto, so I think you'll be
content now ;-)
History
Date User Action Args
2022-04-11 14:56:15adminsetgithub: 42908
2006-02-15 22:45:39ellisjcreate