This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Why not drop the _active list?
Type: Stage:
Components: None Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: atila-cheops, hvb_tup, loewis, nnorwitz, tfaujour
Priority: normal Keywords:

Created on 2006-03-29 07:16 by hvb_tup, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (11)
msg27934 - (view) Author: HVB bei TUP (hvb_tup) Date: 2006-03-29 07:16
I am using a modified version of subprocess.py,

where I have removed the _active list and all 
references to it.

I have tested it (under Windows 2000) and there were 
no errors.

So what is the reason for managing the _active list 
at all? Why not drop it?
msg27935 - (view) Author: Tristan Faujour (tfaujour) Date: 2006-03-29 13:59
Logged In: YES 
user_id=1488657

I agree.

The use of _active makes subprocess.py thread-UNsafe.

See also: Bug #1199282

In order to have a thread-safe subprocess.py, I commented
out the call to _cleanup() in Popen.__init__(). As a side
effect, _active becomes useless.
msg27936 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-03-29 20:41
Logged In: YES 
user_id=21627

The purpose of the _active list is to wait(2) for open
processes. It needs to stay.
msg27937 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2006-03-30 07:43
Logged In: YES 
user_id=33168

If you always called wait() the _active list isn't
beneficial to you.  However, many people do not call wait
and the _active list provides a mechanism to cleanup zombied
children.  This is important for many users.

If you need thread safely, you can handle the locking
yourself before calling poll()/wait().
msg27938 - (view) Author: cheops (atila-cheops) Date: 2006-03-30 08:04
Logged In: YES 
user_id=1276121

what happens if you are doing a _cleanup (iterating over a
copy of _active) in multiple threads?
can it not happen then that you clean up a process 2 times?

thread 1 starts a _cleanup: makes a copy of _active[:] and
starts polling
thread 2 starts a _cleanup: makes a copy of _active[:] and
starts polling

thread 1 encounters a finished process and removes it from
_active[]
thread 2 does not know the process is removed, finds the
same process finished and tries to remove it from _active
but this fails, because thread 1 removed it already

so the action of cleaning up should maybe be serialized
if 1 thread is doing it, the other one should block

everyone who needs this can of course patch the
subprocess.py file, but shouldn't this be fixed in the library?
msg27939 - (view) Author: cheops (atila-cheops) Date: 2006-03-30 08:06
Logged In: YES 
user_id=1276121

the same problem probably exists in popen2.py
there _active is also used
so if something is fixed in subprocess.py, it should
probably also be fixed in popen2.py
msg27940 - (view) Author: cheops (atila-cheops) Date: 2006-03-30 08:17
Logged In: YES 
user_id=1276121

also #1214859 is interesting, has a patch
msg27941 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-03-30 16:31
Logged In: YES 
user_id=21627

attila-cheops, please read the 2.5 version of popen2.

popen2 now only adds processes to _active in __del__, not in
__init__, so the problem with the application still wanting
to wait/poll is solved.

Multiple threads simultaneously isn't a problem, since it is
(or should be) harmless to invoke poll on a process that has
already been waited for. For only one of the poll calls, the
wait will actually succeed, and that should be the one that
removes it from the _active list.

The same strategy should be applied to subprocess.
msg27942 - (view) Author: cheops (atila-cheops) Date: 2006-04-05 08:27
Logged In: YES 
user_id=1276121

the implementation in 2.5 of popen2 seems good
if you or astrand could patch subprocess.py accordingly,
that would be great.
msg27943 - (view) Author: cheops (atila-cheops) Date: 2006-04-10 14:55
Logged In: YES 
user_id=1276121

see patch #1467770
msg27944 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-04-10 15:58
Logged In: YES 
user_id=21627

This was fixed with said patch.
History
Date User Action Args
2022-04-11 14:56:16adminsetgithub: 43107
2006-03-29 07:16:44hvb_tupcreate