This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: os.popen() hangs on {Free,Open}BSD
Type: Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: jackjansen, jasonrm
Priority: normal Keywords:

Created on 2003-02-27 01:12 by jasonrm, last changed 2022-04-10 16:07 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
hang jasonrm, 2003-02-27 01:13
hang jasonrm, 2003-02-27 01:14
hang jasonrm, 2003-02-27 01:15
fstat.txt jasonrm, 2003-03-03 20:06
linux-lsof.txt jasonrm, 2003-03-04 18:45
Messages (7)
msg14870 - (view) Author: Jason R. Mastaler (jasonrm) Date: 2003-02-27 01:12
I'm seeing odd behavior with os.popen() on 
certain platforms such as FreeBSD and OpenBSD. 
 
FWIW, I posted about this problem almost two 
years ago on c.l.p, and never got a resolution. 
 
See 
http://mail.python.org/pipermail/python-list/2001-May/042373.html 
 
Now, many Python and FreeBSD upgrades later, 
I see the problem still exists. Now however, I need to 
find a solution as I can't workaround it any longer. 
 
I've attached about the simplest test case I could 
produce below. You can call it from a .forward or 
.qmail file.  
 
On Free/OpenBSD, the program runs but leaves a  
python process lying around: 
 
jasonrm    142  0.0  0.2  2576 2008  ??  I     5:42PM   
0:00.02 python /tmp/hang (python2.2) 
 
On other platforms such as Linux, the program also 
runs, but no process is left hanging around.  
 
I've been able to reproduce the problem on Free/OpenBSD 
only. The problem doesn't appear on IRIX, BSD/OS or  
Linux. 
 
Please let me know if I can do anything more to help 
debug this.  
msg14871 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-03-03 10:57
Logged In: YES 
user_id=45365

Just for fun I tried it on MacOSX, but running from the shall, not qmail, and I don't see any problems there. At least, not if I pass a small input message. If I pass a large input message I get a "broken pipe", but that is to be expected (as true doesn't really read it's input).

My guess is that this is a third-party error, somehow related to how qmail sets up stdin/stdout/stderr (apparently slightly different on Free/OpenBSD and the other systems). Try finding out what stdin/stdout/stderr refer to (with fstat or something). 
msg14872 - (view) Author: Jason R. Mastaler (jasonrm) Date: 2003-03-03 20:05
Logged In: YES 
user_id=85984

> Try finding out what stdin/stdout/stderr refer to 
> (with fstat or something).

Can you be more specific here?

I've attached the output of `fstat -m -p 15437' below
in case that helps.
msg14873 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-03-04 10:33
Logged In: YES 
user_id=45365

Something like fstat is indeed what I was suggesting. The fstat output shows your Python script is getting stdin from a file, and stdout/stderr go to a pipe (the same one). You also have another pip open on filedescriptors 3/4.

Try the same thing on Linux (or another system where things just work) and see if there's a difference in what files are open. 
msg14874 - (view) Author: Jason R. Mastaler (jasonrm) Date: 2003-03-04 18:45
Logged In: YES 
user_id=85984

Attached is lsof output on a Linux box. I had to insert
a sleep after the write but before the close in order to
capture it.
msg14875 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2003-03-04 20:45
Logged In: YES 
user_id=45365

That confirms my suspicion that this is a third-party error: filedescriptors 2 and 3 are readwrite on BSD, write-only on Linux.

A final test to confirm this: on BSD, add code like this just before calling popen():
os.close(1)
os.close(2)
os.open("/dev/null", "w")
os.open("/dev/null", "w")

If this makes the hang go away then the real problem is that whatever program executes your .forward (qmail?) erroneously opens a read-write pipe for your stdout and stderr. You can't do this with pipe() AFAIK, but maybe with socketpair() or something.
msg14876 - (view) Author: Jason R. Mastaler (jasonrm) Date: 2003-03-05 20:09
Logged In: YES 
user_id=85984

Your test indeed does make the hang go away, thanks.
I guess I'll turn my efforts to qmail. Since this doesn't
seem like a Python problem, feel free to close this out.
History
Date User Action Args
2022-04-10 16:07:10adminsetgithub: 38064
2003-02-27 01:12:17jasonrmcreate