This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: asyncore.file_dispatcher should not take fd as argument
Type: Stage:
Components: Library (Lib) Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: josiahcarlson Nosy List: dhoulder, giampaolo.rodola, jhylton, josiahcarlson
Priority: normal Keywords:

Created on 2004-09-10 02:14 by dhoulder, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
file_dispatcher_bug.py dhoulder, 2004-09-10 02:14 Demonstration of problem and a solution
Messages (6)
msg22414 - (view) Author: david houlder (dhoulder) Date: 2004-09-10 02:14
Only relevant to posix.
asyncore.file_dispatcher closes the file descriptor
behind the file object, and not the file object itself.
When another file gets opened, it gets the next
available fd, which on posix, is the one just released
by the close.

Tested on python 2.2.3 on RedHat Enterprise Linux 3 and
python 2.2.1 on HP Tru64 unix. See attached script for
details and a solution. 'case 1' should show the
problem regardless of the garbage collection strategy
in python. 'case 2' relies on the file object being
closed as soon as the last reference to it disappears,
which seems to be the (current?) behaviour.

[djh900@dh djh900]$ python file_dispatcher_bug.py
case 1:
 (Read 'I am the first pipe\n' from pipe)
 (pipe closing. fd== 3 )
 (Read '' from pipe)
firstPipe.read() says 'I am the second pipe\n'
firstPipe.fileno()== 3
secondPipe.fileno()== 3

case 2:
 (Read 'I am the first pipe\n' from pipe)
 (pipe closing. fd== 3 )
 (Read '' from pipe)
secondPipe.fileno()== 3
dispatcher.secondPipe.read() says
Traceback (most recent call last):
  File "file_dispatcher_bug.py", line 77, in ?
    print "dispatcher.secondPipe.read() says",
repr(dispatcher.secondPipe.read())
IOError: [Errno 9] Bad file descriptor
[djh900@dh djh900]$ 
msg22415 - (view) Author: Jeremy Hylton (jhylton) (Python triager) Date: 2004-11-07 15:23
Logged In: YES 
user_id=31392

I'm not sure whether you propose a change to asyncore or are
describing a pattern that allows you to use a pipe with it
safely.  And, looking at your code more closely, I think
pipe is confusing, because you're not talking about
os.pipe() right?
msg22416 - (view) Author: david houlder (dhoulder) Date: 2004-11-17 23:43
Logged In: YES 
user_id=1119185

In an ideal world I'd propose replacing the guts of
file_wrapper() and file_dispatcher() by my pipe_wrapper()
and PipeDispatcher(), since the general problem of closing
the file descriptor behind the python object applies to all
python objects that are based on a file descriptor, not just
pipes.

So, yes, probably best not to call it pipe_dispatcher(). And
I guess file_dispatcher() may be in use by other peoples'
code and changing it to take a file object rather than an fd
will break that.

Maybe file_dispatcher.__init__() could be changed to take
either an integer file descriptor or a file object as it's
argument, and behave like the current file_dispatcher() when
given an fd, and like pipe_dispatcher() when given a
file-like object (i.e. any object with fileno() and close()
methods will probably be enough). I'm happy to whip up an
example if people think that's a good idea.
msg22417 - (view) Author: Josiah Carlson (josiahcarlson) * (Python triager) Date: 2007-01-06 22:48
I believe that asyncore.file_dispatcher taking a file descriptor is fine.  The problem is that the documentation doesn't suggest that you os.dup() the file handle so that both the original handle (from a pipe, file, etc.) can be closed independently from the one being used by the file_dispatcher.  In the case of socket.makefile(), the duplication is done automatically, so there isn't the same problem.

My suggested fix would be to accept a file or a file handle.  For files, we first get its file number via the standard f.fileno(), and with that, or the handle we are provided, we os.dup() the handle.
msg22418 - (view) Author: david houlder (dhoulder) Date: 2007-01-12 01:58
Yep, dup()ing the fd and using that for the lifetime of the object sounds like a good, simple fix. Wish I'd thought of it :-)
msg69222 - (view) Author: Josiah Carlson (josiahcarlson) * (Python triager) Date: 2008-07-03 18:14
Fixed in trunk, will be fixed in 3.0 this weekend.
History
Date User Action Args
2022-04-11 14:56:06adminsetgithub: 40890
2008-07-03 18:14:03josiahcarlsonsetstatus: open -> closed
resolution: fixed
messages: + msg69222
2008-03-20 00:56:20giampaolo.rodolasetnosy: + giampaolo.rodola
2004-09-10 02:14:51dhouldercreate