This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Patch to support lots of file descriptors
Type: Stage:
Components: Interpreter Core Versions: Python 2.4
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: loewis, svenberkvens
Priority: normal Keywords: patch

Created on 2006-02-19 16:34 by svenberkvens, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
patch-Include::Python.h svenberkvens, 2006-02-19 16:34 Patch for the Include/Python.h file
Messages (6)
msg49540 - (view) Author: Sven Berkvens-Matthijsse (svenberkvens) Date: 2006-02-19 16:34
Off-the-shelf Python on FreeBSD (and probably other
OS'es as well) has a limit on the number of file
descriptors that select can use. This is limited by the
size of the fdset structures.

On FreeBSD, and probably in many OS'es, a user can set
the size of the fdset structures if he defines
FD_SETSIZE before he includes sys/types.h.

My patch sets the FD_SETSIZE to a substantial number.
msg49541 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-02-27 21:54
Logged In: YES 
user_id=21627

That patch is wrong: you should only change the value of FD_SETSIZE on systems 
where this actually works. Unfortunately, I cannot think of a method to determine 
this reliably.
msg49542 - (view) Author: Sven Berkvens-Matthijsse (svenberkvens) Date: 2006-02-28 08:50
Logged In: YES 
user_id=175845

Is it perhaps possible to add a test to the configuring
process that tests whether something like:

#define FD_SETSIZE 10240
#include <sys/types.h>
#include <unistd.h>

int main(int argc, char **argv)
{
#if FD_SETSIZE == 10000
    return 0;
#else
    return 1;
#endif
}

can be preprocessed, compiled and run correctly? This will
most probably fail on systems where FD_SETSIZE is not
settable, and succeed on systems where it is.

Thanks for responding,
Sven
msg49543 - (view) Author: Sven Berkvens-Matthijsse (svenberkvens) Date: 2006-02-28 08:51
Logged In: YES 
user_id=175845

Oops, 10000 should obivously match 10240 here...

Sven
msg49544 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-02-28 22:48
Logged In: YES 
user_id=21627

This test is not good enough. On systems where FD_SETSIZE is not "settable", 
it still is a macro, and setting it will succeed. The problem is that on these 
systems, select(2) might refuse to process more than FD_SETSIZE file 
descriptors, and/or fd_set might just buffer-overrun.

In these cases, the cure would be worse than the desease: you would be able 
to pass that many file descriptors to select, but the system would never tell 
you when they are ready.

If you want to use many file descriptors, just use poll(2) instead of select(2).
msg49545 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-02-14 10:12
Apparently, there is no activity on this patch, so I'm eventually rejecting it.
History
Date User Action Args
2022-04-11 14:56:15adminsetgithub: 42924
2006-02-19 16:34:05svenberkvenscreate