This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python polls unnecessarily every 0.1 second when interactive
Type: Stage:
Components: Library (Lib) Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: akuchling Nosy List: akuchling, mwh, nnorwitz, richardb
Priority: critical Keywords:

Created on 2006-09-05 14:42 by richardb, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
readline_poll.patch richardb, 2006-09-05 14:42 Patch to resolve problem.
Messages (11)
msg29759 - (view) Author: Richard Boulton (richardb) Date: 2006-09-05 14:42
When python is running an interactive session, and is
idle, it calls "select" with a timeout of 0.1 seconds
repeatedly.  This is intended to allow PyOS_InputHook()
to be called every 0.1 seconds, but happens even if
PyOS_InputHook() isn't being used (ie, is NULL).

To reproduce:
 - start a python session
 - attach to it using strace -p PID
 - observe that python repeatedly

This isn't a significant problem, since it only affects
idle interactive python sessions and uses only a tiny
bit of CPU, but people are whinging about it (though
some appear to be doing so tongue-in-cheek) and it
would be nice to fix it.

The attached patch (against Python-2.5c1) modifies the
readline.c module so that the polling doesn't happen
unless PyOS_InputHook is not NULL.
msg29760 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2006-09-07 13:24
Logged In: YES 
user_id=11375

Original report:
http://perkypants.org/blog/2006/09/02/rfte-python

This is tied to the version of readline being used; the
select code is only used if HAVE_RL_CALLBACK is defined, and
a comment in Python's configure.in claims it's only defined
with readline 2.1.  Current versions of readline are 4.3 and
5.1; are people still using such an ancient version of readline?
msg29761 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2006-09-07 14:02
Logged In: YES 
user_id=11375

Recent versions of readline can still support callbacks if
READLINE_CALLBACK is defined, so I could test the patch by
running 'CFLAGS=-DREADLINE_CALLBACK' and re-running configure.

Applied as rev. 51815 to the trunk, so the fix will be in
Python 2.6.  The 2.5 release manager needs to decide if it
should be applied to the 2.5 branch.
msg29762 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2006-09-07 14:12
Logged In: YES 
user_id=6656

I'd be cautious about applying this to 2.5: we could end up with the same 
problem currently entertaining python-dev, i.e. a signal gets delivered to a non-
main thread but the main thread is sitting in a select with no timeout so any 
python signal handler doesn't run until the user hits a key.

HAVE_READLINE_CALLBACK is defined when readline is 2.1 *or newer* I think...
msg29763 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2006-09-07 14:17
Logged In: YES 
user_id=11375

HAVE_READLINE_CALLBACK was not defined with readline 5.1 on
Ubuntu Dapper, until I did the configure/CFLAG trick.

I didn't think of a possible interaction with signals, and
will re-open the bug while trying to work up a test case.
msg29764 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2006-09-07 14:38
Logged In: YES 
user_id=11375

On looking at the readline code, I think this patch makes no
difference to signals.

The code in readline.c for the callbacks looks like this:

has_input = 0;
while (!has_input) {
  ...
  has_input = select.select(rl_input);
}

if (has_input > 0) {read character}
elif (errno == EINTR) {check signals}

So I think that, if a signal is delivered to a thread and
select() in the main thread doesn't return EINTR, the old
code is just as problematic as the code with this patch. 
The (while !has_input) loop doesn't check for signals at all
as an exit condition.

I'm not sure what to do at this point.  I think the new code
is no worse than the old code with regard to signals. Maybe
this loop is buggy w.r.t. to signals, but I don't know how
to test that.
msg29765 - (view) Author: Richard Boulton (richardb) Date: 2006-09-08 09:34
Logged In: YES 
user_id=9565

HAVE_READLINE_CALLBACK is defined by configure.in whenever
the readline library on the platform supports the
rl_callback_handler_install() function.  I'm using Ubuntu 
Dapper, and have libreadline 4 and 5 installed (more
precisely, 4.3-18 and 5.1-7build1), but only the -dev
package for 5.1-7build1.  "info readline" describes
rl_callback_handler_install(), and configure.in finds it, so
I'm surprised it wasn't found on akuchling's machine.

I agree that the code looks buggy on platforms in which
signals don't necessarily get delivered to the main thread,
but looks no more buggy with the patch than without.
msg29766 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2006-09-08 13:12
Logged In: YES 
user_id=11375

That's exactly my setup.  I don't think there is a -dev
package for readline 4.

I do note that READLINE_CALLBACKS is defined in
/usr/include/readline/rlconf.h, but Python's readline.c
doesn't include this file, and none of the readline headers
include it.  So I don't know why you're finding the function!
msg29767 - (view) Author: Richard Boulton (richardb) Date: 2006-09-08 14:30
Logged In: YES 
user_id=9565

I'm finding the function because it's defined in the
compiled library - the header files aren't examined by
configure when testing for this function.  (this is because
configure.in uses AC_CHECK_LIB to check for
rl_callback_handler_install, which just tries to link the
named function against the library).  Presumably, rlconf.h
is the configuration used when the readline library was
compiled, so if READLINE_CALLBACKS is defined in it, I would
expect the relevant functions to be present in the compiled
library.

In any case, this isn't desperately important, since you've
managed to hack around the test anyway.
msg29768 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-01-17 06:47
I'm fine if this patch is applied.  Since it was applied to trunk, it seems like it might as well go into 2.5.1 as well.  I agree it's not that high priority, but don't see much reason to wait either.  OTOH, I won't lose sleep if it's not applied, so do what you think is best.
msg29769 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2007-01-22 16:10
Applied to 2.5.1 in rev. 53516.
History
Date User Action Args
2022-04-11 14:56:20adminsetgithub: 43941
2006-09-05 14:42:57richardbcreate