This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: non-ascii readline input crashes python
Type: Stage:
Components: None Versions:
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: hyeshik.chang, mwh, phr
Priority: normal Keywords:

Created on 2004-08-14 18:19 by phr, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (6)
msg22067 - (view) Author: paul rubin (phr) Date: 2004-08-14 18:19
Start Python 2.3.4 on Linux command line:

$ python
Python 2.3.4 (#1, Jul 24 2004, 19:45:58)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>>

Enter the string "2×3" (2 times 3, the symbol between 2
and 3 is a multiplication sign, I think Unicode c397 or
maybe 97c3 if I have it byte swapped) and hit return. 
Python reports a syntax error.  Hit Ctrl-P to edit the
line with readline.  Use ctrl-D to delete the first
character of the input line, and Python instantly
crashes with a seg fault.
msg22068 - (view) Author: Hyeshik Chang (hyeshik.chang) * (Python committer) Date: 2004-08-14 21:10
Logged In: YES 
user_id=55188

What locale do you use?  I couldn't reproduce the problem on
any of utf-8, iso8859-1, C, euc-kr locale.
And, can you please attach a backtrace for the segfault point?
msg22069 - (view) Author: paul rubin (phr) Date: 2004-08-15 01:55
Logged In: YES 
user_id=72053

Backtrace follows.  I don't know how to tell my locale.  My
LOCALE shell variable is not set to anything--is there
something else I need to check?  Since the "2×3" displays
properly in my terminal window, I think I'm likely using UTF-8.


(gdb) r
Starting program: /usr/local/bin/python
[New Thread 1074951328 (LWP 6561)]
Python 2.3.4 (#1, Jul 24 2004, 19:45:58)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> 2×3
  File "<stdin>", line 1
    2×3
     ^
SyntaxError: invalid syntax
>>> 2×3
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1074951328 (LWP 6561)]
0x401eb687 in _rl_get_char_len () from /usr/lib/libreadline.so.4
(gdb) bt
#0  0x401eb687 in _rl_get_char_len () from
/usr/lib/libreadline.so.4
#1  0x401eb701 in _rl_compare_chars () from
/usr/lib/libreadline.so.4
#2  0x401e0030 in rl_redisplay () from /usr/lib/libreadline.so.4
#3  0x401de816 in rl_redisplay () from /usr/lib/libreadline.so.4
#4  0x401d2588 in readline_internal_char () from
/usr/lib/libreadline.so.4
#5  0x401d2695 in readline_internal_char () from
/usr/lib/libreadline.so.4
#6  0x401d26ce in readline_internal_char () from
/usr/lib/libreadline.so.4
#7  0x401d219f in readline () from /usr/lib/libreadline.so.4
#8  0x40019074 in call_readline (sys_stdin=0x4212fcc0,
sys_stdout=0x4212fe20,
    prompt=0x40177254 ">>> ")
    at /home/phr/python/Python-2.Modules/readline.c:679
#9  0x080e6110 in PyOS_Readline (sys_stdin=0x4212fcc0,
sys_stdout=0x4212fe20,
    prompt=0x40177254 ">>> ") at Parser/myreadline.c:183
#10 0x08056c47 in tok_nextc (tok=0x817f428) at
Parser/tokenizer.c:683
#11 0x08055d8f in tok_get (tok=0x817f428,
p_start=0xbfffecc4, p_end=0xbfffecc8)
    at Parser/tokenizer.c:1014
#12 0x080559de in PyTokenizer_Get (tok=0x817f428,
p_start=0xbfffecc4,
    p_end=0xbfffecc8) at Parser/tokenizer.c:1414
#13 0x0805543a in parsetok (tok=0x817f428, g=0x8126b88,
start=256,
    err_ret=0xbfffed10, flags=0) at Parser/parsetok.c:125
#14 0x080cb0e2 in PyRun_InteractiveOneFlags (fp=0x4212fcc0,
    filename=0x80f004a "<stdin>", flags=0xbfffedf8) at
Python/pythonrun.c:738
#15 0x080caf3b in PyRun_InteractiveLoopFlags (fp=0x4212fcc0,
    filename=0x80f004a "<stdin>", flags=0xbfffedf8) at
Python/pythonrun.c:690
#16 0x080cc2ee in PyRun_AnyFileExFlags (fp=0x4212fcc0,
    filename=0x80f004a "<stdin>", closeit=0, flags=0xbfffedf8)
    at Python/pythonrun.c:653
#17 0x08054d32 in Py_Main (argc=0, argv=0xbfffee74) at
Modules/main.c:415
#18 0x080548ab in main (argc=1, argv=0xbfffee74) at
Modules/python.c:23
#19 0x420156a4 in __libc_start_main () from /lib/tls/libc.so.6
msg22070 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2004-08-16 12:29
Logged In: YES 
user_id=6656

You should check the $LANG or $LC_ALL env vars.  Or just run
'locale' in the shell.

What version of readline are you using?  Both instinct, and
looking at the backtrace, suggest a readline bug to me.
msg22071 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2004-09-07 16:02
Logged In: YES 
user_id=6656

ping
msg22072 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-04-07 09:07
Logged In: YES 
user_id=6656

Closing for want of response.
History
Date User Action Args
2022-04-11 14:56:06adminsetgithub: 40757
2004-08-14 18:19:09phrcreate