This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: raw_input problem with readline and UTF8
Type: Stage:
Components: Library (Lib) Versions: Python 2.3
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: airog, ajaksu2, donut, mwh
Priority: normal Keywords:

Created on 2005-01-15 18:10 by airog, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (4)
msg60631 - (view) Author: Casey Crabb (airog) Date: 2005-01-15 18:10
Backspace doesn't remove all bytes of a multi-byte
UTF-8 character.

To reproduce the problem:
$ export LANG=en_US.UTF-8
$ python
Python 2.3.4 (#1, Jun 11 2004, 16:35:29) 
[GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r3, ssp-3.3-7,
pie-8.5.3)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import readline
>>> raw_input() # ä, return
ä
'\xc3\xa4'
>>> raw_input() # ä, backspace, return

'\xc3'
>>> 


A small C program does not have the same problem:

#include <stdlib.h>
#include <stdio.h>
#include <readline/readline.h>
#include <readline/history.h>

void pprint(const char *s);

int main(void) {
	char *line;

	for (;;) {
		line = readline("> ");
		if (!line)
			break;
		pprint(line);
		free(line);
	}

	return 0;
}

void pprint(const char *s) {
	while (*s) {
		if (isprint(*s))
			putchar(*s);
		else
			printf("\\x%x", *s & 0xff);
		s++;
	}
	putchar('\n');
}
msg60632 - (view) Author: Matthew Mueller (donut) Date: 2005-02-07 07:36
Logged In: YES 
user_id=65253

Hi, it looks like this might be the same problem already
fixed in "[ 914291 ] Fix readline for utf-8 locales", but
your python version is from before the fix went in.  Can you
try again with python 2.4, or 2.3.5 (when it is released, or
the release23-maint branch of cvs)?

Also, you could check if python's readline.so is linked to
an older version of libreadline.
msg60633 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-04-07 09:07
Logged In: YES 
user_id=6656

Is this still relavent?
msg81511 - (view) Author: Daniel Diniz (ajaksu2) * (Python triager) Date: 2009-02-09 22:39
Please close this, reportedly fixed and no further info from OP.
History
Date User Action Args
2022-04-11 14:56:09adminsetgithub: 41445
2009-02-10 02:10:00benjamin.petersonsetstatus: open -> closed
resolution: out of date
2009-02-09 22:39:25ajaksu2setnosy: + ajaksu2
messages: + msg81511
2005-01-15 18:10:58airogcreate