This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: 8-bit string literal with iso8859 coding => crash
Type: Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: loewis Nosy List: loewis, lpd, nnorwitz, rozhnov, zgoda
Priority: normal Keywords:

Created on 2004-01-01 17:30 by lpd, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
out.py lpd, 2004-01-01 17:30 2-line 'program' that causes the crash
valgrind.out nnorwitz, 2004-01-01 19:53
out.py nnorwitz, 2004-01-01 19:54 shorter program (string) that still crashes
Messages (7)
msg19503 - (view) Author: L. Peter Deutsch (lpd) Date: 2004-01-01 17:30
Environment:
Python 2.3.3, built from unmodified out-of-the-tarball
sources
(./configure, make, make install)
Red Hat Linux 7.3, SMP build from source (2 Athlon CPUs)

Symptom:
python out.py
Segmentation fault
msg19504 - (view) Author: L. Peter Deutsch (lpd) Date: 2004-01-01 17:32
Logged In: YES 
user_id=8861

'make test' ran with no problems.
msg19505 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2004-01-01 19:53
Logged In: YES 
user_id=33168

Looks like there may be an off-by-1 error in the
implementation of PEP263.  Attached is a valgrind report. 
Martin, do you have any ideas?
msg19506 - (view) Author: Jarek Zgoda (zgoda) Date: 2004-01-02 08:58
Logged In: YES 
user_id=92222

Python 2.3.2 (#49, Oct  2 2003, 20:02:00) [MSC v.1200 32 bit
(Intel)] on win32
Windows 2kSP4
Effect is the same - Python crashes with "memory can not be
written".
msg19507 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2004-02-16 14:44
Logged In: YES 
user_id=21627

It's much worse than off-by-one. fp_readl reads size-1 bytes
from the input stream, reencodes them as UTF-8, and tries to
fit them into size bytes. In the example, it tries to put
8860 bytes into space for 8192 bytes.

I believe fp_readl should extend the buffer, using code
similar to tok_nextc.

Alternatively, fp_readl could, conservatively, request only
size/4 bytes from the underlying input stream, knowing that,
for most encodings, converting to UTF-8 will increase the
size at worst by a factor of 4. The assertion that the
result still fits should then become a FatalError.
msg19508 - (view) Author: Mike Rozhnov (rozhnov) Date: 2004-04-22 12:00
Logged In: YES 
user_id=1026824

Is it possible to fix this bug before Python 2.4 release?

For now choices when using 8-bit literals are:
 - always receive warning about PEP-263
 - possible crash after editing any source module with 8-bit
literals in any time

Sorry. I can't fix this bug myself (don't have and use
Visual C).
msg19509 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2005-08-25 07:30
Logged In: YES 
user_id=21627

This bug is now fixed in the CVS, for 2.5 and 2.4.2.
History
Date User Action Args
2022-04-11 14:56:01adminsetgithub: 39748
2004-01-01 17:30:32lpdcreate