This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: files with long lines and an encoding crash
Type: Stage:
Components: Unicode Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mhammond Nosy List: lemburg, loewis, mhammond
Priority: release blocker Keywords:

Created on 2003-01-09 13:32 by mhammond, last changed 2022-04-10 16:06 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
tokenizer.c.patch mhammond, 2003-01-09 13:34 patch
crash.py mhammond, 2003-01-09 13:35 test case
Messages (6)
msg13951 - (view) Author: Mark Hammond (mhammond) * (Python committer) Date: 2003-01-09 13:32
If a source file declares an encoding, but has a line >
512 characters, a buffer overflows.  debug builds
assert, but release builds trudge along.

Problem is that when reading an encoded line, the
'size' param is not passed to the underlying readline
function.

Attaching a test case and a patch.
msg13952 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2003-01-09 13:54
Logged In: YES 
user_id=38388

Assigned to Martin since he wrote the source code encoding
support.
msg13953 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2003-01-14 08:25
Logged In: YES 
user_id=21627

I can't reproduce this. What specific assertion failure do
you get, on what system (Windows, I guess)? Can you please
attach the test case?
msg13954 - (view) Author: Mark Hammond (mhammond) * (Python committer) Date: 2003-01-14 11:21
Logged In: YES 
user_id=14198

Test case already attached, as I mentioned.  See crash.py. 
Sorry, I should have mentioned the assertion failure is
inside the debug crt, which detects the 512 byte buffer
being overwritten due to the lack of a 'size' arg.
msg13955 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2003-01-14 12:16
Logged In: YES 
user_id=21627

I see. The patch is fine, please apply it.
msg13956 - (view) Author: Mark Hammond (mhammond) * (Python committer) Date: 2003-01-14 23:18
Logged In: YES 
user_id=14198

Thanks.

/cvsroot/python/python/dist/src/Parser/tokenizer.c,v  <-- 
tokenizer.c
new revision: 2.71; previous revision: 2.70
History
Date User Action Args
2022-04-10 16:06:07adminsetgithub: 37746
2003-01-09 13:32:21mhammondcreate