This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: marshal may crash on truncated input
Type: Stage:
Components: Library (Lib) Versions: Python 2.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: effbot Nosy List: ajaksu2, effbot, mwh, nnorwitz
Priority: normal Keywords:

Created on 2005-02-14 11:14 by effbot, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
marshal-safety.diff mwh, 2005-04-19 14:58 fix attempt no. 1
Messages (5)
msg24297 - (view) Author: Fredrik Lundh (effbot) * (Python committer) Date: 2005-02-14 11:14
marshal doesn't behave well on truncated or otherwise 
malformed input.  here's a short demo script, from a 
recent comp.lang.python thread:

:::

the problem is that the following may or may not reach 
the "done!" statement, somewhat depending on python 
version, memory allocator, and what data you pass to 
dumps.

import marshal

data = marshal.dumps((1, 2, 3, "hello", 4, 5, 6))

for i in range(len(data), -1, -1):
    try:
        print marshal.loads(data[:i])
    except EOFError:
        print "EOFError"
    except ValueError:
        print "ValueError"

print "done!"

(try different data combinations, to see how far you get 
on your platform...)

fixing this should be relatively easy, and should result in 
a safe unmarshaller (your application will still have to 
limit the amount of data fed into load/loads, of course).

:::

(also note that marshal may raise either EOFError or 
ValueError exceptions, again somewhat depending on 
how the file is damaged.  a little consistency wouldn't 
hurt, but I'm not sure if/how this can be fixed...)
msg24298 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-04-19 14:58
Logged In: YES 
user_id=6656

I think the attached fixes this example, and another involving marshalled 
sets.

I spent a while feeding random data to marshal a few days ago and found 
that the commonest problem was attempting to allocate really huge 
sequences.  Also, the TYPE_STRINGREF is horribly fragile, but I'm 
hoping Martin's going to fix that (he has a bug filed against him, anyway).

Can you test/check it in?  My marshal.c has rather a lot of local changes.

Also, a test suite entry would be nice...
msg24299 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-05-27 09:36
Logged In: YES 
user_id=6656

Ping!
msg24300 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-09-30 05:19
Logged In: YES 
user_id=33168

This works in CVS, but still not in 2.4.  I guess the patch
didn't get backported.
msg81508 - (view) Author: Daniel Diniz (ajaksu2) * (Python triager) Date: 2009-02-09 22:14
Reported as fixed by nnorwitz in msg24300, 2.4 is not supported anymore.
History
Date User Action Args
2022-04-11 14:56:09adminsetgithub: 41576
2009-02-10 03:38:23benjamin.petersonsetstatus: open -> closed
resolution: fixed
2009-02-09 22:14:18ajaksu2setnosy: + ajaksu2
messages: + msg81508
2005-02-14 11:14:22effbotcreate