This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Memory error on AIX in email.Utils._qdecode
Type: Stage:
Components: Extension Modules Versions: Python 2.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: barry, customdesigned, mwh
Priority: normal Keywords:

Created on 2003-10-16 17:40 by customdesigned, last changed 2022-04-10 16:11 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
analfail customdesigned, 2003-10-16 17:40 test data to cause quoted printable decoding failure on AIX
binascii.c.diff barry, 2003-11-21 19:56
Messages (10)
msg18671 - (view) Author: Stuart D. Gathman (customdesigned) Date: 2003-10-16 17:40
The following scriptlet works correctly on RedHat
python2-2.2.3, but gets a MemoryError on AIX
python-2.2.3.  This is the only anomoly for AIX python
I have seen.

import email

fp = open('analfail')
msg = email.message_from_file(fp)
for part in msg.walk():
  if part.get_main_type() == 'text':
    txt = part.get_payload(decode=True)
    #del msg["content-transfer-encoding"]
    msg.set_payload(txt)
fp.close()
print msg.as_string()

It doesn't matter whether the 'del' is there.
msg18672 - (view) Author: Stuart D. Gathman (customdesigned) Date: 2003-10-16 17:43
Logged In: YES 
user_id=142072

I forgot to include the traceback.

$ python t.py
Traceback (most recent call last):
  File "t.py", line 8, in ?
    txt = part.get_payload(decode=True)
  File "/usr/local/lib/python2.2/email/Message.py", line
197, in get_payload
    return Utils._qdecode(payload)
  File "/usr/local/lib/python2.2/quopri.py", line 161, in
decodestring
    return a2b_qp(s, header = header)
MemoryError
msg18673 - (view) Author: Stuart D. Gathman (customdesigned) Date: 2003-10-16 17:47
Logged In: YES 
user_id=142072

It looks like the problem is with a2b_qp in the binascii
extension module.  So I am changing the Category.
msg18674 - (view) Author: Stuart D. Gathman (customdesigned) Date: 2003-10-16 20:00
Logged In: YES 
user_id=142072

Fixed binascii as follows:
--- ./Modules/binascii.c.bms    Mon Mar 17 06:34:43 2003
+++ ./Modules/binascii.c        Thu Oct 16 15:55:34 2003
@@ -1036,7 +1036,7 @@
        /* We allocate the output same size as input, this
is overkill */
        odata = (unsigned char *) calloc(1, datalen);
 
-       if (odata == NULL) {
+       if (odata == NULL && datalen > 0) {
                PyErr_NoMemory();
                return NULL;
        }

This bug will manifest not just on AIX, but any system which
returns NULL from malloc or calloc when allocated size is 0.
msg18675 - (view) Author: Stuart D. Gathman (customdesigned) Date: 2003-10-16 20:14
Logged In: YES 
user_id=142072

Might as well fix b2a_qp while we are at it:
--- ./Modules/binascii.c.aix    Mon Mar 17 06:34:43 2003
+++ ./Modules/binascii.c        Thu Oct 16 16:08:25 2003
@@ -1036,7 +1036,7 @@
        /* We allocate the output same size as input, this
is overkill */
        odata = (unsigned char *) calloc(1, datalen);

-       if (odata == NULL) {
+       if (odata == NULL && datalen > 0) {
                PyErr_NoMemory();
                return NULL;
        }
@@ -1206,7 +1206,7 @@

        odata = (unsigned char *) calloc(1, odatalen);

-       if (odata == NULL) {
+       if (odata == NULL && odatalen > 0) {
                PyErr_NoMemory();
                return NULL;
        }
msg18676 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2003-10-17 11:46
Logged In: YES 
user_id=6656

Why does binascii call calloc directly?  Shouldn't it be
calling PyMem_New?
msg18677 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2003-11-21 19:56
Logged In: YES 
user_id=12800

Or at least PyMem_MALLOC, right?  See attached patch
candidate.  If this fixes the problem for you (and if it
passes Michael's muster), I'll commit this to the 2.3cvs.
msg18678 - (view) Author: Stuart D. Gathman (customdesigned) Date: 2003-11-21 21:08
Logged In: YES 
user_id=142072

Are you sure that the following code doesn't depend on
calloc clearing the memory to 0?  Does PyMem_MALLOC clear to 0?
msg18679 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2003-11-21 21:29
Logged In: YES 
user_id=12800

Whoops, you're right.   I don't think PyMem_MALLOC
guarantees that, so the patch isn't quite right (we'd have
to zero it out ourselves).
msg18680 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2004-05-11 02:05
Logged In: YES 
user_id=12800

Fixed, finally, by replacing the calloc/free calls with
calls to PyMem_Malloc and PyMem_Free.
History
Date User Action Args
2022-04-10 16:11:46adminsetgithub: 39420
2003-10-16 17:40:06customdesignedcreate