This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Memory leak in the email package
Type: Stage:
Components: Library (Lib) Versions: Python 2.4
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: barry Nosy List: barry, effbot, ken668, nnorwitz
Priority: normal Keywords:

Created on 2005-12-09 01:03 by ken668, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
memleak.eml ken668, 2005-12-09 01:03 input file for message_from_string
Messages (11)
msg27007 - (view) Author: ken668 (ken668) Date: 2005-12-09 01:03
memory leak in email.message_from_string. This is what
I did to create a leak.

You used the attached file, memleak.eml.

f = open("memleak.eml")
buffer = f.read()
f.close()

# now buffer has the email string
msg = email.message_from_string(buffer)
msg = None # this should free the memory but it doesn't

# The memory that is used in msg is not completely free
msg27008 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-12-11 20:13
Logged In: YES 
user_id=33168

What causes you to believe this is a memory leak?  I ran
this under valgrind and it doesn't report any leaks.
msg27009 - (view) Author: ken668 (ken668) Date: 2005-12-12 15:53
Logged In: YES 
user_id=1400763

I added these three lines after the line "msg=None"

import gc
print "gc.collect()\n\n", gc.collect()
print "gc.garbage\n\n", gc.garbage

If you pipe the output of gc.garbage to a file, you will see
the email message you just sent are still be in the memory.

Everytime I call email.message_from_string, a copy of the
message will be kept in the memory even after I set the
returned value to None.

I tried the same thing with email package email-2.5.tar.gz,
that memory was freed right after I set the variable "msg"
to None.

Thanks

Ken

msg27010 - (view) Author: ken668 (ken668) Date: 2005-12-12 15:59
Logged In: YES 
user_id=1400763

My mistake. I had also called gc.get_objects() too. It was
gc.get_objects() that outputed the content of the email
message. It was not gc.garbage. Sorry about that.
msg27011 - (view) Author: ken668 (ken668) Date: 2005-12-12 16:00
Logged In: YES 
user_id=1400763

My mistake. I had also called gc.get_objects() too. It was
gc.get_objects() that outputed the content of the email
message. It was not gc.garbage. Sorry about that.
msg27012 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-12-15 05:56
Logged In: YES 
user_id=33168

Do you still believe there is a problem or can this report
be closed?
msg27013 - (view) Author: ken668 (ken668) Date: 2005-12-15 06:44
Logged In: YES 
user_id=1400763

Yes, it is a bug. My mistake was when I gave the details, I
should have mentioned gc.get_objects(), not gc.garbage.
gc.get_objects() showed the leaks not gc.garbage.
msg27014 - (view) Author: Fredrik Lundh (effbot) * (Python committer) Date: 2005-12-20 16:44
Logged In: YES 
user_id=38376

This is not a leak (a leak means that the process will
grow if you *repeat* the operation, not that things are
unexpectedly left in the object memory).

And it's not a bug, either; the "copy" of the message
that's returned by get_objects() is the contents of the
buffer variable in the test program.
msg27015 - (view) Author: ken668 (ken668) Date: 2005-12-20 18:37
Logged In: YES 
user_id=1400763

May be you did not look at my test program more closely.
Please look at the statement msg=None. The should removed
all references of the object. The memory will definely grow
if you repeat the operation.  Why don't you run the
email.message_from_string in a loop and see whether the
memory grows. Thanks.
msg27016 - (view) Author: Fredrik Lundh (effbot) * (Python committer) Date: 2005-12-20 19:03
Logged In: YES 
user_id=38376

Nope.

If you do msg=None, you'll release a reference to the
message object, and all direct, non-cyclic members.
Any cyclic references held by the message object (or
created by the email parser) will be removed at a later
time, by the garbage collector.

And even if you add an explicit gc.collect() call to
your program, that won't remove the contents of the
*buffer* variable.

(If you run the program in a loop *without* an explicit
gc.collect() call, the process will appear to grow for
a while, since you're creating objects faster than the
GC can remove them.  Add gc.collect() to the loop, and
the memory use will be rock solid.)
msg27017 - (view) Author: ken668 (ken668) Date: 2005-12-20 22:11
Logged In: YES 
user_id=1400763

I assume the new email package has cyclic references in it.
My program has garabage collection disabled because of
performance issues. That means the email package will cause
memory leak in my program. I will use the old package
instead. Thanks for the help.

History
Date User Action Args
2022-04-11 14:56:14adminsetgithub: 42672
2005-12-09 01:03:44ken668create