This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: mmap module leaks file descriptors on UNIX
Type: Stage:
Components: Extension Modules Versions: Python 2.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: kdart, majid, nnorwitz
Priority: normal Keywords:

Created on 2006-02-03 02:25 by majid, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
mmap_leak.py majid, 2006-02-03 02:27 test case
mmap_patch.txt majid, 2006-02-03 02:44 patch for fd leak in mmapmodule on UNIX
Messages (6)
msg27412 - (view) Author: Fazal Majid (majid) Date: 2006-02-03 02:25
On UNIX, mmapmodule.c makes a call to dup(2) prior to
mapping the file descriptor into memory (oddly enough,
it doesn't map the new fd resulting from dup(), but the
old one).

The close method does not release this file descriptor,
however, leading to a leak. If mmap is used repeatedly,
the process will eventually run out of file
descriptors, as can easily be shown with this test case:

import sys, os, mmap

for i in xrange(2000):
  f = os.open('/dev/zero', os.O_RDWR)
  m = mmap.mmap(f, 10000)
  os.close(f)
  a = m[0:100]
  m[100:200] = 'F' * 100
  m.close()
msg27413 - (view) Author: Fazal Majid (majid) Date: 2006-02-03 02:27
Logged In: YES 
user_id=110477

SourceForge munged the formatting of the test case, so I am
attaching a copy as well.
msg27414 - (view) Author: Fazal Majid (majid) Date: 2006-02-03 02:44
Logged In: YES 
user_id=110477

The following patch closes the file descriptor when the mmap
object's close method is called, and seems to resolve this
problem.
msg27415 - (view) Author: Keith Dart (kdart) Date: 2006-02-03 14:51
Logged In: YES 
user_id=16527

I would also like to add that the following no longer works
in Python 2.4:

_buf = mmap.mmap(-1, 8192,
flags=mmap.MAP_PRIVATE|mmap.MAP_ANONYMOUS,
prot=mmap.PROT_READ|mmap.PROT_WRITE )

This is because the fd value of -1 raises an exception
because it is being dup-ed inside the mmap module. But the
fd is ignored in anonymous mmaps and does not need to be
dup-ed. This worked in older Python (2.3). Is the dup() call
really necessary?
msg27416 - (view) Author: Fazal Majid (majid) Date: 2006-02-03 17:43
Logged In: YES 
user_id=110477

From my reading of the source, the fd is kept around just to
support the mmap.size() and mmap.resize() methods. I tend to
agree, tying down a fd is wasteful.
msg27417 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2006-02-05 03:25
Logged In: YES 
user_id=33168

A similar fix was checked in to 41368 (2.4) and 41366 on 01
Nov 2005.  Thanks.  There shouldn't be a problem using 2.4
SVN HEAD or 2.5 HEAD.
History
Date User Action Args
2022-04-11 14:56:15adminsetgithub: 42864
2006-02-03 02:25:55majidcreate