This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: tarfile.extract() may cause file fragmentation on Windows XP
Type: Stage:
Components: Library (Lib) Versions: Python 2.4
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: lars.gustaebel Nosy List: enochjul, georg.brandl, lars.gustaebel
Priority: normal Keywords:

Created on 2006-10-28 21:22 by enochjul, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (4)
msg30402 - (view) Author: Enoch Julias (enochjul) Date: 2006-10-28 21:22
When I use tarfile.extract() to extract all the files
from a large tar archive with lots of files tends to
cause file fragmentation in Windows. Apparently NTFS
cluster allocation interacts badly with such operations
if Windows is not aware of the size of each file.

The solution is to use a combination of the Win32 APIs
SetFilePointer() and SetEndOfFile() before writing to
the target file. This helps Windows choose a contiguous
free space for the file.

I tried it on the 2.6 trunk by calling file.truncate()
(which seems to implement the appropriate calls on
Windows) to set the file size before writing to a file.
It helps to avoid fragmentation for the extracted files
on my Windows XP x64 system.

Can this be added to tarfile to improve its performance
on Windows?
msg30403 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-10-29 08:55
Logged In: YES 
user_id=849994

Can you try to come up with a patch?
msg30404 - (view) Author: Enoch Julias (enochjul) Date: 2006-10-31 05:07
Logged In: YES 
user_id=6071

I submitted patch #1587674 for this, though I am not sure if
it is a good idea to use truncate() for such a purpose.
msg30405 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2007-01-22 19:09
Closed due to lack of interest, see discussion at #1587674.
History
Date User Action Args
2022-04-11 14:56:21adminsetgithub: 44173
2006-10-28 21:22:48enochjulcreate