This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: file to store pickled object should be opened in binary mode
Type: Stage:
Components: Documentation Versions: Python 2.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: draghuram, georg.brandl, terry.reedy, tim.peters
Priority: normal Keywords:

Created on 2006-02-27 14:51 by draghuram, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (7)
msg27627 - (view) Author: Raghuram Devarakonda (draghuram) (Python triager) Date: 2006-02-27 14:51
Hi,

To unpickle objects on UNIX that were pickled on
windows, the file should be opened in binary mode for
read and write on windows. It will be nice to have this
mentioned in the doc. Something along these lines will
be good enough:

"Please note that if you are writing the pickled object
to a file on windows, make sure it is opened in binary
mode. Otherwise, the file may not be able to be used to
unpickle on other non-windows platforms".

Thanks,
Raghu. 
msg27628 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2006-03-06 03:36
Logged In: YES 
user_id=593130

For ASCII protocol pickles, intended for human 
readability, one should use text mode to be able to read 
the file, for instance, in Notepad.

For either binary protocol, I would think binary mode 
should be best even to reread on Windows.  If anything is 
added, I would put
"On Windows, open files in a mode (text/binary) that 
matches the protocol."
at the bottom of http://docs.python.org/lib/node64.html

Looking forward to 3.0, when file mode might be more 
significant (text/binary corresponding to unicode/bytes), 
the "for windoews' qualifier might be omitted, but this 
might be premature.
msg27629 - (view) Author: Raghuram Devarakonda (draghuram) (Python triager) Date: 2006-03-06 15:11
Logged In: YES 
user_id=984087


Hi,

The point is that even if ASCII protocol is used on
windows(as I was), the unpickling process would fail on
windows. So the pickle file should be opened in binary mode
on windows even for ASCII protocol. This may make reading
the file in notepad difficult but then, cross platform
support should take precedence, I suppose. Another option
would be for python to do line end translations for ASCII
pickle files. 

Raghu.
msg27630 - (view) Author: Raghuram Devarakonda (draghuram) (Python triager) Date: 2006-03-06 15:12
Logged In: YES 
user_id=984087


There seems to be a mistane in my earlier comment. I meant
the unpickling on *UNIX* fails even if ASCII was used on
windows.

Raghu.
msg27631 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2006-03-06 19:03
Logged In: YES 
user_id=593130

Any text file written with /r/n in Windows can have 
problems when read on *nix or Mac or ???, so this problem 
is not specific to pickle or even to Python.  I am under 
the impression that the new universal newline support was 
intended to fix such problems (when used).  But as a Win-
only user (currently) I have no experience with it.
msg27632 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2006-03-06 19:48
Logged In: YES 
user_id=31435

Pickle files should be opened in binary mode, regardless of
pickle protocol, regardless of platform, and regardless of
whether reading or writing.  That protocol 0 used to be
called "text mode" was an historic mistake, and was given
that name by a Unix-head who wouldn't know the difference
between text and binary mode if it suck its teeth in their
posterior and tore off an entire bloody buttock :-)
msg27633 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-03-07 20:30
Logged In: YES 
user_id=849994

I recently added a note to the docs about this. I it's not
enough, feel free to reopen.
History
Date User Action Args
2022-04-11 14:56:15adminsetgithub: 42952
2006-02-27 14:51:00draghuramcreate