On Windows XP, reading a file produced by Windows XP with universal line endings produces twice as many lines!

Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> open("winlineend").read()
'Hello\r\n'
>>> open("winlineend", "rU").read()
'Hello\n\n'


I would expect the last to give "Hello\n". 

I created a file "test.txt" with notepad whose contents are "hello\r\n". Both open().read() and open("rU").read() returned 'hello\n'. I tested with both 2.5 and 2.5.1 (installed using installers from python.org) and the result is same on both. Can you elaborate your test case more? How is this file "winlineend" created?

I create the file as follows: 

>>> import os
>>> file = open("test.txt", "w")
>>> file.write("Hello" + os.linesep)
>>> file.close()
>>> open("test.txt").read()
'Hello\r\n'
>>> open("test.txt", "rU").read()
'Hello\n\n'

> file.write("Hello" + os.linesep)

The line separator for a file opened in *text*mode* is *always* \n in Python code. It gets converted to os.linesep when you write the file; and conversely, os.linesep get translated into a single \n when you read the file.
On Windows, os.linesep is "\r\n". The argument to file.write above is  "Hello\r\n". That "\n" gets translated into "\r\n" when it is written, so the actual file contents will be "Hello\r\n\n"

In short: if you open a file in text mode (the default) *don't* use os.linesep to terminate lines. Only use os.linesep when writing text files open in binary mode (and that's not a common case).

I see. This seems like something of a gotcha to me: is it documented anywhere? Seems to make os.linesep not very useful in fact, especially if "\n" will work fine everywhere and is shorter. 

It isn't at all obvious that file.write(os.linesep) will in fact write two lines...

Yes, it's not so obvious, and the same question was posted a few days ago on comp.lang.python.
I'll review the documentation and make it more clearly stated.

To the OP: what does "open("winlineend", "rb").read()" return?

It returns "Hello\r\r\n" (not "Hello\r\n\n" as was suggested earlier) Don't know quite how it does this from simply writing os.linesep.

Okay, that settles it.

When you write "\r\n" to a Windows text file, it writes "\r\r\n"
(since "\n" is converted to "\r\n"). When universal newline mode
sees that, it thinks of it as Mac-lineend followed by a Windows-
lineend.

The doc fix has been committed, closing this one as "won't fix".

Which doc has been changed? Can I review it somewhere? 

The doc for os.linesep. The change is in Subversion, and will be at docs.python.org/dev when the build script runs the next time.