This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: windows unicode write is messing up the EOL.
Type: Stage:
Components: Windows Versions: Python 2.4
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: loewis, visike
Priority: normal Keywords:

Created on 2005-09-27 10:43 by visike, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (3)
msg26399 - (view) Author: Istvan Visegradi (visike) Date: 2005-09-27 10:43
User has an Unicode UTF-16 input file and trys to read
it line-by-line and write it back to a new file.
With the read line the program does nothing.
code example: 
...
ln=fi.readline()
fo.write(ln)
...

In linux this code works perfectly with the input
UTF-16 type file.
In windows Python put an extra \x0D into the EOL.

EOL should look like: 0D 00 0A 00
In windows it will be: 0D 00 0D 0A 00

Please inform when correction is available.
msg26400 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2005-09-27 19:57
Logged In: YES 
user_id=21627

Can you provide a small test program that demonstrates the
problem?
I cannot reproduce it:

>>> f=codecs.open("f.txt","w",encoding="utf-16")
>>> f.write(u"Hallo\n")
>>> f.write(u"Welt\r\n")
>>> f.close()
>>> open("f.txt").read()
'\xff\xfeH\x00a\x00l\x00l\x00o\x00\n\x00W\x00e\x00l\x00t\x00\r\x00\n\x00'

This looks right to me.
msg26401 - (view) Author: Istvan Visegradi (visike) Date: 2005-09-29 07:54
Logged In: YES 
user_id=1352468

I made the wrong file open without codecs. With codecs it is
opening OK and works correctly. Sorry for my beginners
problem but in the documentation regarding Unicode is not
too clear for me. (I think the problem is probably with me).
History
Date User Action Args
2022-04-11 14:56:13adminsetgithub: 42418
2005-09-27 10:43:18visikecreate