This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: file.tell() returns illegal value on Windows
Type: Stage:
Components: Windows Versions: Python 2.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: georg.brandl, pterk, tim.peters, tom_goddard
Priority: high Keywords:

Created on 2006-01-04 01:53 by tom_goddard, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
tellbug.py tom_goddard, 2006-01-04 01:56
Messages (5)
msg27226 - (view) Author: Tom Goddard (tom_goddard) Date: 2006-01-04 01:53
The file tell() method returns an illegal value that causes an exception 
when passed to seek().  This happens on Windows when a file that 
contains unix-style newlines '\n' is opened and read in text mode 'r'.  
Below is code that produces the problem on Windows 2.4.2 in an IDLE 
shell.

The bug does not occur when using mode 'rU' or 'rb'.  But I expect 
correct behaviour with mode 'r'.  My understanding is that 'rU' 
translates line endings to '\n' in the returned string while mode 'r' still 
correctly reads the lines using readline(), recognizing all 3 common 
endline conventions, but does not translate (ie includes \n\r or \r or \n 
in returned string).

The error in the tell() return value depends on how long the file is.  
Changing the 'more\n'*10 line in the example code will cause different 
incorrect return values.

Previous bug reports have mentioned problems with tell/seek when 
using file iterators, the file.next() method and the "for line in file:" 
construct.  This bug is different and just involves readline() and tell() 
with mode 'r'.

Example code tellbug.py follows:

wf = open('testdata', 'wb')
wf.write('01234\n56789\n'+ 'more\n'*10)
wf.close()

f = open('testdata', 'r')
f.readline()
t = f.tell()
print t
f.seek(t)

-------

Running gives:

>>> execfile('tellbug.py')
-5

Traceback (most recent call last):
  File "<pyshell#14>", line 1, in -toplevel-
    execfile('tellbug.py')
  File "tellbug.py", line 9, in -toplevel-
    f.seek(t)
IOError: [Errno 22] Invalid argument
msg27227 - (view) Author: Peter van Kampen (pterk) Date: 2006-01-16 03:24
Logged In: YES 
user_id=174455

I have done some additional research on this.

It appears the bug is in ftell of the windows c-library and
it seems to be a 'known problem'. See:

http://www.mathworks.com/access/helpdesk/help/techdoc/ref/ftell.html

More here:
http://www.cygwin.com/faq/faq.api.html
http://gnuwin32.sourceforge.net/compile.html (search for
ftell in both cases)

Also see:

http://sourceforge.net/tracker/index.php?func=detail&aid=1381717&group_id=5470&atid=105470

So even though the file isn't opened with 'rt' I gather from
Tim Peter's remark about 't' on windows that the default is
't' anyway unless you really try. 

So this example is *exactly* what the matlab-people found.
That is nice to know I guess but it doesn't really help to
solve the problem.

It seems to me there are three solutions.

1. ftell could be fixed for unix-files on windows.* Not very
likely to happen soon. 
2. Python could special-case this for this specific
scenario. Way beyond my capabilities and probably too much
hassle in any case especially since the 'fix' is so easy
(just open with 'rb').
3. Leave it as is but document it. I have submitted a
(doc-)patch 1407021
(http://sourceforge.net/tracker/index.php?func=detail&aid=1407021&group_id=5470&atid=305470)
for the library reference (libstdtypes.tex) that reads:

Please note that tell() can return illegal values (after an
fgets()) on Windows when reading files with UNIX-style
line-endings. Use 'rb'-mode to circumvent this
problem.

* MS provides this note in the compatibility section of the
c-runtime libraries: "Note: In this version of Visual C++,
UNIX compatibility information has been removed from the
function descriptions."

In case your wondering what 'this version' is: VS.NET and VS
6.0. Apperantly there were no other 'previous versions'. 
msg27228 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-01-20 09:15
Logged In: YES 
user_id=1188172

Documented as a Windows bug in revisions 42102,42103.
msg27229 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2006-01-20 15:35
Logged In: YES 
user_id=31435

Well, it's not a Windows bug.  Platforms are allowed to
insist on just about any restrictions they like for text
files, and it's on the user's head if they open a non-text
file in text mode.  The C standard leaves all behavior
undefined in such cases.  Text files on Windows have \r\n
line ends, and Windows isn't required to do anything in
particular if a file with even one \n line end is opened in
text mode.  It so happens that a lot of things work the same
way they work on Linux anyway, but not all.
msg27230 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-01-20 17:04
Logged In: YES 
user_id=1188172

Acknowledged. I didn't use the word "bug" in the note anyway.
History
Date User Action Args
2022-04-11 14:56:14adminsetgithub: 42757
2006-01-04 01:53:18tom_goddardcreate