This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: for lin in file: file.tell() tells wrong
Type: Stage:
Components: Documentation Versions: Python 2.2
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, facundobatista, hdima, jackjansen, jlgijsbers, jvr, loewis, tim.peters
Priority: normal Keywords:

Created on 2002-11-29 09:13 by hdima, last changed 2022-04-10 16:05 by admin. This issue is now closed.

Messages (13)
msg13474 - (view) Author: Dmitry Vasiliev (hdima) Date: 2002-11-29 09:13
Consider following piece of code:

f = file("test.txt", "rb")
pos = f.tell()
for line in f:
	print "%u: '%s'" % (pos, line)
	pos = f.tell()

During the code execution we have following result:

0 'Line 1'
63 'Line 2'
63 'Line 3'
...
63 'Line 9'

However, following piece of code works fine:

f = file("test.txt", "rb")
while True:
	pos = f.tell()
	line = f.readline()
	if line == "":
		break
	print "%u: '%s'" % (pos, line)

It prints:

0 'Line 1'
7 'Line 2'
14 'Line 3'
...
56 'Line 9'

It seems a file iterator makes file.tell() to tell
positions of some internal blocks used by the iterator.
msg13475 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-11-29 14:30
Logged In: YES 
user_id=21627

Would you like to investigate this further? There is no need 
to, but if you find a solution and a patch, there is a better 
chance that this is fixed before 2.3.
msg13476 - (view) Author: Dmitry Vasiliev (hdima) Date: 2002-11-29 14:46
Logged In: YES 
user_id=388573

I'll try to dig in.
msg13477 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2002-11-29 15:42
Logged In: YES 
user_id=31435

"for x in file" does do its own chunking under the covers, for 
speed, and seek() and tell() are indeed not to be used on a 
file being iterated over in this way.
msg13478 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2002-11-29 16:01
Logged In: YES 
user_id=21627

Shouldn't tell raise an exception then?
msg13479 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2002-11-29 16:57
Logged In: YES 
user_id=31435

Possibly.  Maybe something fancier could be done too, to 
give "expected" results. although I don't know how to 
approach that on Windows for text-mode files (byte 
arithmetic on seek/tell results doesn't work at all for 
those).  I'd take it to Python-Dev.
msg13480 - (view) Author: Jack Jansen (jackjansen) * (Python committer) Date: 2002-11-29 22:07
Logged In: YES 
user_id=45365

Actually, all file operations behave the same (well, all: I tried one: readline():-): they behave as if the whole file has been read. I.e. file.readline() within a "for line in file" will return an empty string.

If a file iterator behaves as if it has read the complete file at once on instantiation (never mind what it actually does: if it *behaves* as if this happens) I think this should just be documented and that's it.
msg13481 - (view) Author: Dmitry Vasiliev (hdima) Date: 2002-12-04 11:24
Logged In: YES 
user_id=388573

File.next() uses readahead buffer (8192 bytes for now).
Calling file.seek() drops the buffer, but other operations
don't. I think it's possible for file.tell() to return right
result (I'll try to make a patch). But all these quirks
should be documented.
msg13482 - (view) Author: Just van Rossum (jvr) * (Python triager) Date: 2003-01-05 21:25
Logged In: YES 
user_id=92689

This bug is very similar to #524804, which was closed as
"won't fix". Unless it's recategorized as a documentation
bug, I suggest to close this one as a duplicate.
msg13483 - (view) Author: Dmitry Vasiliev (hdima) Date: 2003-01-08 10:17
Logged In: YES 
user_id=388573

I agree. Unfortunately a small patch doesn't work here. :-(
But I suggest to recategorize it as a documentation bug.
msg13484 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2003-05-22 04:09
Logged In: YES 
user_id=357491

As suggested by Just and Dimtry I am reclassifying this as a doc bug.
msg13485 - (view) Author: Johannes Gijsbers (jlgijsbers) * (Python triager) Date: 2004-11-07 19:29
Logged In: YES 
user_id=469548

Closed accoding to rationale in #1036626.
msg13486 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2005-01-11 03:15
Logged In: YES 
user_id=752496

Closing it, check bug 1036626 for rationale.
History
Date User Action Args
2022-04-10 16:05:57adminsetgithub: 37550
2002-11-29 09:13:16hdimacreate