This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_queue fails occasionally
Type: Stage:
Components: Interpreter Core Versions: Python 2.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mwh Nosy List: mwh, tim.peters
Priority: normal Keywords:

Created on 2004-08-17 14:54 by mwh, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (5)
msg22099 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2004-08-17 14:54
I guess this is some kind of race condition.  I see
test_queue fail about 5% of the time (judged by running
./python ../Lib/test/regrtest.py test_queue test_queue
test_queue test_queue ... with a couple of hundred
test_queue's, but I've seen it fail as part of a more
normal test run too).

It's always this failure:

test test_queue failed -- blocking function '<bound
method FailingQueue.put of
<test.test_queue.FailingQueue instance at 0x4041b184>>'
appeared not to block

and then, at least sometimes, you get this traceback:

Exception in thread Thread-3408:
Traceback (most recent call last):
  File
"/home/mwh/src/python/dist/src/Lib/threading.py", line
442, in __bootstrap
    self.run()
  File
"/home/mwh/src/python/dist/src/Lib/test/test_queue.py",
line 23, in run
    self.fn(*self.args)
  File "/home/mwh/src/python/dist/src/Lib/Queue.py",
line 129, in get
    item = self._get()
  File
"/home/mwh/src/python/dist/src/Lib/test/test_queue.py",
line 58, in _get
    return Queue.Queue._get(self)
AttributeError: 'NoneType' object has no attribute 'Queue'

somewhat later, usually while a subsequent test is running.

This is a debug build from CVS as of an hour or so back
on Redhat Linux 9 (so, in particular, with the NPTL
threading libray, not LinuxThreads).
msg22100 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-08-18 02:43
Logged In: YES 
user_id=31435

FYI, Guido and I first noticed that test_queue sometimes fails 
a few years ago.  It's apparently much more likely to fail on a 
non-Windows box, perhaps because Windows is happy to 
switch threads frequently.  I ran it for half an hour today on 
a WinXP box, and it didn't fail at all.  IIRC, it's not easy to fix -
- there are threads, and some of the coordination is done via 
time.sleep() calls.

I'll leave this assigned to me for a few more days, but it's 
unlikely I'll find time to "do something" about it.
msg22101 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-08-20 02:32
Logged In: YES 
user_id=31435

OK, it looks like your particular primary failure mode is 
inevitable at times.  If I boost the sleep in _TriggerThread.run
(), it fails every time, in the way you describe, on my box.

The difficulty appears to be one of test design.  
_doBlockingTest believes that no block_func passed to it can 
possibly return while the queue remains blocked, but the 
failing test is passing it a block_func that returns via raising 
FailingQueueException regardless of whether the queue is 
blocked.  So what we see is just the result of a thread race 
in the test.

But as Freud said, insight does not imply a cure <wink>.
msg22102 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-08-20 03:30
Logged In: YES 
user_id=31435

Michael, please give current CVS a try.  I believe I truly fixed 
the test race at the heart of your specific primary problem, 
and greatly decreased the likelihood of two other broad 
classes of sporadic race-related failures in the test driver.
msg22103 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2004-08-20 12:41
Logged In: YES 
user_id=6656

That seems to have done the trick!  At least, I've run
test_queue a couple of hundred times with no failure, and
I'd have expected ~10 or so before I updated.

After reading the checkin comments: I suspect you guessed as
much, but system load was not a factor here.

Closing.
History
Date User Action Args
2022-04-11 14:56:06adminsetgithub: 40773
2004-08-17 14:54:16mwhcreate