This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: thread.error: release unlocked lock on Queue put
Type: Stage:
Components: Interpreter Core Versions: Python 2.3
process
Status: closed Resolution: later
Dependencies: Superseder:
Assigned To: Nosy List: corvus, rhettinger, snacktime, spiv, tim.peters
Priority: normal Keywords:

Created on 2004-12-07 14:48 by corvus, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (7)
msg23570 - (view) Author: John Speno (corvus) Date: 2004-12-07 14:48
System is Python 2.3.3 on Solaris 9. 

I'm using the classic Python thread model. Main thread puts stuff into
a Queue.Queue() and worker threads get stuff out of it and do their 
thing.

On occaision, I get an exception in the main thread when it tries to
put something into the Queue.

   File "/usr/local/bin/poller.py", line 47, in fragnap_it
     work_queue.put((foo, bar, baz))
   File "/usr/local/lib/python2.3/Queue.py", line 106, in put
     self.fsema.release()
   thread.error: release unlocked lock

This error still happens intermittently, and I haven't been able to 
reduce it to a simple case. I can alter my applications to do 
something useful for debugging this problem when it happens, if you 
can figure out what "something useful" would be. Just let me know.

The queue is unbounded, and I'm using blocking get/put.

The main thread creates 20 worker threads, and 1 recorder thread. 
Then it puts upto 1500 items in the Queue.Queue named 
work_queue. The 20 worker threads take stuff out work_queue, 
process some using pure python code, or others by using 
subprocess.py. Results from the workers are put into another 
Queue.Queue named results_queue. The Recorder thread gets from 
the results_queue and writes stuff to a MySQL db. I've never seen the 
error happen when putting into the results_queue, only from the main 
thread putting into the work_queue.

Thread defines in my python config.h are:

#define HAVE_PTHREAD_H 1
#define HAVE_PTHREAD_SIGMASK 1
#define HAVE_THREAD_H 1
#define PTHREAD_SYSTEM_SCHED_SUPPORTED 1
#define SIZEOF_PTHREAD_T 4
#define WITH_THREAD 1

And all the thread related python tests (test_thread, test_threading, 
and test_queue) pass.
msg23571 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2004-12-10 17:35
Logged In: YES 
user_id=80475

Tim, was this something you already fixed in Py2.4?
msg23572 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-12-10 17:45
Logged In: YES 
user_id=31435

No, and there are no known bugs in Queue in 2.3.j.  If John 
can't whittle it down to a test case that fails for others, I 
expect this bug is hopeless.  We talked about it on c.l.py, 
and what he's seeing should be impossible.  Presuming it's a 
Solaris-specific bug.
msg23573 - (view) Author: John Speno (corvus) Date: 2004-12-10 17:57
Logged In: YES 
user_id=2138

Given that I can't reproduce it in a simple test, and it happens so rarely in 
my real applications, I'm okay with notice of it just being in google and here 
on sf. Maybe someday someone will encounter it again and we can make 
progress.
msg23574 - (view) Author: Andrew Bennetts (spiv) Date: 2005-02-08 04:33
Logged In: YES 
user_id=50945

Looks like someone is seeing something similar on FreeBSD
and Python 2.4, with the added bonus of segfaults:
   
http://twistedmatrix.com/pipermail/twisted-python/2005-February/009474.html

They are apparently able to reproduce the problem, too --
but a 3rd-party extension module (psycopg) is involved, so
who knows...
msg23575 - (view) Author: Chris Ochs (snacktime) Date: 2005-02-08 04:51
Logged In: YES 
user_id=31170

Me being that someone....  Here is some extra information in 
addition to what is in the referenced post on twisted-python.

Besides twisted, the only other third party modules I am using 
are psycopg and pycrypto.  I'm going to pull out the code 
that uses those modules one at a time and see if I can 
narrow it down.  The segfaults/bus errors are easy to repeat 
and happen every time unless gdb is attached, in which case 
I get the exception.  For some reason it seems like gdb does 
something that makes the code return the exception instead 
of segfault.

I'll try to put together the smallest test case I can that 
segfaults and post it here.  

Chris
chris@paymentonline.com
msg23576 - (view) Author: Chris Ochs (snacktime) Date: 2005-02-08 05:21
Logged In: YES 
user_id=31170

The problem stems from pycrypto.  Not knowing a whole lot 
about threads I'm not sure if it's actually a pycrypto bug or a 
python bug, but I'll send a message to the pycryto guys and 
see what they say.  The exact method that causes the 
segfault is RSA encrypt()
History
Date User Action Args
2022-04-11 14:56:08adminsetgithub: 41291
2004-12-07 14:48:11corvuscreate