In my setup (using python3) the idle event doesn't seem to work. There seems to be a race condition in the event handling. Example:
execute_some_task.delay()
self.worker.idle.wait()
# something else
This doesn't work for me because self.worker.idle.wait() is executed before the idle event is cleared and thus passes immediately. That's because the events are not handled in the same thread as the call to delay(), thus the order in which the call the wait() and the call to idle.clear() are executed is undefined.
There's actually is a simple way to make those events work: use a simple (lock-protected) counter for the currently running tasks. Increment it on signal.after_task_sent and decrement it on signal.task_postrun.
signal.after_task_sent is guaranteed to run in the same thread as the call to delay.
In my setup (using python3) the idle event doesn't seem to work. There seems to be a race condition in the event handling. Example:
This doesn't work for me because
self.worker.idle.wait()is executed before the idle event is cleared and thus passes immediately. That's because the events are not handled in the same thread as the call todelay(), thus the order in which the call thewait()and the call toidle.clear()are executed is undefined.There's actually is a simple way to make those events work: use a simple (lock-protected) counter for the currently running tasks. Increment it on
signal.after_task_sentand decrement it onsignal.task_postrun.signal.after_task_sentis guaranteed to run in the same thread as the call todelay.