Python Queue Module .join() and .task_done()














































Python Queue Module .join() and .task_done()




   Previous article link:-
-----------------------------------------------------------------------------------------------------------------------------------------
  NOTE:- For better understanding of this article go through threading module:-
                           https://docs.python.org/3/library/threading.html
-----------------------------------------------------------------------------------------------------------------------------------------
queue.join() and queue.task_done()

  Description:-

  1) SYNTAX:- queue.task_done()
                       After completion of every task queue.task_done() sends a signal to queue that the
                         task has been completed. For each item retrieved using get() a call to task_done()
                         is made. If join() is currently blocking, it will resume when all items have been
                         processed.

   2) SYNTAX:-   queue.join()
                         Queue.join () actually means to wait until the queue is empty before performing other operations.
                               When you call queue.join() in the main thread, all it does is block the main threads until the 
                               workers have processed everything that's in the queue. It does not stop the worker threads,
                               which continue executing their infinite loops. If the worker threads are non-daemon, their                                                 continuing execution prevents the program from stopping irrespective of whether the main 
                               thread has finished.
 
    The reason for discussing both these topics together is because they are related to each other.

  •     Suppose the thread is fetched from the queue every time, but task_done () is not executed, the join()                    cannot determine whether the queue has ended. Performing a join () at the end cannot wait for the               result and will hang.
  •     Every task_done() deletes an element from the queue, so that at the end of the join, the queue length                  is determined based on whether the queue length is zero, so that the main thread is executed.
  
   Let's have a look at the example for better understanding:-
    
   example:- Suppose I gave you a box of work assignment, do I care about when you've taken
                      everything out of the box?
                      No, I don't. What do I really care is the work is done. Looking at an empty box does
                      not tell me that the work is completed. You and the other guys might still be working
                      on stuffs you took out of box.

                      queue.task_done() lets workers say when a task is done. Someone waiting for all the 
                      work to be done with queue.join() will wait until enough task_done() calls have been
                      made, not when the the queue is empty.

  Let's understand this concept with a piece of code:-

import queue
import threading

q = queue.Queue()


def task(num):
while not q.empty(): # get values from threads
print(q.get())
print(f" Task {num} finished")
q.task_done() #sends signal to queue for every task completed


class Thread(threading.Thread): #creating thread class

def __init__(self, num, q):
threading.Thread.__init__(self)
self.num = num
self.q = q

def run(self):
return_value = "Working on task: " + str(self.num)
self.q.put(return_value)
task(self.num)


def main():
for i in range(1, 6):
t = Thread(i, q) #thread created
t.daemon = True
t.start()

q.join() # wait for threads to finish
print("All tasks completed")


if __name__ == "__main__":
main() #main function called
      Output:-

"C:UsersShubhani_PandeyDesktopHarshit Pandeypythonpython.exe" "C:/Users/Shubhani_Pandey/Desktop/Harshit Pandey/python_files/test.py"
Working on task: 1
Task 1 finished
Working on task: 2
Task 2 finished
Working on task: 3
Task 3 finished
Working on task: 4
Task 4 finished
Working on task: 5
Task 5 finished
All tasks completed

Process finished with exit code 0
  Above example very well demonstrates how queue.join() and queue.task_done() are related to each
  other. For every task done a call to task_done() is made and after completion of all the tasks                  queue.join() unblocks.
  
  This is all about this article. Next will discuss about queue.PriorityQueue().

  Next article link:

Comments