Async Pyramid example done right
After speaking with Marius Gedminas at freenode, he gave me enough hints to rewrite my previous async view example with locks instead of Value, which is prone to race conditions. I also added a queue to allow jobs to wait for being processed.
from multiprocessing import Process, Lock, Queue job = 0 q = Queue(maxsize=3) lock = Lock() def work(): import time; time.sleep(8) job = q.get() print("Job done: {0}".format(job)) print("Queue size: {0}\n".format(q.qsize())) if not q.empty(): work() else: lock.release() def my_view(request): global job if not q.full(): job += 1 q.put(job) # Not running if lock.acquire(False): Process(target=work).start() print("Job {0} submitted and working on it".format(job)) else: print("Job {0} submitted while working".format(job)) else: print("Queue is full") print("Queue size: {0}\n".format(q.qsize())) return {'project':'asyncapp'}
With every request a job is sent. Here the queue accepts 3 jobs. The recursion in work makes sure there is only 1 process working at a time.
I will leave my previous example with Value because it’s easier to understand but this version is much safer.
Update: You can avoid the use of locks by using 2 queues.