Beanstalk + Tokyo Tyrant = Edamame, a fast persistent distributed priority job queue
Edamame combines the Beanstalk priority queue with a Tokyo Tyrant database and God monitoring to produce a persistent distributed priority job queue system.
Send Edamame questions to the Infinite Monkeywrench mailing list
Like beanstalk, edamame:
- is fast, scalable, lightweight and distributed
- is a job queue, not just a messaging queue: it offers priority job scheduling (not just FIFO) and reliable scheduling (jobs that time out are re-assigned)
- supports multiple queues (‘tubes’)
Additionally, its lightweight backing store and monitoring scripts mean edamame:
- is persistent and recoverable
- allows you to query and enumerate jobs
- allows named jobs
- provides reasonably-good availability.
Edamame includes a few nifty toys:
- Scripts for God to monitor and restart the daemons
- Command-line management scripts to load. enumerate, empty, and show stats for the db+queue
- The start of a lightweight web frontend in Sinatra.