edamame :: job queue
Beanstalk + Tokyo Tyrant = Edamame, a fast persistent distributed priority job queue
Edamame combines the Beanstalk priority queue with a Tokyo Tyrant database and God monitoring to produce a persistent distributed priority job queue system.
- Description
- Installation instructions
- Usage notes
- Alternatives to Edamame, Beanstalk, Tokyo Tyrant or God
- License
Send Edamame questions to the Infinite Monkeywrench mailing list
Description
Like beanstalk, edamame:
- is fast, scalable, lightweight and distributed
- is a job queue, not just a messaging queue: it offers priority job scheduling (not just FIFO) and reliable scheduling (jobs that time out are re-assigned)
- supports multiple queues (‘tubes’)
Additionally, its lightweight backing store and monitoring scripts mean edamame:
- is persistent and recoverable
- allows you to query and enumerate jobs
- allows named jobs
- provides reasonably-good availability.
Edamame includes a few nifty toys:
- Scripts for God to monitor and restart the daemons
- Command-line management scripts to load. enumerate, empty, and show stats for the db+queue
- The start of a lightweight web frontend in Sinatra.
More info
Credits
edamame was written by Philip (flip) Kromer (flip@infochimps.org / @mrflip) for the infochimps project
Help!
Send edamame questions to the Infinite Monkeywrench mailing list
Edamame News
- 19 Aug 2009 » Announcing Edamame!
infochimps.org | find, share or sell any dataset in the world
github/mrflip | mrflip's code on github.com
wukong | hadoop made easy
edamame | fast persistent job queue
monkeyshines | api scraper
wuclan | social media scraper
Thanks to github.com/mojombo for the swanky HTML layout.
