tools to organize, explore and comprehend massive information streams
This is the home for my open-source projects. I founded the infochimps.org project — a website to find, share or sell any dataset in the world. Get data? Got data? Go infochimps.
Together with Dhruv Bansal and other infochimps, we’re developing the Infinite Monkeywrench: a toolkit for rapid development of effective data-munging scripts.
You can also find me on twitter, at mrflip.com, and … well, I bet you can spot the pattern.
Some components of the Infinite Monkeywrench:
- edamame, a fast persistent job queue
- wukong, Hadoop made so easy a chimpanzee can use it.
- monkeyshines, a fast, flexible distributed guided scraper
- wuclan, massive-scale exploration of social networks.
- IMW (Infinite Monkeywrench), testbed for monkeywrench components
- “infochimps-data,”http://github.com/mrflip/infochimps-data our own collection of data mungers to feed infochimps.org