mrflip :: data farmer

tools to organize, explore and comprehend massive information streams

This is the home for my open-source projects. I founded the infochimps.org project — a website to find, share or sell any dataset in the world. Get data? Got data? Go infochimps.

Together with Dhruv Bansal and other infochimps, we’re developing the Infinite Monkeywrench: a toolkit for rapid development of effective data-munging scripts.

You can also find me on twitter, at mrflip.com, and … well, I bet you can spot the pattern.

Some components of the Infinite Monkeywrench:

• edamame, a fast persistent job queue
• wukong, Hadoop made so easy a chimpanzee can use it.
• monkeyshines, a fast, flexible distributed guided scraper
• wuclan, massive-scale exploration of social networks.
• IMW (Infinite Monkeywrench), testbed for monkeywrench components
• “infochimps-data,”http://github.com/mrflip/infochimps-data our own collection of data mungers to feed infochimps.org