Hadoop is a MapReduce framework that has literally sprung into the vernacular of “big data” developers everywhere. But coding to the raw Hadoop APIs can be a real chore. Data analysts can express what they want in more English-like vocabularies, but it seems the Hadoop APIs require us to be the translator to a less comprehensible functional and data-centric DSL.
The Cascading framework gives developers a convenient higher level abstraction for querying and scheduling complex jobs on a Hadoop cluster. Programmers can think more holistically about the questions being asked of the data and the flow that such data will take without concern for the minutia.
We’ll explore how to set up, code to, and leverage the Cascading API on top of a Hadoop sample or production cluster for a more effective way to code MapReduce applications all while being able to think in a more natural (less than fully MapReduce) way.
During this presentation, we’ll also explore Cascading’s Clojure-based derivative, Cascalog, and how functional programming paradigms and language syntax are emerging as the next important step in big-data thinking and processing.
Matthew is an energetic 15 year veteran of enterprise software development and world-traveling open source educator. Matthew guides the Training efforts at GitHub.com and is author of the Git Master Class series for O’Reilly, co-author of the O’Reilly’s Version Control with Git book, co-author of the Presentation Patterns book, a speaker on the No Fluff Just Stuff tour, an author of three of the top 10 DZone RefCards, including the Git RefCard, and President of the Denver Open Source Users Group.
