November 13 to 15 2015, Santa Clara, USA.

Speakers

Speaker "Maxime Beauchemin" Details

Name :
maxime beauchemin
Company :
Title :
Data Engineer
Topic :

Airflow - An open source workflow engine


Abstract :

Airflow - An open source workflow engine Company DAGs (directed acyclic graphs) of jobs are evolving rapidly, and orchestrating and monitoring jobs inside and outside of Hadoop has been a growing challenge for all data driven companies. Airflow is a new platform developed at Airbnb that allows data workers to programmatically author data pipelines. It offers an extensible set of operators that can be used to orchestrate workflows across many data systems as well as a rich user interface to monitor and troubleshoot complex DAGs. Airflow is the future of complex batch processing for companies with complex analysis needs.

Profile :
Maxime Beauchemin recently joined Airbnb as a Data Engineer developing tools to help streamline and automate ETL processes. He mastered his data warehousing fundamentals while at Ubisoft and was an early adopter of Hadoop/PIG while at Yahoo! in 2007. More recently at Facebook he developed "analytics as a service" frameworks around engagement and growth metrics computation, anomaly detection, and cohort analysis. In his free time, he`s a father of 3 and a digital artist. More about his projects on his blog http://mistercrunch.blogspot.com/
x

Get latest updates of Big Data Bootcamp
sent to your inbox.

Weekly insight from industry insiders.
Plus exclusive content and offers.