January 20 to 22 2016, Santa Clara, USA.

Speakers

Speaker "Maosong Fu" Details

Name :
maosong fu
Company :
Title :
Software Engineer
Topic :

Twitter Heron: Stream processing at scale

Abstract :

Storm has long served as the main platform for real-time analytics at Twitter. However, as the scale of data being processed in real- time at Twitter has increased, along with an increase in the diversity and the number of use cases, many limitations of Storm have become apparent. We need a system that scales better, has better debug-ability, has better performance, and is easier to manage – all while working in a shared cluster infrastructure. We considered various alternatives to meet these needs, and in the end concluded that we needed to build a new real-time stream data processing system. This talk will present the design and implementation of the new system, called Heron. Heron is now the de facto stream data processing engine inside Twitter, and share our experiences from running Heron in production.

Profile :
Maosong Fu is the technical lead for ​Heron and ​Real Time Analytics at Twitter. ​​He ​worked on various components of Heron including Heron instance, metrics manager, scheduler etc. He ​is the author of ​few publications in distributed area​ and has a master's degree from Carnegie Mellon University and bachelor's from Huazhong University of Science and Technology.
x

Get latest updates of Big Data Bootcamp
sent to your inbox.

Weekly insight from industry insiders.
Plus exclusive content and offers.