September 08 to 10 2014, Santa Clara, USA.


Speaker "Sheetal Dolas" Details

Name :
sheetal dolas
Company :
Title :
Principal Architect
Topic :

Analyzing 1.2 million network packets per second in real time

Abstract :

Real time network packet analysis is extremely critical for any network security. Ability to collect all the network traffic and analyze it in real time is an elephantine challenge. Especially, during Denial of Service attacks, there could be millions of packets travelling over network per second. Not many systems can capture, analyze, store and provide alerts/insights at this rate. We at Cisco and Hortonworks together built a solution named OpenSOC that can capture, deeply inspect and analyze these packets; at the rate of 1.2 million packets per second in real time. This talk covers the use case and our use of Kafka-Storm-HBase–ElasticSearch to ingest 1.2 million network packets per second in real time. Specifically, we discuss how we started with just 5K packets per second and scaled the system to handle 1.2 million packets per second, the solution choices, different techniques and strategies, traditional and innovative approaches that made the performance jump through the roof. Attendees can take away learnings from our real-life experience that can help them understand various tuning methods, their tradeoffs and apply them in their solutions. and last thing... OpenSOC is being open sourced. Find out how you can take advantage of it.

Profile :
Sheetal is a Principal Architect working with Hortonworks. He has strong expertise in Hadoop ecosystem with very rich & diverse field experience across various verticals including Telco, Hi Tech, Retail, Internet Companies etc. He has served in key positions as Lead Big Data Architect, SOA Architect in variety of extremely large & complex enterprise programs. Has extensive knowledge of BigData/NoSql technologies including Hadoop/Yarn/Hive/Pig/HBase/Storm/Kafka/ElasticSearch etc. He has defined & established data architectures for multi-petabyte warehouses on Hadoop, has extensive hands on experience in deploying, tuning very large Hadoop clusters & building scalable applications on them.

Get latest updates of 2nd Annual Global Big Data Conference
sent to your inbox.

Weekly insight from industry insiders.
Plus exclusive content and offers.