Why LinkedIn is important to the future of the Internet of things Posted on : Sep 03 - 2015

LinkedIn  LNKD 2.48%  is not often thought of as a technological giant on par with Google or Facebook, but the professional networking pioneer has made some significant contributions to the way that companies capture and analyze the mountains of data they’re generating. Undoubtedly the biggest is an open source technology called Kafka that was originally developed to pass messages between the pieces of LinkedIn’s sprawling web application, but which is now becoming a fundamental component of the Internet of things—the movement to connect everything from toothbrushes to jet engines.

At a high level, Kafka works by receiving “messages” from one system—that Derrick Harris has edited his profile, for example—and sending them to other systems that needs that information, often in real-time. These are things like a standard database, a big data processing engine (e.g., Hadoop) and the system that powers LinkedIn’s People You May Know feature. It’s a relatively simple function that is very difficult to pull off at when you’re a major web company dealing with huge volumes of fast-moving data. View more