Back

 Industry News Details

 
Opening the Door for Wider Hadoop Storage Strategies Posted on : Sep 03 - 2015

The Hadoop framework was created to deliver a balance between performance and data management efficiencies and while this is a good part of the reason it has taken off in recent years, there are still some areas of the platform that are out of balance at the storage layer. At the same time, the demands on Hadoop to run jobs at top speed and energy efficiency are increasing as well—the only thing that has not changed is the ability for users to expand beyond concrete storage options.

Since not all storage media is created equal and must be balanced (for instance, hard disks are valuable for their low cost per gigabyte and high capacity while solid state drives offer excellent performance and energy efficiency, but are more expensive), it stands to reason the ability to tune and rebalance for several storage approaches should be part of Hadoop. However, while platform supports the use of SSDs generally, at the storage layer, it treats all storage devices equally or, more accurately, it is not balanced for the advantages of hard disk drives over SSDs.

If the native Hadoop storage layer that revolves around the Hadoop Distributed File System (HDFS) allows the simultaneous use of the best of both the hard disk and SSD worlds, it stands to reason that if the performance would see a boost, the energy consumption would go down if the right data elements were handled by the best fit storage device, whether that is higher performance and efficiency SSDs that are expensive, but beat the slower but more economically efficient advantages of disk.  View more