Back

 Industry News Details

 
What you missed in Big Data: Scaling analytics Posted on : Dec 01 - 2015

The analytics ecosystem grew a little bigger last week with the launch of a new open-source project from IBM Corp. that aims to help simplify the development of machine learning models for data crunching applications. The aptly-named SystemML framework is a product of the company’s work on Watson that abstracts away many of the complicated implementation details involved in building a production-grade algorithm to let analysts focus their energy on creating new features instead. 

A user only needs to write their code in their preferred language, whether it’s R or the less specalized Python, and trust SystemML to do the rest. The framework automatically determines the fastest way to execute an algorithm based on the characteristics of the hardware on which it’s running in order to spare data scientists the hassle of manually optimizing their machine learning logic, thus removing one of the most time-consuming chores in the entire development process. But there are still many more operational obstacles to performing large-scale data analytics left on the checklist that aren’t nearly as easily addressed.

The challenge is arguably most pronounced in the storage layer, where the need to economically retain the massive amounts of data entering the corporate network has to be balanced with the requirement for high-speed access, which has traditionally come at a significant cost. An Israeli stealth startup called Iguaz.io Ltd. raised $15 million from a group of top local venture capital firms led by Magma Venture Partners to tackle the issue with its upcoming software-defined data management solution, which promises to “address the unique challenges” of large-scale analytics. View more