Apache Spark Updates, Executive Data Strategies: Big Data Roundup Posted on : Nov 30 - 2015

You might expect a slow week for big data news as all the analysts and data scientists focused on optimizing meal preparation, predicting football scores, and surfacing the best prices on the most sought after holiday gifts. But you'd be wrong. This week we had news from IBM and Databricks about Apache Spark, a closer look at how a couple of different companies -- AccuWeather and McGraw-Hill Education -- have built analytics into the services they offer, a great primer on Artificial Intelligence and the future, and more.

Let's start with the IBM and Databricks announcements around Apache Spark.

You may remember that IBM Analytics contributed its SystemML engine for machine learning to the Apache Foundation earlier this year. IBM said the goal was to improve upon Spark's MLlib machine learning algorithms and libraries. This week IBM announced that its SystemML has been accepted into the Apache Incubator program.

Also this week, Databricks started previewing its software-as-a-service implementation of the next version of big data platform Apache Spark. Spark 1.6.0 is not due for general release until mid-December, but big data pros can take it for a test run on Databricks starting now. The new version is focused on improving performance. Check out the full story on Apache Spark's updates this week here. View more