Why Siri Needs to Get Smarter Faster Posted on : Oct 24 - 2016

Apple is the most successful company in the world. But future success for Apple depends to a surprising degree on artificial intelligence.

When you think of Silicon Valley's AI leaders, Apple may not immediately come to mind. But the company's initiatives around AI are "amazing." At least that's what the company's newest senior employee said.

Carnegie Mellon researcher Russ Salakhutdinov announced on Twitter and this week that he's been hired by Apple and that his duties include building and managing the team that will develop future AI for the company. He will also continue to work as a professor at Carnegie Mellon. (An Apple spokesperson confirmed his hire.)

On Google+, Salakhutdinov wrote that "Lots of truly amazing AI and Machine Learning projects are taking place all over Apple" and that he and Apple will build "a team of top-notch Machine Learning and Deep Learning researchers, working on challenging R&D projects as well as conducting fundamental research to advance the state-of-the-art in AI!" He didn't respond to my request for comment.

Salakhutdinov has worked as an associate professor in the Machine Learning Department of the School of Computer Science at Carnegie Mellon University since February. Before that, he did similar research at MIT and the University of Toronto. Besides machine learning and deep learning, he specializes in applying AI to finding patterns buried in huge data sets, such as millions of photos or gigabytes of text.

Apple wouldn't tell me how many people within the company are working on AI. But based on the company's job listings and acqui-hires, it needs more. (In the past year alone, Apple has acquired at least five AI-related startups: Turi, Tuplejump, Emotient, Perceptio, and Vocal IQ.)

APPLE AI? YOU'RE SOAKING IN IT

Apple uses big AI for small things that enhance user experiences. For example, Apple uses AI to improve the iPhone's battery life, identify people and objects in Apple Photos, and predict which apps you'll want to open next in the "Siri App Suggestions" widget on the Today view.

Apple uses machine learning for the iPad Pro's "palm rejection," which enables Apple Pencil users to rest their hand on the screen while writing or drawing without triggering touch responses.

When you're watching a movie on an Apple TV and miss some dialogue, you can ask: "What did she say?" Apple uses AI to figure out who "she" is, and also which line you're asking about.

These are just a few of the hundreds of subtle applications of AI sprinkled throughout Apple products.

One unique feature of Apple's approach to AI: In the interest of privacy, it handles personal data locally on the phone, rather than hoovering it all up and storing it in the cloud, as Google and Amazon do. This is a challenge, because normally user data is harvested and stored in data centers, then used to train neural nets.

OPEN THE POD BAY DOOR, SIRI

Siri is the most obvious place for more noticeable AI at Apple, and the company is actively pursuing the enhancement of Siri with ever-more advanced AI, according to an Apple spokesperson.

Siri is already AI-based. Two years ago, Apple flipped a switch and Siri gained a range of advanced machine learning techniques. Apple has claimed that the AI has dramatically improved Siri's speech recognition and the accuracy of results. Users don't seem to have noticed. Complaints about Siri are still rife.

Veteran tech journalist Walt Mossberg wrote a scathing takedown of Siri this month for Recode and The Verge headlined, "Why does Siri seem so dumb?" Mossberg wrote that "in its current incarnation, Siri is too limited and unreliable to be an effective weapon for Apple in the coming AI wars."

Today's Siri fits solidly within the pantheon of first-generation virtual assistants that includes Google Now, Microsoft's Cortana, Amazon's Alexa, and others, at least from the user perspective. These services create the illusion that the assistant can understand what users say, then return an audible voice response in colloquial speech. They all do a pretty good job under specific circumstances and for specific queries. But they lack more advanced AI, and cannot "figure things out" as well as many users would like them to do.

Refining today's virtual assistants with tomorrow's AI may be an imperative for Silicon Valley's tech giants. It's easy to imagine a future where virtual assistants replace most Google searches, most direct interaction with calendars and to do lists, and most remote controls, buttons, and dials for smart home appliances. We'll just talk, and virtual assistants will do all our bidding.

The company that can deliver the most compelling virtual assistant experience will have an enormous advantage. An AI virtual assistant may become the ultimate gateway drug for a company's other products. For example, if users come to rely upon a future Google AI virtual assistant, they may be incentivized to buy Google and Android phones and Chromebook laptops, use a Google browser, and buy a TV that uses Android TV. And that's a problem for Apple.

The race is on, and Google is probably winning. Google has made big strides in applying AI to identifying people, objects, and places in Google Photos, as well as in the human-like task of choosing the best pictures to be clustered into collections of "highlights" and "memories." Google's Smart Reply, available in Google Inbox and the Allo messaging app, offers suggested replies based on shockingly advanced neural networking technology.

Tech giants have been gobbling up AI startups like breakfast cereal. Apple, Google, Microsoft, Intel, IBM, Yahoo, eBay, Amazon, AOL, Nokia, Facebook, Twitter, Salesforce, and Oracle have collectively grabbed 140 AI-related companies since 2011, according to research firm CB Insights.

But Google leads the pack in the acquisition of AI-related startups both quantitatively (11 startups in the past five years) and, arguably, qualitatively: Google acquired DeepMind two years ago. The DeepMind system is capable of learning all on its own from its own memory without human input.

Tech pundits had high hopes for Google's new Assistant product, which promised to bring more advanced AI into the virtual assistant world. Google Assistant is available in Google's newish Allo messaging app, its Home virtual assistant appliance, and in the new line of Google Pixel smartphones. The company intends to add Assistant to Android Wear watches, the Chrome browser, and even car dashboards. View More