Steve Jurvetson: Deep Learning

black_boxbeyond_engineeringDeep Learning: Intelligence from Big Data
Sep 16, 2014
Stanford Graduate School of Business

A machine learning approach inspired by the human brain, Deep Learning is taking many industries by storm. Empowered by the latest generation of commodity computing, Deep Learning begins to derive significant value from Big Data. It has already radically improved the computer’s ability to recognize speech and identify objects in images, two fundamental hallmarks of human intelligence.

Industry giants such as Google, Facebook, and Baidu have acquired most of the dominant players in this space to improve their product offerings. At the same time, startup entrepreneurs are creating a new paradigm, Intelligence as a Service, by providing APIs that democratize access to Deep Learning algorithms.

Steve Jurvetson, Partner, DFJ Ventures

Adam Berenzweig, Co-founder and CTO, Clarifai
Naveen Rao, Co-founder and CEO, Nervana Systems
Elliot Turner, Founder and CEO, AlchemyAPI
Ilya Sutskever, Research Scientist, Google Brain

19:24 Subsystem inscrutability. Black box.

20:04 Beyond Engineering
Danny Hillis, The Pattern on the Stone: “The greatest achievement of our technology may well be the creation of tools that allow us to go beyond engineering–that allow us to create more than we can understand”

33:40 Convolutional Neural Networds

34:10 current deep learning model: recognizes 10,000 categories (classes).
average adult: 90,000 words (in English, not counting personal names)

1:09:00 we can propagate a signal with almost 0 loss. Biology can’t.
Will AI look like human intelligence? The underlying structure is different (substrate: silicon vs. neurons). With constraints, AI’s behavior will look however we want.


state of the art in reverse image search (01/01/2015):
the original (and only) source of the following image is placed low in results:

Big Data has spawned a cult of infallibility

Forget YOLO: Why ‘Big Data’ Should Be The Word Of The Year
by Geoff Nunberg
December 20, 2012

Whatever the sticklers say, data isn’t a plural noun like “pebbles.” It’s a mass noun like “dust.”

It’s only when all those little chunks are aggregated that they turn into Big Data; then the software called analytics can scour it for patterns

You idly click on an ad for a pair of red sneakers one morning, and they’ll stalk you to the end of your days.
It makes me nostalgic for the age when cyberspace promised a liberating anonymity.
I think of that famous 1993 New Yorker cartoon by Peter Steiner: “On the Internet, nobody knows you’re a dog.
Now it’s more like, “On the Internet, everybody knows what brand of dog food you buy.”

In some circles, Big Data has spawned a cult of infallibility — a vision of prediction obviating explanation and math trumping science.
In a manifesto in Wired, Chris Anderson wrote, “With enough data, the numbers speak for themselves.”

The trouble is that you can’t always believe what they’re saying.
When you’ve got algorithms weighing hundreds of factors over a huge data set, you can’t really know why they come to a particular decision or whether it really makes sense.

When I was working with systems like these some years ago at the Xerox Palo Alto Research Center, we used to talk about a 95 percent solution.
So what if Amazon’s algorithms conclude that I’d be interested in Celine Dion’s greatest hits, as long as they get 19 out of 20 recommendations right?
But those odds are less reassuring when the algorithms are selecting candidates for the no-fly list.

I don’t know if the phrase Big Data itself will be around 20 years from now, when we’ll probably be measuring information in humongobytes.
People will be amused to recall that a couple of exabytes were once considered big data, the way we laugh to think of a time when $15,000 a year sounded like big money.
But 19 out of 20 is probably still going to be a good hit rate for those algorithms, and people will still feel the need to sort out the causes from the correlations — still asking the old question, what are patterns for?


May 14, 2012

June 7, 2015