Why Machine Learning Is A Priority for Andreessen Horowitz in 2015
Marc Andreessen and Ben Horowitz are legendary in the Silicon Valley, having invested early in firms like Twitter, Facebook, Box, Skype, and Groupon. So when the Sand Hill Road experts with an eye for picking winners put machine learning near the top of its list of trends to watch out for in 2015, it was a cue that the technology is on the cusp of having a breakout year.
The way that Andreessen Horowitz sees things, machine learning and big data are basically inseparable. “Machine learning is to big data as human learning is to life experience,” a general partner in the $4 billion venture firm, Peter Levine, wrote recently on the company’s blog. “We interpolate and extrapolate from past experiences to deal with unfamiliar situations. Machine learning with big data will duplicate this behavior, at massive scales.”
The combination of big data and machine learning opens up a three-step process. First you collect data, then you analyze it, and finally you make your predictions. Up until recently, these steps have been disconnected, Levine writes. The early Hadoop stack and its reliance on batch-oriented MapReduce were good at collecting and storing data, but it was lousy at making predictions at a real-time human scale.
That’s starting to change as new parallel, in-memory frameworks, notably Apache Spark, are allowing us to get more granular and more real-time with predictions, Levine notes. “A lot of iteration needs to occur on a continual basis for the system to get smart, for the machine to ‘learn’–explore the data, visualize it, build a model, ask a question, an answer comes back, bring in other data, and repeat the process,” he writes.
Clearly Levine and the other technology experts at Andreessen Horowitz are bullish on machine learning as one of the key underlying ingredients that will make the big data pie so delicious (not to mention profitable). The company clearly has an eye for startups and established firms that are experts in the field.
- ADATAO – Founded by engineers and data scientists from Google, Yahoo, and Amazon, Adato uses a combination of machine learning and deep learning techniques to help people analyze data in Hadoop. Andreessen Horowitz participated in a $13 million Series A round in August 2014.
- Factual – Los Angeles firm hosts a real-time analytics stack that specializes on enriching mobile location signals with other data sources. Andreessen Horowitz led a $25 million Series A round in 2010.
- GoodData – A cloud-based business intelligence service aimed at giving end users self-serviced data discovery and analytics capabilities. Andreessen Horowitz participated in a $25.7 million in a Series E round in September.
- Platfora – Develops an end-to-end big data analytics application that runs atop Hadoop and allows users to glean insight from unstructured data. Andreessen Horowitz participated in a $38 million Series C round in March 2014.
But here’s the rub: While the companies above are focused, to one degree or another, on providing big data analytics, for the next wave of innovation, you won’t necessarily want to (or be able to) invest directly in machine learning startups. Instead, as Andreessen Horowitz sees it, machine learning technology, along with the big data feeds that drive it, will be infused everywhere, in all the applications you use.
“My belief is every application will be re-constituted to take advantage of this trend,” Levine notes. “And thanks to big data and big compute innovations, we finally have the ingredients to really make this happen. We’re at the threshold of a significant acceleration in machine intelligence that can benefit businesses and society at large.”
Just as Neo Technology’s CEO recently predicted that graph databases will be everywhere in six years, so too will machine learning algorithms. That ubiquity might make picking the winning big data analytic software companies a little harder for folks like Andreessen Horowitz and the other smart moneymen in the Valley. But having smart, big-data infused applications will be a boon for the rest of us.