The 2012 Strata + Hadoop World conference was week before last in New York City. Cloudera co-presented the conference with O’Reilly Media this year, and we were really pleased with how the event turned out. Of course we launched Cloudera Impala, but there was a ton of news from companies across the Apache Hadoop ecosystem. Andrew Brust over at ZDNet wins the prize for comprehensive coverage of all the announcements. I also liked Tony Baer’s excellent roll-up of all the SQL news on the OnStrategies blog.
One piece of coverage crossed my inbox this past week that is not generally available. Peter Goldmacher is a Managing Director and Senior Research Analyst for The Cowen Group, a financial services company headquartered in Manhattan. Cowen helps its clients invest wisely, and Peter’s job is to research and report on industry trends that could shape that investment. Peter and his colleague Joe del Callar wrote up an excellent analysis of the Big Data market after attending Strata + Hadoop World. Because their report is published primarily for Cowen’s clients, it’s not easy to link to. Peter has, however, graciously given me permission to excerpt it here. Thank you, Peter!
The title of the report is Quick Take: Hadoop World Observations: One Step Closer to Mainstream Adoption. It makes the argument that Hadoop isn’t yet mainstream, but that customer adoption and vendor investment have accelerated in the last twelve months, and that an inflection point is looming. Peter and Joe say:
In our mind, the signs that Hadoop is about to inflect will include:
1. Meaningful and repeatable projects: Early adopters will pioneer use cases and fast followers will replicate those use cases. Once more mainstream companies see these use cases and are comfortable that the vanguard has derisked the implementation, the use case goes big time. …
2. M&A: The market won’t support 50+ Big Data infrastructure companies. … Every successful software company wants to leverage its installed base with more products and when we start to see the separation of winners from losers, we’ll see the venture community doubling down on winners and cutting losses on their losers. This will come in the form of M&A.
3. Apps companies built on Big Data Infrastructure: … The reason Oracle emerged as the dominant database vendor was because of its partnerships with VARs and ISVs that helped abstract the complexity of working directly with the database and created a new market opportunity of large and repeatable use cases like financials and manufacturing. … We expect to see a lot more Big Data Apps companies getting started in the coming years.
From our perspective at Cloudera, all three points are exactly right.
Our customers don’t care nearly as much about Hadoop as they care about engaging better with their customers or preventing fraud in their transaction flows. People choose the platform because it solves problems that they care about. As a company, we talk much less about technology, and much more about use cases that matter to lots of organizations.
M&A activity has been modest so far, but we expect it to accelerate. I gave a talk at the beginning of 2012 laying out twelve predictions for Hadoop for the year, and number zero (the bonus prediction) was that we’d see some acquisitions before end of year. VMware picked up analytics tool vendor Cetas not long after, but there’s been less activity than I had expected and there are just two months left to go in 2012. Nevertheless, we’re still betting on more acquisitions of smaller players in the near term, because the Cowen argument is sound: The market needs consolidation to concentrate investment.
That trend will be balanced by new companies being created at a fast pace to commercialize innovative ideas for Big Data analytics out of industry and academia. The pace of new product releases and announcements by tools and applications vendors has been torrid this year. Established vendors like Microstrategy, Informatica, SAS and others have delivered Hadoop-capable products. Specialist BI and analytics companies like Tableau, Pentaho and Splunk have expanded their product lines to embrace Hadoop. Emerging companies with products purpose-built for Hadoop, like Datameer, Continuuity, Wibidata, Platfora and Trifacta have entered the fray. This activity is crucial. Since customers care about solving real problems, they need tools and apps that are easy to use and that exploit the strength of the platform while hiding its complexity.
In the report, Peter and Joe go on to identify three categories of winners as Big Data proliferates. Big Data infrastructure providers (we count ourselves in this category at Cloudera) will produce a handful of dominant new companies in the industry. Big Data application providers will be much more numerous and will create a rich and valuable ecosystem for data processing and analysis. It is the third category, though, that caught my eye and that convinced me to sit down and write a blog post.
I’ll let Peter and Joe speak again:
3. Big Data Practitioners: We believe the biggest winners in the Big Data world aren’t the Big Data technology vendors, but rather the companies that will leverage Big Data technology to create entirely new businesses or disrupt legacy businesses.
Past as prologue, if we look at the history of ERP, there were over 200 companies created to capitalize on the automation of standard business processes. This means that investors in 1990 had less than a 0.5% chance of picking either SAP or ORCL as the ultimate winners in the space. However, if an investor had purchased stock in the 30 components of the Dow in 1990 that were all deploying ERP, that investor would have benefited from a 35% decline in General and Administrative costs as a percentage of revenues, a five-fold increase in revenues as automation enabled massive scale, and an almost eight-fold increase in market cap.
Of course we at Cloudera believe we are creating a very valuable company for the long term — we wouldn’t be doing this otherwise. Nevertheless, I agree entirely with the conclusion in the Cowen report. What’s exciting about this platform isn’t the opportunity it’s giving to vendors like us. It’s the value that Hadooop is unlocking in Big Data for the industry and for society at large. Companies that embrace the platform and the tools to ask bigger questions are the ones that will profit most.
This echoes the spread of a generation of open source platform technologies like Linux, MySQL and JBoss. Of course there were entrepreneurs and companies behind each of those projects, and some of them did pretty well. Think, though, about companies like Google and Amazon, that adopted those projects and built their businesses on top of them. Think of whole industries, like financial services, that embraced them to do more things faster, better and more cheaply. That’s where the real money was made.
You want to get rich on Big Data? Use it!