Today’s Hadoop World talk comes from Peter Sirota, who leads Amazon Web Service’s Elastic MapReduce team. In this talk, Peter provides more detail on the platform, shares some new features, and shows how the AWS community, from customers to developers, are making things easier with Hadoop.
Today’s Hadoop World talk comes from Eric Baldeschwieler, Yahoo!’s VP of Hadoop Development. In this talk, Eric highlights Yahoo’s contributions to development and testing of Hadoop at scale, and goes into detail about how Yahoo! uses Hadoop to deliver several popular services. A major thanks to Eric, and everyone else at Yahoo! for their ongoing contributions to, and investment in, Apache Hadoop – it’s hard to imagine where the project would be today without such dedication.
One service that Cloudera provides for our customers is help with tuning and optimizing MapReduce jobs. Since MapReduce and HDFS are complex distributed systems that run arbitrary user code, there’s no hard and fast set of rules to achieve optimal performance; instead, I tend to think of tuning a cluster or job much like a doctor would treat a sick human being. There are a number of key symptoms to look for, and each set of symptoms leads to a different diagnosis and course of treatment.
At Cloudera, we’re always working to make it easier for you to work with Hadoop and integrate Hadoop-based systems in with your existing data sources. One example of how we accomplish this is Sqoop, a database import tool developed at Cloudera that allows you to easily copy data between databases and HDFS. We originally announced this tool in June, but we’ve been steadily improving it since then. It can now talk with several more databases than before,
Today’s Hadoop World talk comes from Owen O’Malley and talks about some of the biggest challenges facing Hadoop: Security and API Compatibility.
Over the past several months, Yahoo! has been leading the charge in both areas. This work will enable wider use of Hadoop within Yahoo! as well as lower the barrier for new users – particularly those working with sensitive data. A big thanks to Yahoo! and everyone else in the community helping out.