Cloudera Developer Blog · Hadoop Posts
In software development, there is no substitute for having choices. Furthermore, freedom of choice – between frameworks, APIs, and languages — is a major fuel source for platform adoption across any successful ecosystem.
In the case of development on CDH, the open source core of Cloudera’s Big Data platform containing Apache Hadoop and related ecosystem projects, the choices have expanded dramatically in the past three weeks:
Among Cloudera’s engineer-presenters at Strata + Hadoop World 2013 this week, Philip Zeyliger (“Tricks for Distributed System Debugging and Diagnosis“) was particularly fortunate to have been interviewed by O’Reilly Media editor Meghan Blanchette on camera.
In the following 8-minute interview, Philip offers an overview of common pain points and failures when debugging distributed systems:
We are just a weekend away from the Biggest. Strata + Hadoop World. Ever.
There are a number of special “users” with roles to play in the Apache Hadoop environment. For your reference, we have summarized them below as of CDH 4.4. Kerberos principals (used for authentication in a secure cluster) are not covered here.
The specific user IDs listed are the ones created by default on installation but they are configurable unless otherwise indicated.
The release of Apache Hadoop 2, as announced today by the Apache Software Foundation, is an exciting one for the entire Hadoop ecosystem.
Cloudera engineers have been working hard for many months with the rest of the vast Hadoop community to ensure that Hadoop 2 is the best it can possibly be, for the users of Cloudera’s platform as well as all Hadoop users generally. Hadoop 2 contains many major advances, including (but not limited to):
In a fast-moving project like Apache Hadoop, there are always exciting new features introduced in each release. While it is tempting to make the most of these new features by upgrading to the latest release, users are often concerned about their code continuing to run.
In this post, you’ll get an overview of the the Hadoop API annotations and compatibility policies. Hadoop annotates specific APIs to be safe for use by end-users. By using these APIs, users can ensure their code works across a set of releases and be aware of what releases it might not work against.
Hadoop Releases and Compatibility
Customer Spotlight: Learn How Edo Closes the Advertising Loop with Hadoop at Cloudera Sessions Milwaukee
The Cloudera Sessions fall series is coming to a close next week, but first we’ll make a final stop in Milwaukee, Wisconsin (on Oct. 17), where attendees will hear about edo — a company that is revolutionizing the advertising space by closing the loop between promotions and point-of-sale transactions.
In Milwaukee, edo CTO Jeff Sippel will engage in a fireside chat with Cloudera’s VP of marketing, Alan Saldich. At edo, Jeff is responsible for the strategy, planning, and execution for the systems — including Apache Hadoop — that power the edo offer platforms.
Below please find our regularly scheduled quarterly update about where to find tech talks by Cloudera employees this year – this time, for October through December 2013. Note that this list will be continually curated during the period; complete logistical information may not be available yet.
As always, we’re standing by to assist your meetup by providing speakers, sponsorships, and schwag!
|Oct. 1||Aarhus, Denmark||GOTO Aarhus||Eva Andreeason on Hadoop use cases|
|Oct. 8||Sunnyvale, Calif.||Hadoop Happy Hour||Kathleen Ting and Jarek Cecho sign books!|
|Oct. 9||Santa Clara, Calif.||IEEE BigData Conference||Amr Awadallah on Hadoop use cases|
|Oct. 9||San Francisco||SF Hadoop Users||Eric Sammer on Hadoop app development (panelist)|
|Oct. 10||Sydney||DataCon||Sean Owen on data science|
|Oct. 15||Durham, NC||TriHUG||Mark Miller on Solr+Hadoop|
|Oct. 15||Mountain View, Calif.||Oracle NoSQL & Big Data Meetup||Mike Olson on virtues of key-value stores|
|Oct. 15-17||Burlingame, Calif.||Big Data TechCon||Apache Hive workshop with Mark Grover|
|Doug Cutting on the Hadoop revolution|
|Hadoop app development (CDK) workshop with Ryan Blue|
|Jonathan Seidman on extending data infrastructure with Hadoop|
|Jonathan Seidman on the Hadoop ecosystem|
|Himanshu Vashishtha on HBase use cases|
|Kate Ting on Apache ZooKeeper|
|Kate Ting on 7 Deadly Hadoop Misconfigurations|
|Oct. 16||Dallas, Tex.||DFW Big Data||John Ringhofer on Impala|
|Oct. 17||Milwaukee, Wis.||Cloudera Sessions||Hadoop app development lab (on CDK) with Ryan Blue|
|Oct. 17||St. Louis, Mo.||St. Louis HUG||Tom Wheeler on Parquet|
|Oct. 18||Munich||HUG Munich||Lars George on Impala|
|Oct. 22||London||UK HUG||Sean Owen on Scalable Big learning|
|Oct. 23||Seattle||Seattle Scalability Meetup||Ronan Stokes on Cloudera Search|
|Oct. 24||Palo Alto, Calif.||Bay Area HBase User Group||Michael Stack on HBase 0.96|
|Oct. 24||Raleigh, NC||All Things Open||Josh Wills on open source innovation|
|Oct. 28-30||New York||Strata Conference + Hadoop World 2013||Mike Olson on Hadoop’s impact on data management|
|Doug Cutting on the future of Hadoop|
|Henry Robinson on workload diversity in Hadoop|
|Hadoop app development (CDK) workshop with Eric Sammer|
|Matt Brandwein on leveraging mainframe data with Hadoop|
|Aaron T. Myers and Shreepadma Venugopalan on Hadoop security|
|Jayant Shekar on machine data analytics|
|Amandeep Khurana on Monsanto’s use case for Hadoop & HBase|
|Philip Zeyliger on debugging distributed systems|
|Greg Rahn on Impala performance tuning|
|Jon Hsieh on HBase roadmap|
|Oct. 28||New York||NYC HUG||Arvind Prabhakar on Apache Sentry (incubating)|
|Oct. 28||New York||Sqoop User Meetup||Abe Elmahrek on the Sqoop2 app for Hue|
|Oct. 29||New York||Impala + Parquet Meetup||Greg Rahn on Impala+Parquet performance tuning|
|Oct. 29||New York||Cloudera Manager Meetup||Aditya Achara on Cloudera Manager success stories|
|Oct. 30||New York||Apache Sentry User Meetup||Arvind Prabhakar and Shreepadma Venugopalan with a Sentry overview|
|Oct. 30||Philadelphia||Chariot Data IO Conference||Lars George on HBase sizing as well as on Parquet|
|Nov. 6||Chantilly, Va.||Open Source Search Conference||Alex Moundalexis on Search+Hadoop|
|Nov. 6||Munich||JAX Munich||Lars George on HBase and Impala|
|Nov. 7||Tokyo||Cloudera World Tokyo||Kiyoshi Mizumaru on CDH|
|Sho Shimauchi on Cloudera Manager|
|Tatsuo Kawasaki witha Hadoop 101|
|Daisuke Kobayashi on Hadoop ops|
|Nov. 11||London||UK HUG||Marcel Kornacker on Impala|
|Nov. 12-13||London||Strata London||Sean Owen on Scalable Big Learning; Tom White on Hadoop app development with CDK|
|Nov. 12||San Francisco||QCon SF||Josh Wills on machine learning|
|Nov. 13||Washington DC||LISA 2013||John Ridley on Hadoop 101 for sysadmins|
|Nov. 14||Seoul||Tech Planet Korea||Michael Stack on HBase roadmap|
|Nov. 14||Tokyo||Cloudera Manager Meetup||Sho Shimauchi, Kiyoshi Mizumaru: What is Cloudera Manager?|
|Nov. 14||Antwerp||Devoxx Belgium||Tom White on building Hadoop apps with CDK|
|Nov. 16||Los Angeles||Big Data Camp LA||Alex Behm on Impala|
|Nov. 20||Boulder, Colo.||Boulder/Denver Big Data Meetup||John Darrah on Hadoop 101|
|Dec. 2||Tokyo||Cloudera Manager Meetup||Sho Shimauchi, Kiyoshi Mizumaru: What is Cloudera Manager?|
History teaches us that ecosystem growth is fueled by enthusiasm, tools (including frameworks and APIs), and knowledge in roughly equal measures. To this point, the Apache Hadoop ecosystem has been blessed with the first two ingredients – thanks to the magic of open source – but in the third category, there is still plenty of work to be done.
Welcome to our third edition of “This Month in the Ecosystem,” a digest of highlights from September 2013 (never intended to be comprehensive; for completeness, see Hadoop Weekly).
Note: there were a few other interesting developments this week, but out of respect for the calendar, I’ll address them next month.