Cloudera Developer Blog · Hadoop Posts

Cascading, Spring, and Spark: Development Choices for CDH Users Expand

In software development, there is no substitute for having choices. Furthermore, freedom of choice – between frameworks, APIs, and languages — is a major fuel source for platform adoption across any successful ecosystem.

In the case of development on CDH, the open source core of Cloudera’s Big Data platform containing Apache Hadoop and related ecosystem projects, the choices have expanded dramatically in the past three weeks:

Tips for Debugging Distributed Systems

Among Cloudera’s engineer-presenters at Strata + Hadoop World 2013 this week, Philip Zeyliger (“Tricks for Distributed System Debugging and Diagnosis“) was particularly fortunate to have been interviewed by O’Reilly Media editor Meghan Blanchette on camera.

In the following 8-minute interview, Philip offers an overview of common pain points and failures when debugging distributed systems:

See You Next Week at Strata + Hadoop World 2013!

Strata

We are just a weekend away from the Biggest. Strata + Hadoop World. Ever.

Guide to Special Users in the Hadoop Environment

There are a number of special “users” with roles to play in the Apache Hadoop environment. For your reference, we have summarized them below as of CDH 4.4. Kerberos principals (used for authentication in a secure cluster) are not covered here.

The specific user IDs listed are the ones created by default on installation but they are configurable unless otherwise indicated.

Apache Hadoop 2 is Here and Will Transform the Ecosystem

The release of Apache Hadoop 2, as announced today by the Apache Software Foundation, is an exciting one for the entire Hadoop ecosystem.

Cloudera engineers have been working hard for many months with the rest of the vast Hadoop community to ensure that Hadoop 2 is the best it can possibly be, for the users of Cloudera’s platform as well as all Hadoop users generally. Hadoop 2 contains many major advances, including (but not limited to):

Writing Hadoop Programs That Work Across Releases

In a fast-moving project like Apache Hadoop, there are always exciting new features introduced in each release. While it is tempting to make the most of these new features by upgrading to the latest release, users are often concerned about their code continuing to run.

In this post, you’ll get an overview of the the Hadoop API annotations and compatibility policies. Hadoop annotates specific APIs to be safe for use by end-users. By using these APIs, users can ensure their code works across a set of releases and be aware of what releases it might not work against.

Hadoop Releases and Compatibility

Customer Spotlight: Learn How Edo Closes the Advertising Loop with Hadoop at Cloudera Sessions Milwaukee

The Cloudera Sessions fall series is coming to a close next week, but first we’ll make a final stop in Milwaukee, Wisconsin (on Oct. 17), where attendees will hear about edo — a company that is revolutionizing the advertising space by closing the loop between promotions and point-of-sale transactions.

In Milwaukee, edo CTO Jeff Sippel will engage in a fireside chat with Cloudera’s VP of marketing, Alan Saldich. At edo, Jeff is responsible for the strategy, planning, and execution for the systems — including Apache Hadoop — that power the edo offer platforms.

Where to Find Cloudera Tech Talks Through December 2013

Below please find our regularly scheduled quarterly update about where to find tech talks by Cloudera employees this year – this time, for October through December 2013. Note that this list will be continually curated during the period; complete logistical information may not be available yet.

As always, we’re standing by to assist your meetup by providing speakers, sponsorships, and schwag!

Date City Venue Speaker(s)
Oct. 1 Aarhus, Denmark GOTO Aarhus Eva Andreeason on Hadoop use cases
Oct. 8 Sunnyvale, Calif. Hadoop Happy Hour Kathleen Ting and Jarek Cecho sign books!
Oct. 9 Santa Clara, Calif. IEEE BigData Conference Amr Awadallah on Hadoop use cases
Oct. 9 San Francisco SF Hadoop Users Eric Sammer on Hadoop app development (panelist)
Oct. 10 Sydney DataCon Sean Owen on data science
Oct. 15 Durham, NC TriHUG Mark Miller on Solr+Hadoop
Oct. 15 Mountain View, Calif. Oracle NoSQL & Big Data Meetup Mike Olson on virtues of key-value stores
Oct. 15-17 Burlingame, Calif. Big Data TechCon Apache Hive workshop with Mark Grover
Doug Cutting on the Hadoop revolution
Hadoop app development (CDK) workshop with Ryan Blue
Jonathan Seidman on extending data infrastructure with Hadoop
Jonathan Seidman on the Hadoop ecosystem
Himanshu Vashishtha on HBase use cases
Kate Ting on Apache ZooKeeper
Kate Ting on 7 Deadly Hadoop Misconfigurations
Oct. 16 Dallas, Tex. DFW Big Data John Ringhofer on Impala
Oct. 17 Milwaukee, Wis. Cloudera Sessions Hadoop app development lab (on CDK) with Ryan Blue
Oct. 17 St. Louis, Mo. St. Louis HUG Tom Wheeler on Parquet
Oct. 18 Munich HUG Munich Lars George on Impala
Oct. 22 London UK HUG Sean Owen on Scalable Big learning
Oct. 23 Seattle Seattle Scalability Meetup Ronan Stokes on Cloudera Search
Oct. 24 Palo Alto, Calif. Bay Area HBase User Group Michael Stack on HBase 0.96
Oct. 24 Raleigh, NC All Things Open Josh Wills on open source innovation
Oct. 28-30 New York Strata Conference + Hadoop World 2013 Mike Olson on Hadoop’s impact on data management
Doug Cutting on the future of Hadoop
Henry Robinson on workload diversity in Hadoop
Hadoop app development (CDK) workshop with Eric Sammer
Matt Brandwein on leveraging mainframe data with Hadoop
Aaron T. Myers and Shreepadma Venugopalan on Hadoop security
Jayant Shekar on machine data analytics
Amandeep Khurana on Monsanto’s use case for Hadoop & HBase
Philip Zeyliger on debugging distributed systems
Greg Rahn on Impala performance tuning
Jon Hsieh on HBase roadmap
Oct. 28 New York NYC HUG Arvind Prabhakar on Apache Sentry (incubating)
Oct. 28 New York Sqoop User Meetup Abe Elmahrek on the Sqoop2 app for Hue
Oct. 29 New York Impala + Parquet Meetup Greg Rahn on Impala+Parquet performance tuning
Oct. 29 New York Cloudera Manager Meetup Aditya Achara on Cloudera Manager success stories
Oct. 30 New York Apache Sentry User Meetup Arvind Prabhakar and Shreepadma Venugopalan with a Sentry overview
Oct. 30 Philadelphia Chariot Data IO Conference Lars George on HBase sizing as well as on Parquet
Nov. 6 Chantilly, Va. Open Source Search Conference Alex Moundalexis on Search+Hadoop
Nov. 6 Munich JAX Munich Lars George on HBase and Impala
Nov. 7 Tokyo Cloudera World Tokyo Kiyoshi Mizumaru on CDH
Sho Shimauchi on Cloudera Manager
Tatsuo Kawasaki witha Hadoop 101
Daisuke Kobayashi on Hadoop ops
Nov. 11 London UK HUG Marcel Kornacker on Impala
Nov. 12-13 London Strata London Sean Owen on Scalable Big Learning; Tom White on Hadoop app development with CDK
Nov. 12 San Francisco QCon SF Josh Wills on machine learning
Nov. 13 Washington DC LISA 2013 John Ridley on Hadoop 101 for sysadmins
Nov. 14 Seoul Tech Planet Korea Michael Stack on HBase roadmap
Nov. 14 Tokyo Cloudera Manager Meetup Sho Shimauchi, Kiyoshi Mizumaru: What is Cloudera Manager?
Nov. 14 Antwerp Devoxx Belgium Tom White on building Hadoop apps with CDK
Nov. 16 Los Angeles Big Data Camp LA Alex Behm on Impala
Nov. 20 Boulder, Colo. Boulder/Denver Big Data Meetup John Darrah on Hadoop 101
Dec. 2 Tokyo Cloudera Manager Meetup Sho Shimauchi, Kiyoshi Mizumaru: What is Cloudera Manager?

Let a Thousand Hadoop How-Tos Bloom

History teaches us that ecosystem growth is fueled by enthusiasm, tools (including frameworks and APIs), and knowledge in roughly equal measures. To this point, the Apache Hadoop ecosystem has been blessed with the first two ingredients – thanks to the magic of open source – but in the third category, there is still plenty of work to be done.

This Month in the Ecosystem (September 2013)

Welcome to our third edition of “This Month in the Ecosystem,” a digest of highlights from September 2013 (never intended to be comprehensive; for completeness, see Hadoop Weekly).

Note: there were a few other interesting developments this week, but out of respect for the calendar, I’ll address them next month.

Newer Posts Older Posts