Cloudera Engineering Blog · Hadoop Posts
In a fast-moving project like Apache Hadoop, there are always exciting new features introduced in each release. While it is tempting to make the most of these new features by upgrading to the latest release, users are often concerned about their code continuing to run.
In this post, you’ll get an overview of the the Hadoop API annotations and compatibility policies. Hadoop annotates specific APIs to be safe for use by end-users. By using these APIs, users can ensure their code works across a set of releases and be aware of what releases it might not work against.
Hadoop Releases and Compatibility
Customer Spotlight: Learn How Edo Closes the Advertising Loop with Hadoop at Cloudera Sessions Milwaukee
The Cloudera Sessions fall series is coming to a close next week, but first we’ll make a final stop in Milwaukee, Wisconsin (on Oct. 17), where attendees will hear about edo — a company that is revolutionizing the advertising space by closing the loop between promotions and point-of-sale transactions.
In Milwaukee, edo CTO Jeff Sippel will engage in a fireside chat with Cloudera’s VP of marketing, Alan Saldich. At edo, Jeff is responsible for the strategy, planning, and execution for the systems — including Apache Hadoop — that power the edo offer platforms.
Below please find our regularly scheduled quarterly update about where to find tech talks by Cloudera employees this year – this time, for October through December 2013. Note that this list will be continually curated during the period; complete logistical information may not be available yet.
As always, we’re standing by to assist your meetup by providing speakers, sponsorships, and schwag!
|Oct. 1||Aarhus, Denmark||GOTO Aarhus||Eva Andreeason on Hadoop use cases|
|Oct. 8||Sunnyvale, Calif.||Hadoop Happy Hour||Kathleen Ting and Jarek Cecho sign books!|
|Oct. 9||Santa Clara, Calif.||IEEE BigData Conference||Amr Awadallah on Hadoop use cases|
|Oct. 9||San Francisco||SF Hadoop Users||Eric Sammer on Hadoop app development (panelist)|
|Oct. 10||Sydney||DataCon||Sean Owen on data science|
|Oct. 15||Durham, NC||TriHUG||Mark Miller on Solr+Hadoop|
|Oct. 15||Mountain View, Calif.||Oracle NoSQL & Big Data Meetup||Mike Olson on virtues of key-value stores|
|Oct. 15-17||Burlingame, Calif.||Big Data TechCon||Apache Hive workshop with Mark Grover|
|Doug Cutting on the Hadoop revolution|
|Hadoop app development (CDK) workshop with Ryan Blue|
|Jonathan Seidman on extending data infrastructure with Hadoop|
|Jonathan Seidman on the Hadoop ecosystem|
|Himanshu Vashishtha on HBase use cases|
|Kate Ting on Apache ZooKeeper|
|Kate Ting on 7 Deadly Hadoop Misconfigurations|
|Oct. 16||Dallas, Tex.||DFW Big Data||John Ringhofer on Impala|
|Oct. 17||Milwaukee, Wis.||Cloudera Sessions||Hadoop app development lab (on CDK) with Ryan Blue|
|Oct. 17||St. Louis, Mo.||St. Louis HUG||Tom Wheeler on Parquet|
|Oct. 18||Munich||HUG Munich||Lars George on Impala|
|Oct. 22||London||UK HUG||Sean Owen on Scalable Big learning|
|Oct. 23||Seattle||Seattle Scalability Meetup||Ronan Stokes on Cloudera Search|
|Oct. 24||Palo Alto, Calif.||Bay Area HBase User Group||Michael Stack on HBase 0.96|
|Oct. 24||Raleigh, NC||All Things Open||Josh Wills on open source innovation|
|Oct. 28-30||New York||Strata Conference + Hadoop World 2013||Mike Olson on Hadoop’s impact on data management|
|Doug Cutting on the future of Hadoop|
|Henry Robinson on workload diversity in Hadoop|
|Hadoop app development (CDK) workshop with Eric Sammer|
|Matt Brandwein on leveraging mainframe data with Hadoop|
|Aaron T. Myers and Shreepadma Venugopalan on Hadoop security|
|Jayant Shekar on machine data analytics|
|Amandeep Khurana on Monsanto’s use case for Hadoop & HBase|
|Philip Zeyliger on debugging distributed systems|
|Greg Rahn on Impala performance tuning|
|Jon Hsieh on HBase roadmap|
|Oct. 28||New York||NYC HUG||Arvind Prabhakar on Apache Sentry (incubating)|
|Oct. 28||New York||Sqoop User Meetup||Abe Elmahrek on the Sqoop2 app for Hue|
|Oct. 29||New York||Impala + Parquet Meetup||Greg Rahn on Impala+Parquet performance tuning|
|Oct. 29||New York||Cloudera Manager Meetup||Aditya Achara on Cloudera Manager success stories|
|Oct. 30||New York||Apache Sentry User Meetup||Arvind Prabhakar and Shreepadma Venugopalan with a Sentry overview|
|Oct. 30||Philadelphia||Chariot Data IO Conference||Lars George on HBase sizing as well as on Parquet|
|Nov. 6||Chantilly, Va.||Open Source Search Conference||Alex Moundalexis on Search+Hadoop|
|Nov. 6||Munich||JAX Munich||Lars George on HBase and Impala|
|Nov. 7||Tokyo||Cloudera World Tokyo||Kiyoshi Mizumaru on CDH|
|Sho Shimauchi on Cloudera Manager|
|Tatsuo Kawasaki witha Hadoop 101|
|Daisuke Kobayashi on Hadoop ops|
|Nov. 11||London||UK HUG||Marcel Kornacker on Impala|
|Nov. 12-13||London||Strata London||Sean Owen on Scalable Big Learning; Tom White on Hadoop app development with CDK|
|Nov. 12||San Francisco||QCon SF||Josh Wills on machine learning|
|Nov. 13||Washington DC||LISA 2013||John Ridley on Hadoop 101 for sysadmins|
|Nov. 14||Seoul||Tech Planet Korea||Michael Stack on HBase roadmap|
|Nov. 14||Tokyo||Cloudera Manager Meetup||Sho Shimauchi, Kiyoshi Mizumaru: What is Cloudera Manager?|
|Nov. 14||Antwerp||Devoxx Belgium||Tom White on building Hadoop apps with CDK|
|Nov. 16||Los Angeles||Big Data Camp LA||Alex Behm on Impala|
|Nov. 20||Boulder, Colo.||Boulder/Denver Big Data Meetup||John Darrah on Hadoop 101|
|Dec. 2||Tokyo||Cloudera Manager Meetup||Sho Shimauchi, Kiyoshi Mizumaru: What is Cloudera Manager?|
History teaches us that ecosystem growth is fueled by enthusiasm, tools (including frameworks and APIs), and knowledge in roughly equal measures. To this point, the Apache Hadoop ecosystem has been blessed with the first two ingredients – thanks to the magic of open source – but in the third category, there is still plenty of work to be done.
Welcome to our third edition of “This Month in the Ecosystem,” a digest of highlights from September 2013 (never intended to be comprehensive; for completeness, see Hadoop Weekly).
Note: there were a few other interesting developments this week, but out of respect for the calendar, I’ll address them next month.
It’s common to hear people describe themselves as being “left-brained” or “right-brained” based on their tendency to be more logical and mathematically driven (left-brained), or, conversely, to be intuitive and creatively driven (right-brained). For example, people who prefer math over art are often considered left-brained. People who get a higher verbal score on their SATs than for math are often considered right-brained.
In general, language and creative writing are considered right-brained exercises. Many people also associate marketing and advertising as a right-brained function, whereas engineering is considered very left-brained.
In this installment of “Meet the Project Founder,” we speak with Josh Wills (@josh_wills), Cloudera’s Senior Director of Data Science and founder of Apache Crunch and Cloudera ML.
What led you to your project idea(s)?
When I first started at Cloudera in 2011, I had a fairly vague job description, no real responsibilities, and wasn’t all that familiar with the Apache Hadoop stack, so I started working on various pet projects in order to learn more about the tools and the use cases in domains like healthcare and energy.
I’ve always held a strong bias that education is most effective when the student learns by doing. As a developer of technical curricula, my goal is to have training participants engage with real and relevant problems as much as possible through hands-on exercises. The high rate at which Apache Hadoop is changing, both as a technology and as an ecosystem, makes developing Cloudera training courses not only demanding but also seriously fun and rewarding.
I recently undertook the challenge of upgrading the Cloudera Administrator Training for Apache Hadoop. I more than quadrupled the amount of hands-on exercises from the previous version, adding a full day to the course. At four days, it’s now the most thorough training for Hadoop administrators and truly the best way to start building expertise.
In December 2012, we described how an internal application built on CDH called Cloudera Support Interface (CSI), which drastically improves Cloudera’s ability to optimally support our customers, is a unique and instructive use case for Apache Hadoop. In this post, we’ll follow up by describing two new differentiating CSI capabilities that have made Cloudera Support yet more responsive for customers:
Why would any company be interested in searching through its vast trove of email? A better question is: Why wouldn’t everybody be interested?
Email has become the most widespread method of communication we have, so there is much value to be extracted by making all emails searchable and readily available for further analysis. Some common use cases that involve email analysis are fraud detection, customer sentiment and churn, lawsuit prevention, and that’s just the tip of the iceberg. Each and every company can extract tremendous value based on its own business needs.