While the vast majority of the Hadoop development discussion takes place on the Apache Jira and various project mailing lists, it’s often useful to meet face to face for high bandwidth discussion. To that end, Facebook hosted the first Apache Hadoop contributors meeting yesterday at their campus in Palo Alto. Cloudera, Facebook, Yahoo! and the Apache HBase team were well-represented. It was great to see a broad cross section of Hadoop developers in one room. Contributor meetings will be held on a monthly basis, at a rotating location. While any Hadoop project contributor is welcome to attend, the current focus of the meetings is HDFS and MapReduce. The goal of the discussion is to surface and flesh out ideas rather than make decisions, which happens on the development lists. If you’ve got ideas to add check out the meeting notes and continue the discussion.
Sanjay Radia kicked off the meeting with a discussion of development priorities. Hadoop has become a platform and industry standard for data storage and analytics. What advances are most important to users? How do we continue to innovate without disrupting the installed base? Development must maintain and improve the quality that has allowed companies to adopt Hadoop in their production environments. Fortunately there is broad agreement among contributors on development priorities: availability, compatibility, security, scalability and performance.
The next topic was releases. Tom White gave an update on the upcoming 0.21 release. The release has branched and is now in feature freeze; the focus is now shifting to closing out issues that block the release. You can hear more about the 0.21 release at the next Bay Area Hadoop user group. Chris Douglas recently proposed Hadoop adopt release managers. Owen O’Malley led a discussion on the role and responsibilities of the release manager, and the Hadoop release process.
The agenda for the next meetup is a more formal way to propose and adopt new features. We’d love to see new faces, so if you’re a Hadoop contributor and want to attend, sign up for the next meetup. If you’d like to contribute to Hadoop, the Hadoop wiki is a good place to start. See you next month at the new Cloudera HQ!