One of the complexities of Apache Hadoop is the need to deploy clusters of servers, potentially on a regular basis. At Cloudera, which at any time maintains hundreds of test and development clusters in different configurations, this process presents a lot of operational headaches if not done in an automated fashion. In this post, I’ll describe an approach to cluster automation that works for us, as well as many of our customers and partners.
At Cloudera engineering,
API access was a new feature introduced in Cloudera Manager 4.0 (download free edition here.). Although not visible in the UI, this feature is very powerful, providing programmatic access to cluster operations (such as configuration and restart) and monitoring information (such as health and metrics). This article walks through an example of setting up a 4-node HDFS and MapReduce cluster via the Cloudera Manager (CM) API.
Cloudera Manager API Basics
The CM API is an HTTP REST API,
Hue 2.0.1 has just been released. 2.0.1 represents major improvement on top of the Hue 1.x series. To list a few key new features:
- Frontend has been re-implemented as full screen pages.
- Hue supports LDAP (OpenLDAP and Active Directory). Hue can be configured to authenticate against LDAP. Additionally, Hue can import users and groups from LDAP, and refresh group membership from LDAP.
- Hue supports per-application authorization.
The HUE (aka. Hadoop User Experience) project [download|installation|manual] started as Cloudera Desktop about a year ago. The old name “Desktop” really refers to a desktop look-and-feel, since HUE is a web UI for Hadoop. Beyond delivering a suite of web applications, it is also a platform for building custom applications with a nice UI library. Gradually, we realized how much value such a UI platform would bring to the community,
What can you do with a lot of extra space? No, I do not mean HDFS. I am talking about the new Cloudera Global Headquarters.
Cloudera’s old office became so crammed that our CEO had to take calls from his car, and our Architect shared a desk. So the new space in Palo Alto was much anticipated (not to mention its extreme proximity to Fry’s). It features eight conference rooms,