Developing Applications for HUE
Yesterday’s post gave an overview of the HUE (aka. Hadoop User Experience) project which was released in CDH3b2 and available on github. HUE is a graphical “desktop” style web application that runs in modern browsers (Firefox, Chrome, Safari, and IE8+) that allows users to interact with a Hadoop installation as if it were just another computer. They browse the file system, create and manage user accounts, view and edit files, upload files, and then use some Hadoop-specific applications like the Job Browser and Beeswax (our Hive app). Here’s a quick demo from yesterday’s post running through Beeswax. It’s about 10 minutes long, but even if you only watch the first 2 or 3 you’ll get an idea of what HUE is and what it can do.
This post is focused on what it means to develop for HUE, the source of which is available on github for all your forking pleasure.
Developing For HUE On The Server Side
Applications for HUE are usually implemented in Django, a popular MVC web framework that understands the application namespaces. On top of that, the SDK lets the application bundle and start helper daemons which might, for example, talk to various interfaces in Hadoop, HDFS, or one of the numerous other applications that ship with CDH3b2 (such as Hive in the video example above).
To develop applications for HUE developers need only be familiar with basic web development practices. Learning Django is pretty straight forward; there are a lot of terrific resources online, including a free online book, not to mention the excellent documentation on the Django project site itself.
Working with Web 1.0
Pulling Web 1.0 into HUE With JFrame
These “Web 1.0″ views aren’t that fun to use though. To convert these raw-looking “Web 1.0″ apps into slick user experiences with sortable tables, forms that validate inputs as you enter values, graphs, pretty navigation links and so on, developers pepper their HTML with semantic instructions that HUE knows how to convert into these highly interactive components.
Developing On the Client Side
Instead, we require only that they decorate the HTML they return to have certain properties. Here’s what the HTML returned in the example above looks like:
<table class="sortable" data-filters="HtmlTable FitText-Children" data-fit-text="td" cellpadding="0" cellspacing="0"> <thead> <tr> <th>ID</th> <th>TimeZone</th> <th>Name</th> <th>GEO Latitude</th> <th>GEO Longitude</th> </tr> </thead> <tbody> <tr> <td>22</td> <td>New York City, New York, United States of America</td> <td>America/New_York</td> <td>40.7255</td> <td>-73.9983</td> </tr> <tr> <td>23</td> <td>San Francisco, California, United States of America</td> <td>America/Los_Angeles</td> <td>37.7587</td> <td>-122.433</td> </tr> </tbody> </table>
<table class="sortable" data-filters="HtmlTable FitText-Children" data-fit-text="td" ...
These HTML5 data tags tell Behavior to run specific filters against these elements. These filters transform it into the screenshot above.
The HUE SDK UI Library
Currently the SDK library in HUE features roughly three dozen of these UI patterns and we add more every day. If you download and install HUE you’ll find an application called “JFrame Gallery” that demos all of them. It’s important to note that applications built in HUE don’t have to use these patterns. It’s easy to add your own if we’re missing something or if you don’t like the way the default one works. Whenever we write new applications we reuse these default patterns, but occasionally we find a need for a pattern that only makes sense for that application. When this happens we still write these HTML transformers, but they remain private to that specific application.
In today’s post I’ve only scratched the surface of how all this works. If you’d like to get a richer explanation I suggest you read through the SDK walkthrough and maybe download HUE and follow along, creating a simple web app using our tools.
We’re hoping to see people in the Hadoop community jump in and develop more applications for HUE and contribute to the ones already there. One of the benefits to this consolidated environment, aside from usability, the ability to cross-link apps, and not having to re-invent the wheel every time you want to browse the file system, is that anyone can jump in and contribute. If you think HUE could solve your problems if only it had feature X then we hope you’ll consider forking the repository on github and getting involved. If you have an idea for a killer app to make Hadoop better, we hope HUE is the right place for you to develop it. Sign up for the HUE Developer Group on Google Groups and let us hear from you.
If you’re looking for ideas of things you might develop on HUE, give one of these a shot:
- An HBase explorer
- A Yahoo! Pipes-style UI for Oozie
- A social gaming app that gives people points for doing interesting things to data (e.g. running a Hive query, using a Combiner, etc.) so you can quickly figure out the data experts in your organization.
- Surprise us. Go nuts.
Look for more posts in the future that focus more on the server side of HUE development.