A New Web UI for Spark

The team behind Hue, the open source Web UI that makes Apache Hadoop easier to use, strikes again with a new Spark app.

Editor’s note: This post was recently published on the Hue blog. We republish it here for your convenience.

Hi Spark Makers!

Hue application for Apache Spark (incubating) was recently created. It lets users execute and monitor Spark jobs directly from their browser and be more productive.

We previously released the app with an Apache Oozie submission backend but switched to the Spark Job Server (SPARK-818) contributed by Ooyala and Evan’s team at the last Spark Summit. This new server will enable a real interactivity with Spark and is closer to the community.

We hope to work with the community and have support for Python, Java, direct script submission without compiling/uploading and other improvements in the future!

Get Started!

Currently only Scala jobs are supported and programs need to implement this trait and be packaged into a jar. Here is a WordCount example. To learn more about Spark Job Server, check its README.

Requirements

We assume you have Scala installed on your system.

Get Spark Job Server

Currently on github on this branch:

git clone https://github.com/ooyala/incubator-spark.git spark-server
cd spark-server
git checkout -b jobserver-preview-2013-12 origin/jobserver-preview-2013-12

 

Then type:

sbt/sbt
project
jobserver re-start

 

Get Hue

Currently only on github (will be in CDH 5 Beta 2):

https://github.com/cloudera/hue#getting-started

If Hue and Spark Job Server are not on the same machine, update the hue.ini property in desktop/conf/pseudo-distributed.ini:

[spark]
  # URL of the Spark Job Server.
  server_url=http://localhost:8090/

 

Get a Spark Example to Run

Then follow this walkthrough and create the example jar that is used in the video demo.

As usual, feel free to comment in the Hue community forum or @gethue! About questions directly related to Job Server, participate on the pull requestSPARK-818, or the Spark user list!

Filed under:

1 Response
  • doug chang / June 30, 2014 / 9:28 AM

    jobserver commands after a cd into the github directory should be:

    osprompt>sbt/bin/sbt
    sbtprompt>project job-server
    sbtprompt>restart

Leave a comment


three + = 6