Doing DevOps with Cloudera Manager

More and more customers are using automation/configuration management frameworks alongside Cloudera Manager.

As Apache Hadoop clusters continue to grow in size, complexity, and business importance as the foundational infrastructure for an Enterprise Data Hub, the use cases for a robust and mature management console expand. 

Dev Ops

As those clusters become larger and more complex, many operators are looking to use configuration management/automation frameworks like Ansible, Chef, and Puppet alongside Cloudera Manager to help automate the deployment and configuration process. (Others are rolling their own via the open source Cloudera Manager REST API.)

Here are some examples of how-to’s that cover these technologies: 

Blogger James Ruddy recently added to this list with posts about  1) using Puppet to deploy Cloudera Manager (based on the Puppet module written by Mike Arnold) and then 2) using Cloudera Manager to deploy Hadoop. Thanks to James for sharing his hard-earned knowledge!

If you done something similar with Chef or Salt, let us know in comments.

Justin Kestelyn is Cloudera’s developer outreach director.

3 Responses
  • Loren J. Rittle / December 18, 2013 / 1:19 PM

    We have created the Salt state (similar to an ansible playbook) to install cloudera-manager-agent and cloudera-manager-daemons on at least gateway machines to be further managed by the GUI (it may be enough to work for core cluster nodes managed by the GUI but we haven’t tried that). It is 33 lines of salt configuration plus 6 lines of non-templated /etc/cloudera-scm-agent/config.ini.

  • Justin Kestelyn (@kestelyn) / December 19, 2013 / 4:32 PM


    How about a guest blog post for us on that, if you’re open to sharing?

  • Marek Obuchowicz (KoreKontrol Hadoop Platform) / July 18, 2014 / 3:10 AM

    Is there any chance to get SaltStack implementation? We’re using salt as it seems to be the most effective in terms of ease of development and understanding for developers (after years of using puppet) – any shared code would definitely help doing things better :)

Leave a comment

+ 7 = nine