What’s New in Cloudera Altus Director 6.1?

Categories: CDH Cloud Cloudera Director

Cloudera Altus Director helps you deploy, scale, and manage Cloudera clusters on AWS, Microsoft Azure, or Google Cloud Platform. Altus Director both enables and enforces the best practices of big data deployments and cloud infrastructure. Altus Director’s enterprise-grade features deliver a mechanism for establishing production-ready clusters in the cloud for big data workloads and applications in a simple, reliable, automated fashion. In this post, you will learn about new functionality and changes in release 6.1.

Cloudera Altus Director Overview

For those new to Altus Director, let’s revisit what it does.

  • On-demand creation and termination of clusters: Altus Director can allocate and configure Cloudera Manager instances and highly available CDH clusters in various cloud providers. A single Altus Director instance can manage multiple cloud provider environments as well as the separate lifecycles of multiple Cloudera Manager instances and clusters.
  • Multi-cloud support: Altus Director supports creating clusters in Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) through its cloud provider plugin architecture. A single Altus Director instance can work with multiple cloud providers at once. Because the plugin specification is open source, you can create a plugin to support other providers, either in-house or public.
  • On-demand grow and shrink of clusters: One of the main benefits of running CDH clusters in the cloud is being able to provision additional instances when demand increases, and to terminate instances when demand decreases. Altus Director, in concert with Cloudera Manager, does the work required to add new instances to and remove existing ones from your CDH clusters.
  • Programmatic and repeatable instantiation of clusters: Altus Director can consume cluster definitions specified in HOCON configuration files submitted through the Altus Director CLI or in JSON or HOCON input sent to the Altus Director API. A cluster definition can include custom scripts to run after instance provisioning and cluster setup, or before cluster termination, to perform tasks like installing additional packages, configuring system settings, or saving important data.
  • Long-running cluster support: Long-running clusters often require actions like upgrading CDH and Cloudera Manager, changing the topology of the cluster, and reconfiguring the cluster. Altus Director supports such modifications when using Cloudera Manager 5.11 and above.
  • Usage-based billing for Cloudera services: Usage-based billing can help you optimize your expenditures for transient clusters. With a pay-as-you-go billing ID from Cloudera, you can use your Cloudera Enterprise license as usual, but you are only charged for CDH services when they are running.
  • Security: Altus Director, like other Cloudera offerings, is committed to enabling secure deployments and applications. Altus Director’s own database is automatically encrypted, and Altus Director helps you configure Cloudera Manager and CDH clusters with Kerberos authentication, TLS for on-the-wire privacy, and Cloudera Navigator for auditing, data lineage, and data discovery.
  • Powerful web user interface: Altus Director’s user interface provides a single dashboard to assess the health of all your clusters across all cloud providers and all Cloudera Manager deployments. It can also be used to bootstrap new clusters, grow and shrink existing clusters, and terminate clusters that are no longer needed. Exploring the web user interface is a great stepping stone to using the configuration file or API to deploy production-ready clusters.

Cloudera Altus Director UI

 

Changes in Cloudera Altus Director 6.1

Strengthened Security Through Improved Proxy Support

With Altus Director 6.1, you can now deploy clusters in a network that can only access the Internet via a proxy. See the documentation for more details.

Improved Cluster Customization Through Scripting Enhancements

Cloudera Altus Director supports a variety of lifecycle scripts to be executed on the cloud instances it provisions. This release features several enhancements to the scripting support.

Instance Pre-Termination Scripts

With Altus Director 6.1, you can now include scripts to run just before an instance is terminated, to perform actions such as:

  • Deregistering from an Active Directory domain
  • Clearing out custom DNS entries
  • Cleaning up networking
  • Saving off system logs or other files

These pre-termination scripts are a natural counterpart to instance bootstrap scripts.

Script Environment Variables

With Altus Director 6.1, you can now provide environment variables for scripts. This makes scripts more readable and reusable.

 

Script Execution Location

Altus Director defaults to running scripts in /tmp, which is forbidden under some security standards. With Altus Director 6.1, you can change this via the lp.ssh.tmpPath server configuration property, whose value specifies the location where scripts are run. This change applies to all scripts, not just user-supplied ones, across all deployments and clusters.

Better Cluster Lifecycle Through Relaxed Image Validation

Cloud provider machine images (such as, AWS AMIs and Azure VM images) may occasionally need to be deleted. For example, if a secret is inadvertently included in an image, the image should be deleted for security reasons. In previous versions of Altus Director, subsequent changes to a cluster using a deleted image could fail validation.

With Altus Director 6.1, image validation has been relaxed to allow for shrinking an instance group that includes an invalid image. Validation is unchanged for new or growing instance groups, to avoid later failures trying to provision instances using a nonexistent image.

More Flexibility Through OpenJDK Support

Cloudera Altus Director can now run on OpenJDK 8, and can install Cloudera Manager and CDH cluster nodes running on OpenJDK 8 as well when using the DIRECTOR_MANAGED JDK installation strategy.

Better Google Cloud Platform Experience Through Subnetworks

Thanks to a community contribution to the open source Altus Director plugin for Google Cloud Platform, you can now configure instances to use subnetworks.

Using Cloudera Altus Director

For more information on what’s new in this release, check out the Altus Director 6.1 section of the New Features and Changes in Altus Director page in the Altus Director documentation. If you’re ready to give the latest version of Altus Director a try, here are the ways you can get started.

Send questions or feedback to the Altus Director community forum.

John S. Adair is a software engineer at Cloudera.

Facebooktwittergoogle_pluslinkedinmailFacebooktwittergoogle_pluslinkedinmail

Leave a Reply

Your email address will not be published. Required fields are marked *