Cloudera Altus Director helps you deploy, scale, and manage Cloudera clusters on AWS, Microsoft Azure, or Google Cloud Platform. Altus Director both enables and enforces the best practices of big data deployments and cloud infrastructure. Altus Director’s enterprise-grade features deliver a mechanism for establishing production-ready clusters in the cloud for big data workloads and applications in a simple, reliable, automated fashion. In this post, you will learn about new functionality and changes in release 6.2.
Cloudera Altus Director Overview
For those new to Altus Director, let’s revisit what it does.
- On-demand creation and termination of clusters: Altus Director can allocate and configure Cloudera Manager instances and highly available CDH clusters in various cloud providers. A single Altus Director instance can manage multiple cloud provider environments as well as the separate lifecycles of multiple Cloudera Manager instances and clusters.
- Multi-cloud support: Altus Director supports creating clusters in Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) through its cloud provider plugin architecture. A single Altus Director instance can work with multiple cloud providers at once. Because the plugin specification is open source, you can create a plugin to support other providers, either in-house or public.
- On-demand grow and shrink of clusters: One of the main benefits of running CDH clusters in the cloud is being able to provision additional instances when demand increases, and to terminate instances when demand decreases. Altus Director, in concert with Cloudera Manager, does the work required to add new instances to and remove existing ones from your CDH clusters.
- Programmatic and repeatable instantiation of clusters: Altus Director can consume cluster definitions specified in HOCON configuration files submitted through the Altus Director CLI or in JSON or HOCON input sent to the Altus Director API. A cluster definition can include custom scripts to run after instance provisioning and cluster setup, or before cluster termination, to perform tasks like installing additional packages, configuring system settings, or saving important data.
- Long-running cluster support: Long-running clusters often require actions like upgrading CDH and Cloudera Manager, changing the topology of the cluster, and reconfiguring the cluster. Altus Director supports such modifications when using Cloudera Manager 5.11 and above.
- Usage-based billing for Cloudera services: Usage-based billing can help you optimize your expenditures for transient clusters. With a pay-as-you-go billing ID from Cloudera, you can use your Cloudera Enterprise license as usual, but you are only charged for CDH services when they are running.
- Security: Altus Director, like other Cloudera offerings, is committed to enabling secure deployments and applications. Altus Director’s own database is automatically encrypted, and Altus Director helps you configure Cloudera Manager and CDH clusters with Kerberos authentication, TLS for on-the-wire privacy, and Cloudera Navigator for auditing, data lineage, and data discovery.
- Powerful web user interface: Altus Director’s user interface provides a single dashboard to assess the health of all your clusters across all cloud providers and all Cloudera Manager deployments. It can also be used to bootstrap new clusters, grow and shrink existing clusters, and terminate clusters that are no longer needed. Exploring the web user interface is a great stepping stone to using the configuration file or API to deploy production-ready clusters.
Changes in Cloudera Altus Director 6.2
Automatic Instance Groups
Director now supports launching instances in an EC2 Auto Scaling Group (ASG) or Azure Virtual Machine Scale Set (VMSS). These instances typically launch more reliably, and in the case of Azure, often support new features earlier than traditional Virtual Machine (VM) instances. Director does not yet support external changes to the size of the ASG or VMSS (e.g. through the cloud provider portal), so continue to use Director’s grow/shrink cluster updates if you need to change the size of such an instance group. Please refer to the Cloudera Director documentation for more information.
Improved Azure functionality
Azure MySQL Database Support
Azure MySQL databases can now be provisioned through Cloudera Altus Director and utilized by Cloudera Manager and other various cluster services such as the Hive Metastore, Sentry, Cloudera Manager itself for its database, and so on. Please refer to the Cloudera Director documentation for more information.
New Azure Instance Types
Azure’s Fsv2-series, Dsv3-series and Esv3-series VM sizes are now supported.
Proxy support
Cloudera Director can now provision Azure instances through an HTTP proxy.
Cloudera Manager Configuration Sync
Cloudera Altus Director has expanded how it monitors Cloudera Manager configurations and synchronizes them with Altus Director’s internal state. The values of configurations for Cloudera Manager and the management service and roles are copied back into the deployment template, so the refreshed values will be present in any subsequent export of the deployment (direct or in the deployment section of a cluster export). Sensitive values are stored as “REDACTED” so they are not exposed via the UI or in the exported configuration.
Cloudera Manager License Update
In addition to monitoring Cloudera Manager configurations, Altus Director will monitor Cloudera Manager for license changes and reflect those as well. This will help new and prospective usage-based billing users. Users who need to update their existing Cloudera Manager license will be able to do so without interruption, and users will be able to add PAYG billing on clusters that were initially set up with a trial license and later migrated to a full license.
Using Cloudera Altus Director
For more information on what’s new in this release, check out the Altus Director 6.2 section of the New Features and Changes in Altus Director page in the Altus Director documentation. If you’re ready to give the latest version of Altus Director a try, here are the ways you can get started.
- Download Altus Director from our website, where you can also find its user guide, to start fresh or upgrade from an existing installation. See the upgrade documentation for detailed instructions on how to upgrade Altus Director.
- Use sample configuration files and scripts as starting points for setting up your clusters.
Send questions or feedback to the Altus Director community forum.
Michael Wilson is a software engineer at Cloudera.
Editor's Choice
