How-to: Write a Cloud Provider Plugin for Cloudera Director

Categories: Cloud How-to

Cloudera Director 1.5 introduces a new plugin architecture to enable support for additional cloud providers. If you want to implement a plugin to add integration with a cloud provider that is not supported out-of-the-box, or to extend one of the existing plugins, these details will get you started.

As discussed in our previous blog post, the Cloudera Director Service Provider Interface (Cloudera Director SPI) defines a Java interface and packaging standards for Cloudera Director plugins. Let’s take a look at what it takes to implement a plugin.

Before You Begin

“The scariest moment is always just before you start.”

—Stephen King, On Writing: A Memoir of the Craft

These instructions are for plugin authors, and assume familiarity with Java, Maven, Git, and GitHub. They also assume that you are familiar with the cloud provider for which you are building the plugin, and that you understand how to use Java to interact with the provider.

You should have access to the following GitHub repositories:

  • Cloudera Director SPI contains the interfaces you will be implementing, numerous infrastructure classes to simplify your implementation, and information on how to package your plugin. The README file on the front page of the repository has detailed implementation instructions and links to the online Javadoc.
  • Cloudera Director SPI TCK contains the technology compatibility kit (TCK) that you use to verify that your plugin works correctly and satisfies the plugin packaging requirements. The README file tells you how to build and run the TCK.
  • Amazon Web Services (AWS) plugin contains a plugin implementation for AWS that you can use as an example or starting point. It was implemented in tandem with the Cloudera Director SPI, and illustrates best practices for plugins. The README file discusses the implementation and provides linkages between the concrete implementation classes in the plugin and the corresponding interfaces and concepts from the Cloudera Director SPI documentation.
  • (optional) Google Cloud Platform plugin contains a plugin implementation for Google Cloud Platform that you can use as an example, especially of good plugin unit testing. Engineers on the Google Cloud Platform team and the Cloudera Director team collaborated to produce this plugin.
  • (optional) Example: Bring Your Own Nodes (BYON) plugin contains a simple plugin implementation that you can use as an example. It has some ideas that might be useful if your plugin is designed to run on your own hardware.

To test your plugin, you will need appropriate credentials on your cloud provider.

Development Strategy

“A beginning is the time for taking the most delicate care that the balances are correct.”

— Frank Herbert, Dune

The Cloudera Director SPI documentation is organized top-down to provide contextual understanding of the cooperating components of a plugin. When implementing a plugin, it is probably easier to work bottom-up. This provides some advantages:

  • The lowest-level classes are the ones that interact with the cloud environment and really form the core of your plugin. If you have written code to interact with the cloud provider before, you will be on familiar ground. The instance classes will correspond closely to domain model objects in the provider’s SDK or web service. The template classes will correspond closely to request objects. The resource provider methods will exercise familiar calling patterns against the provider’s API.
  • The higher-level classes depend on the lower-level classes. If you implement the higher-level classes first, you will probably have to stub out the lower-level classes.

You should rely heavily on the AWS plugin as an example. If you are comfortable with refactoring, you should consider cloning it as a starting point and modifying and renaming classes to meet your needs. The infrastructure classes in the SPI are designed to simplify your implementation, and the AWS implementation already takes advantage of these classes. It is open source with good reason!

You should write tests early. Long before you reach the point of running the Cloudera Director SPI TCK to validate your implementation, you can use tests to verify your interaction with the cloud environment.

Development Tips

“Hanc marginis exiguitas non caperet.”

— Pierre de Fermat, marginal note

This blog post cannot take the place of the detailed instructions in the Cloudera Director SPI documentation (README and Javadoc), but here are a few additional tips to help you along the way:

  • Focus on implementing a compute provider before thinking about implementing a database server provider. Cloudera Director users will still be able to define references to manually created database servers in your cloud environment.
  • The most challenging part of the implementation will be the compute provider itself, especially the allocate method, which provisions compute instances in the cloud environment. Implementing the instance template is just about making sure you have the configuration information you need to define the instances to be created, and implementing the instance class is just about representing the resulting instances. You do the real work in the allocate method.
  • Packaging your plugin is easier if you minimize your plugin’s external dependencies, to simplify the use of the Maven shade plugin.

Getting Help

“Only one person in a million becomes enlightened without a teacher’s help.”

— Bodhidharma, Bloodstream Sermon

For additional help with your plugin implementation, reach out to the Cloudera Director Community Forum.

John Adair is a Software Engineer at Cloudera.