Making a coffee run? Upgrade CDW while you are gone.

Making a coffee run? Upgrade CDW while you are gone.

Upgrading components of Cloudera Data Warehouse (CDW) is as easy as one decision and a few clicks. While the Database (DB) Catalog and Virtual Warehouses (VWs) need to be compatible, the CDW upgrade framework understands the interoperability constraints between them. 

Finding upgrades:

CDW UI also has visual alerts to indicate when a new release is available. The UI periodically checks for new releases and publishes alerts to UI tiles in the form of an “Upgrade” icon like you see below:

Upgrade Icon

It is also possible to check for upgrades manually. From the context menu, select Upgrade to check for new releases.

Latest releases vs. Compatible releases:

Generally speaking, a release is compatible with the current release if it does not require an upgrade of its dependents or dependencies. For DB Catalogs, it means that no upgrades are needed for its VWs. For VWs, it means no upgrade is needed for its DB Catalog.

On the contrary, some releases contain changes that do require upgrade to other components as well. For example, a Hive VW could be using new APIs introduced in newer releases of Hive Metastore/DB Catalog. So upgrading to such a release requires an upgrade to the DB Catalog as well. In contrast, upgrading a DB Catalog does not necessarily require an upgrade to the Hive VW. Such releases are referred to as “Latest releases” in the CDW UI’s upgrade wizard.

This knowledge is embedded into the CDW upgrade framework. Users are shown upgrade choices as “Latest version” vs “Compatible version”. Depending on what new releases are available and what the current version is, users can expect to see one or both choices (or none if the release is current).

Example Scenario:
Current environment:
DB Catalog@R7.2.1.0-240
Hive Virtual Warehouse@R7.2.1.0-240
Impala Virtual Warehouse@R7.2.1.0-240

Available new releases:
DB Catalog@R7.2.1.0-250 → contains new API for faster metadata
Hive Virtual Warehouse@R7.2.1.0-275 → contains new features but no changes to DB Catalog APIs used. 
Impala Virtual Warehouse@R7.2.1.0-300 → contains fix that uses new API added in DB Catalog for faster metadata access

DB Catalog can be upgraded to R7.2.1.0-250 without need to upgrade either of the virtual warehouses. The current VW releases will continue to work with the new release of DB Catalog. In the upgrade wizard, this new release of DB Catalog will be referred to as “Compatible release” as all the APIs from previous releases are still supported.

Hive Virtual Warehouse can be upgraded to R7.2.1.0-275 without having to upgrade DB Catalog. In the upgrade wizard, this new release of Hive Warehouse will be referred to as “Compatible release”

Impala Virtual Warehouse's new release R7.2.1.0-300, has a fix that uses a new API added in DB Catalog,R7.2.1.0-250, to improve cache performance in Impala’s catalogd. So to upgrade Impala, DB Catalog needs to be upgraded first but an upgrade to Hive Virtual Warehouse is not required. In the upgrade wizard, this release of Impala will be referred to as “Latest release”.

What to upgrade?

An environment can consist of warehouses and catalogs at different release levels. As long as the virtual warehouses are compatible with DB Catalog release, they can be upgraded independent of each other. So there can be multiple Hive Warehouses, for the same DB catalog, each at different release levels. This provides great flexibility for individual teams/organizations.

What You Will See:

When you launch the upgrade wizard, you can expect to see one of the following options

1.” Already up-to-date,” in other words, no new releases:

Already up-to-date

2. Upgrade to “Compatible version” or “Cancel”:

Compatible version

3.The wizard shows slightly different buttons for DB catalogs and VWs. One can expect to see something like this:

    • DB CATALOG—Upgrade to either “Latest version” or “Compatible version” or “Cancel”
  • DB CATALOG
    • VW—“Upgrade Virtual Warehouse” (VW upgrade only) or “Upgrade Database Catalog” (requires DB Catalog upgrade) or “Cancel”

Upgrade Virtual Warehouse

 

Clicking on one of these choices initiates the upgrade. The CDW UI shows the state of the upgrade (Upgrading or Starting or Started)

Now back to the one decision the user has to make, what version to upgrade to. What’s it going to be? 

Enjoy your coffee!!

Naveen Gangam
Software Engineer
More by this author

Leave a comment

Your email address will not be published. Links are not permitted in comments.