Cloudera Enterprise 5.7 is Released

Categories: CDH Cloudera Manager Cloudera Navigator Hive Spark

Cloudera Enterprise 5.7 is now generally available (comprising CDH 5.7, Cloudera Manager 5.7, and Cloudera Navigator 2.6).

Cloudera is excited to announce the general availability of Cloudera Enterprise 5.7! Main highlights of this release include production-ready Hive-on-Spark functionality, which will help users accelerate their use of Apache Spark as a data processing standard; 4x performance gains for Apache Impala (incubating); easier cluster configuration and utilization reporting; and end-to-end encryption for Apache Spark data.

The release also contains a long list of incremental improvements across the stack, in addition to the usual hundreds of bug fixes (some of which were uncovered during our multi-dimensional hardening/QA process). Here is a partial list of those improvements (see the Release Notes for a full list):

Performance & Scale

  • Hive-on-Spark GA (graduates from Cloudera Labs)
  • 4x performance gains for Impala: Better join ordering and cardinality estimation, faster query startup, codegen and code optimizations, more
  • Support for the Apache HBase WAL on SSD
  • Support for the HBase-Spark module (graduates from Cloudera Labs)
  • Dramatic performance improvement for backups/DR

Usability & Management

Security & Governance

  • Improved Apache Sentry HDFS sync feature
  • Encryption over the wire/on disk for Spark data
  • Support for Kerberos and LDAP auth on the same HiveServer2 instance
  • New “business views” for data lineage; new managed/secure metadata within Cloudera Navigator

New or Updated Open Source Components

  • Apache Spark 1.6 (including support for Spark SQL and Dataframes in PySpark and the spark.ml package and Pipelines API)
  • Apache HBase 1.2
  • Apache Impala (incubating) 2.5
  • Apache Kafka 0.9 (separate install)

New or Updated Platform Support

  • RHEL/CentOS/OEL 7.2
  • SLES 11 SP4
  • Debian 7.8
  • JDK 7_80 and JDK 8_60

Over the next few weeks, we’ll publish blog posts that cover some of these features in detail. In the meantime:

As always, we value your feedback; please provide any comments and suggestions through our community forums. You can also file bugs via issues.cloudera.org.

facebooktwittergoogle_pluslinkedinmailfacebooktwittergoogle_pluslinkedinmail

13 responses on “Cloudera Enterprise 5.7 is Released

  1. Mathias

    With the new parcel of 5.7 the cloudera manager tells me, i first have to update from cm-version 5.6 to 5.7 – but that’s exactly what i’m trying to do. Any idea what’s the problem about this?

    “Error for parcel CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel : Parcel version 5.7.0-1.cdh5.7.0.p0.45 is not supported by this Cloudera Manager. Upgrade Cloudera Manager to at least 5.7.0 before using this parcel version.”

    1. Justin Kestelyn Post author

      To be clear: you need CM 5.7 installed first before you can download the CDH 5.7 parcel. Have you done that?

      1. Mathias

        I thought i could update both on the fly – that was clearly the problem! Thanks for that!

  2. Emanuele Cesena

    Two questions:
    1. Can Spark Streaming be used with Kafka 0.9? (in the known issues I can read “cannot consume from *secure* Kafka”… what about “normal” Kafka?
    2. Which version of Akka is Spark 1.6 compiled with? (again, in the known issues, I can read about Spark 1.5, but no mentions about 1.6)
    Thanks a lot, E.

  3. Fabricio

    About “Dramatic performance improvement for backups/DR”, what are the issues (jira or some docs) that describe it? Technically, could you explain more, please?

    Thanks

    1. Justin Kestelyn Post author

      Fabricio,

      We’ll publish more details about this soon and will share it with you when we do.

  4. Haitao Wu

    When you say Hive on Spark is GA. Which version of Hive are you talking about? I guess the Spark version is 1.6.0?

  5. Sergey Zaharov

    Hello, I have cloudera mangager 5.7 and activated cdh 5.3.2 . I want to perform upgrade to cdh 5.7, but available parsel is cdh 5.3.10 only. Is i need to do upgrade to cdh 5.3.10 first?

    1. Justin Kestelyn Post author

      Sergey:
      1. Log into the Cloudera Manager Admin console.
      2. From the Home > Status tab, click next to the cluster name and select Upgrade Cluster. The Upgrade Wizard starts.
      3. If the option to pick between packages and parcels displays, select the Use Parcels option.
      4. Click the “Modify the Remote Parcel Repository URLs” link to go to the Parcel Configuration Settings page. When there, follow these instructions to confirm that CM is pointing toward the correct parcel repository:

      http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_parcels.html#cmug_topic_7_11_5

      1. Sergey Zakharov

        Thank you for response, Justin!
        It helped me, cloudera manager saw new 5.7 parcels only when substitution variable {latest_supported} has been used. Repository urls like “http://archive.cloudera.com/cdh5/parcels/latest/” still has no effect.