Faster Swarms of Data : Accelerating Hive Queries with Parquet Vectorization

Categories: CDH Hive Parquet Performance

Background

Apache Hive is a widely adopted data warehouse engine that runs on Apache Hadoop. Features that improve Hive performance can significantly improve the overall utilization of resources on the cluster. Hive processes data using a chain of operators within the Hive execution engine. These operators are scheduled in the various tasks (for example, MapTask, ReduceTask, or SparkTask) of the query execution plan. Traditionally, these operators are designed to process one row at a time.

Read more

What’s New in Cloudera Altus Director 6.1?

Categories: CDH Cloud Cloudera Director

Cloudera Altus Director helps you deploy, scale, and manage Cloudera clusters on AWS, Microsoft Azure, or Google Cloud Platform. Altus Director both enables and enforces the best practices of big data deployments and cloud infrastructure. Altus Director’s enterprise-grade features deliver a mechanism for establishing production-ready clusters in the cloud for big data workloads and applications in a simple, reliable, automated fashion. In this post, you will learn about new functionality and changes in release 6.1.

Read more

Cloudera Enterprise 6.1.0 is Now Available

Categories: Accumulo CDH Cloudera Manager Cloudera Navigator Kafka Kudu Search Tools

We are pleased to announce the general availability of Cloudera Enterprise 6.1.0, the modern platform for machine learning and analytics optimized for the cloud. This release delivers several new capabilities, improved usability, and better performance.

As usual, the release includes a number of quality enhancements, bug fixes, and other improvements across the stack. Here is a partial list of what’s included (see the Release Notes for a full list):

Data Engineering

Cloudera Enterprise 6.1 now supports Spark Structured Streaming and enables micro-batch processing at ~100ms increments enabling ingest to query latencies in the Cloudera platform measured in seconds.

Read more

Assessment of Apache Impala Performance using Cloudera Manager Metrics – Part 1 of 3

Categories: CDH Cloudera Manager Impala Performance

For a user-facing system like Apache Impala, bad performance and downtime can have serious negative impacts on your business. Given the complexity of the system and all the moving parts, troubleshooting can be time-consuming and overwhelming.

In this blog post series, we are going to show how the charts and metrics on Cloudera Manager (CM) can help troubleshoot Impala performance issues. They can also help to monitor the system to predict and prevent future outages.

Read more

Cloudera Altus Director SDX Integration

Categories: Altus CDH Cloud Cloudera Director

Cloudera provides a pathway for sharing metadata from an Altus Director managed cluster with Cloudera Altus Data Engineering or Altus Data Warehouse clusters. This blog post outlines how to use Altus Director to set up the required infrastructure as well as configuring the CDH components to enable this functionality.

SDX for Cloudera Altus persists both Apache Hive metadata and Apache Sentry data access policies independently from clusters in SDX namespaces. In this way,

Read more