After the GA of Apache Kudu in Cloudera CDH 5.10, we take a look at the Apache Spark on Kudu integration, share code snippets, and explain how to get up and running quickly, as Kudu is already a first-class citizen in Spark’s ecosystem.
As the Apache Kudu development team celebrates the initial 1.0 release launched on September 19, and the most recent 1.2.0 version now GA as part of Cloudera’s CDH 5.10 release,
Cloudera Director helps you deploy, scale, and manage Apache Hadoop clusters in the cloud of your choice. Its enterprise-grade features deliver a reliable mechanism for establishing production-ready clusters in the cloud for big-data workloads and applications in a simple, reliable, automated fashion.
Cloudera Director Overview
In this post, you will learn about new functionality in release 2.3, but first, if you’re new to Cloudera Director, let’s revisit what it does.
- On-demand creation and termination of clusters: Using Cloudera Director,
Cloudera is proud to announce that Cloudera Enterprise 5.10 is now generally available (GA). The highlights of this release include the GA of the new columnar storage engine Apache Kudu, improved cloud performance and cost-optimizations, and cloud-native data governance for Amazon S3.
As usual, there are also a number of quality enhancements and bug fixes (learn more about our multi-dimensional hardening/QA process) and other improvements across the stack. Here is a partial list of what’s included (see the Release Notes for a full list):
Apache ZooKeeper is a core infrastructure component in Apache Hadoop stack and is also widely used by many companies for service discovery, configuration management, and so on. Previously ZooKeeper does not support authentication and authorization of servers that are participating in the leader election and quorum forming process; ZooKeeper assumes that every server that is listed in the ZooKeeper configuration file (zoo.cfg) is authenticated. As a result, a server listed in zoo.cfg can join the ensemble even if it is compromised,
You may have heard of the recent (and ongoing) hacks targeting open source database solutions like MongoDB and Apache Hadoop. From what we know, an unknown number of hackers scanned for internet-accessible installations that had been set up using the default, non-secure configuration. Finding the exposure, these hackers then accessed the systems and in some cases deleted the files or held them for ransom.
These attacks were not technologically sophisticated,