According to Domo, on average, every human created at least 1.7 MB of data per second in 2020. That’s a lot of data. For enterprises the net result is an intricate data management challenge that’s not about to get any less complex anytime soon.
Enterprises need to find a way of getting insights from this vast treasure trove of data into the hands of the people that need it. For relatively low amounts of data, public cloud is a possible path for some organizations. For others, petabyte scale, need for control and efficiency, market regulation and data location prevents them from following suit. These companies have instead opted to leverage their existing data centre investment.
Breaking down the on-prem monolith
On-premises, traditional data and analytics clusters are monolithic deployments of tight coupled compute and storage, unable to cope with current business demands of fast and agile use case deployment with services that are statically provisioned to physical infrastructure. Sized for peak demand yet underutilized the majority of the time, issues like resource contention and upgrade complexity (topics of concern for 40% and 45% of organizations respectively according to a recent survey from Cloudera and Red Hat) impact RoI, and increase risk as well as operational overhead.
“One does not simply deploy a private cloud.”
The solution is clear, but the path to it is less so. Turning the data center into a private cloud would bring all the agility and flexibility of public cloud to the control of an on-premises infrastructure. But to paraphrase a popular Lord of the Rings meme: “one does not simply deploy a private cloud”; it’s a sentiment many organizations share. Though keen to gain the benefits of containerization on-premises, it is coupled with significant trepidation. IT budgets are for the larger part taken up keeping the lights on, squeezing innovation. Yet for private cloud and containerization, exactly that is needed. New skills and experience needs to be built or bought, applications rewritten, and security reconsidered.
Private cloud, no hassle
This is exactly the problem CDP Private Cloud now solves with the Embedded Container Service (ECS). ECS bundles and democratizes the skill and expertise needed to deploy private cloud and delivers it as an integral part of the platform. Organizations can focus on leveraging private cloud for their data and analytics rather than spin their wheels deploying it. The capability directly leverages the existing skills and resources used to manage and deploy CDP Private Cloud Base or previous Cloudera platforms. With fully automated installation and configuration, the time to get private cloud itself up and running, including optimization for Data Services applications, is cut in half. The integration of all private cloud management and monitoring in the familiar CDP management console further simplifies its operation. As Roy Illsley, Chief Analyst at Omdia puts it: “the democratization of skill and experience deploying private cloud solutions fits perfectly with the trends we are seeing in the industry for 2021 and beyond as organizations aim to first capitalize on existing investments in skill and resource (their data center), before they leap into new technologies and approaches that promise much yet first require tremendous investment.”
Move to more Data Services
With the private cloud capability in place, organizations can directly address the drawbacks of the traditional cluster deployments and move to Data Services. Straightforward workload isolation drives predictable performance and elimination of missed SLAs and improved SLOs. Auto-scaling and multi-tenancy more than doubles traditional resource utilization for an improved RoI and also reduces the operational management overhead. Yet by far the biggest benefit in light of the challenges of more data and more users wanting to deploy more use cases for more insight is the simplified onboarding driving faster time to value.
In addition to the Cloudera Data Warehouse (CDW) and Cloudera Machine Learning (CML), CDP Private Cloud now also includes Cloudera Data Engineering (CDE). Purpose built for data engineers, CDE is optimized for modern data engineering with containerized, elastic Spark-as-a-Service workloads and advanced orchestration courtesy of Apache Airflow. Centralized monitoring and pipeline management from a single pane of glass streamlines operation and also delivers visual performance profiling and troubleshooting. Together, the Data Services provide the bulk of analytics organizations need for the majority of use cases, and all tie into CDP’s Shared Data Experience (SDX) for consistently secure and fully governed data access control.
Next stop: hybrid data cloud
CDP Private Cloud lets organizations modernize their existing deployments of data and analytics in their data center. Yet with the vast majority of companies operating in a hybrid cloud landscape that spans private AND public clouds, CDP Private Cloud is but the stepping stone to CDP Hybrid Cloud. With SDX as a critical enabler, organizations can unleash the speed of public cloud and the performance of private cloud for true hybrid use cases. Changing business imperatives like performance, regionality or carbon footprint can be incorporated to continually determine the optimal mix of data, workloads and resources to achieve faster and deeper data-driven insight and value.
Take the first step
CDP Private Cloud Data Services 1.3 is now generally available and includes ECS as well as CDE together with many other improved and new capabilities. For a complete overview of the release, please visit the accompanying release notes. CDP Private Cloud Data Services is available as an add-on for existing CDP Private Cloud Base users; both CDP Private Cloud Base and CDP Private Cloud Data Services are available as a free trial.
What use cases and value will you unlock by turning your data center into a true private cloud?