Cloudera Flow Management Goes Cloud-Native with Apache NiFi on Red Hat OpenShift Kubernetes Platform

Ahead of Cloudera’s unveiling of the industry’s first enterprise data cloud at Strata Data New York this week, we collaborated with Red Hat to integrate Apache NiFi — which is the key engine behind Cloudera Flow Management — into Red Hat OpenShift, the industry’s most comprehensive enterprise Kubernetes platform. Apache NiFi offers a no-code data ingestion user interface to build sophisticated data pipelines with drag-and-drop ease. The same interface can be extended from design to testing and execution as well. Together, OpenShift and NiFi offer a robust environment for designing and managing high-volume, high-velocity data ingestion pipelines at scale. Cloudera is offering Apache NiFi on OpenShift in a limited tech preview mode to select customers and plans to include the integration in a future Cloudera Flow Management release.

The key business benefits to our joint customers – and how we achieve them – include:

  • Agility and Faster Time to Market 
    • Kubernetes Operators, which are fully integrated with Red Hat OpenShift 4, codify manual operations to automate the provisioning, scaling, and lifecycle management of containerized applications. Using the NiFi Operator for OpenShift, DevOps teams can simplify and expedite the deployment and scaling of secure Cloudera Flow Management applications, relying on the NiFi Operator to eliminate lengthy and error-prone manual operations.
  • Cost Savings through Operational Efficiency
    • IT Operations
      • Without a container platform, IT operations bears a heavy burden of setting up and maintaining the IT infrastructure underneath data platforms. With OpenShift, Red Hat provides a self-managing platform that automates installation, management, and configuration, as well as many of the application-centric needs like deployment, scaling, scheduling, and networking of containerized applications such as Apache NiFi. Thus IT operations is freed from cost-intensive and error-prone manual updates and scaling to provide reduced downtime, faster update cycles, and more efficient software delivery throughout the application lifecycle. 
    • Infrastructure
      • NiFi on OpenShift also offers increased infrastructure efficiency and workload scalability by running NiFi’s core services in lightweight containers on existing OpenShift deployments.
      • OpenShift provides NiFi application autoscaling based on CPU utilization. As a result, you get right-sized NiFi clusters based on real-time application demand.
  • Reliability and Portability 
    • NiFi benefits from OpenShift’s many native features enabling application high availability. Health checks not only keep NiFi application instances going strong, automatically replacing unhealthy or failed instances, but also extend to the OpenShift nodes/hosts. Inside the cluster, load balancing provides even traffic distribution to NiFi instances. And Kubernetes’ persistent storage subsystem ensures the right application data is attached  to the right NiFi container. 
    • NiFi on OpenShift enables running Nifi on a variety of infrastructures, from on-premises deployments to public clouds, and hybrid installations of multiple clusters in multiple places — so the underlying infrastructure doesn’t matter. Write your NiFi applications once and deploy them in a consistent way across hybrid clouds powered by OpenShift. 

 Stop by the Cloudera booth at Strata Data New York to learn more!

Leave a comment

Your email address will not be published. Links are not permitted in comments.