People intuitively know that self-driving or autonomous cars present complex engineering challenges. Vehicle assembly is the easy part – we’ve been doing that for 100 years. The real challenge is a data challenge, acquiring and managing the data needed to run the vehicles’ brain, eyes, and ears. Autonomous driving technology complexity lies in the ability to ingest, store, analyze, and deploy large volumes of data & the high bandwidth needs of data-in-motion. The current state of digital solutions offers redundant, fragmented systems across the autonomous driving data loop. Automotive manufacturers demand new approaches in computer science and deep learning, along with key technologies that integrate all steps of the autonomous driving development lifecycle.
The challenge of creating the perception layer enabling the car to operate autonomously can’t be understated. A white paper can only scratch the surface of its true complexity.
When you start to understand the entire autonomous driving data cycle, you see the challenges:
- Excessive costs incurred by deploying many different hardware environments, and duplicate software costs due to individual work process environments
- Long development times when building the perception layer because of excessive time spent identifying relevant sensor data sets in multiple silos due to the absence of search capabilities
- Operational challenges arising from multiple frameworks and interfaces result in fragmented user experience, multi-/hybrid-cloud setups explode efforts required for results and the absence of a holistic management suite available to automate operations
- High-risk factors are driven by proprietary solutions, absent end-to-end governance and security, and maybe most importantly, high operational risk and costs associated with multi-vendor integration
The road to a fully autonomous vehicle needs to start with an understanding of how to leverage data. Cloudera can partner with you, steering you on your way through the entire journey from data collection through the autonomous driving data cycle and completing it with the deployment of the perception layer. Cloudera does this through the interrelated capabilities of its open-source solutions. Cloudera Data Platform is comprised of:
- Cloudera Data Flow: Cloudera Data Flow (CDF) provides the abilities to collect, aggregate, compress and encrypt connected vehicle data, prioritize transmission of data from the vehicle to the Cloud or Data Center, buffer data in the event of network interruptions and track the provenance and lineage of streaming data, providing confidence in the origin and usage of data
- Cloudera Search: Cloudera Search provides easy, natural language access to data stored in or ingested into Hadoop, HBase, or cloud storage. For Autonomous Drive engineers and analysts this means google-like discovery and analysis (via user interface or search API) providing the ability to feed applications and machine learning models with the specific driving episodes required, rather than flooding them with superfluous and repetitive data.
- Cloudera Machine Learning: A collaborative, customizable Continuous Integration, Continuous Deployment (CI/CD) environment for machine learning engineers, featuring easy and secure access to all datasets and processing resources within the organization.
- Cloudera Data Engineering: Cloudera Data Engineering is a powerful and cost-effective platform for processing large-scale data sets on-premise or in the Cloud. Within the autonomous drive data lifecycle, Docker containers on Kubernetes are leveraged for mass inference within the perception layer, while Spark on Kubernetes (with orchestration via Apache Airflow) is leveraged for pre-processing, post-processing and mass verification of the perception layer.
Cloudera delivers an enterprise data cloud for any data, anywhere, from the Edge to AI. Powered by the relentless innovation of the open-source community, Cloudera advances digital transformation for the world’s largest enterprises.
If you would like to learn more, join our webinar on June 2, 2020, that features two of our leading subject matter experts in the data lifecycle. Also, check out the white paper that presents key technologies that Cloudera contributes to streamlining all steps of the autonomous driving development lifecycle into a scalable AI-enabled Enterprise Data Cloud.
Editor's Choice
