Cloudera Developer Blog · Use Case Posts
Our thanks to Telvis Calhoun, Zach Hanif, and Jason Trost of Endgame for the guest post below about their BinaryPig application for large-scale malware analysis on Apache Hadoop. Endgame uses data science to bring clarity to the digital domain, allowing its federal and commercial partners to sense, discover, and act in real time.
In my previous post you learned how to index email messages in batch mode, and in near real time, using Apache Flume with MorphlineSolrSink. In this post, you will learn how to index emails using Cloudera Search with Apache HBase and Lily HBase Indexer, maintained by NGDATA and Cloudera. (If you have not read the previous post, I recommend you do so for background before reading on.)
Which near-real-time method to choose, HBase Indexer or Flume MorphlineSolrSink, will depend entirely on your use case, but below are some things to consider when making that decision:
Customer Spotlight: Learn How Edo Closes the Advertising Loop with Hadoop at Cloudera Sessions Milwaukee
The Cloudera Sessions fall series is coming to a close next week, but first we’ll make a final stop in Milwaukee, Wisconsin (on Oct. 17), where attendees will hear about edo — a company that is revolutionizing the advertising space by closing the loop between promotions and point-of-sale transactions.
In Milwaukee, edo CTO Jeff Sippel will engage in a fireside chat with Cloudera’s VP of marketing, Alan Saldich. At edo, Jeff is responsible for the strategy, planning, and execution for the systems — including Apache Hadoop — that power the edo offer platforms.
It’s common to hear people describe themselves as being “left-brained” or “right-brained” based on their tendency to be more logical and mathematically driven (left-brained), or, conversely, to be intuitive and creatively driven (right-brained). For example, people who prefer math over art are often considered left-brained. People who get a higher verbal score on their SATs than for math are often considered right-brained.
In general, language and creative writing are considered right-brained exercises. Many people also associate marketing and advertising as a right-brained function, whereas engineering is considered very left-brained.
In December 2012, we described how an internal application built on CDH called Cloudera Support Interface (CSI), which drastically improves Cloudera’s ability to optimally support our customers, is a unique and instructive use case for Apache Hadoop. In this post, we’ll follow up by describing two new differentiating CSI capabilities that have made Cloudera Support yet more responsive for customers:
Why would any company be interested in searching through its vast trove of email? A better question is: Why wouldn’t everybody be interested?
Email has become the most widespread method of communication we have, so there is much value to be extracted by making all emails searchable and readily available for further analysis. Some common use cases that involve email analysis are fraud detection, customer sentiment and churn, lawsuit prevention, and that’s just the tip of the iceberg. Each and every company can extract tremendous value based on its own business needs.
This week’s Cloudera Sessions roadshow will make it to Denver, Colo., on Thursday, where the customer Fireside Chat will feature Intelligent Software Solutions (ISS) Chief Architect of Global Enterprise Solutions, Wes Caldwell. ISS helps many government organizations – including several within the U.S. Department of Defense — deploy next-generation data management and analytic solutions using a combination of systems integration expertise and custom-built software.
During the Fireside Chat, Cloudera’s COO Kirk Dunn will engage Wes in a conversation to discuss the business use cases for Hadoop that ISS sees most often in the field, primarily within two buckets: batch analytics and real-time applications. Wes will also share his thoughts on some of the more recent innovations within the Apache Hadoop ecosystem, such as Cloudera Impala and Solr integrations.
In its first leg of its tour of the United States earlier this year (see photos here), The Cloudera Sessions proved to be an invaluable single-day event for business and technical leaders exploring practical applications of Apache Hadoop. So valuable, in fact, that we’ve extended the tour with dates/cities this September and October.
We’re kicking off the second leg of our Cloudera Sessions roadshow this week, starting in San Francisco on Wednesday and Philadelphia on Friday. The spring series of the Cloudera Sessions was a big hit, which is why we’re back with a new and improved agenda for the fall, to offer even more options that will help attendees — ranging from developers to line-of-business managers and executives — navigate the Big Data journey. The expanded fall series agenda includes an application development lab (based on CDK) that coincides with the general session throughout the morning, and two tracks for clinics after lunch.
One portion of the general session that was a big hit throughout the spring series and that will return this fall is the Fireside Chat, during which the Cloudera executive host sits with one or two customers to talk about their “real life” experiences and lessons learned with Apache Hadoop. The Fireside Chat gives local customers an opportunity to showcase the work they’re doing, and allows attendees to hear from real users what worked, what didn’t, how they got started with Hadoop, and best practices learned along the way.
One of the first questions Cloudera customers raise when getting started with Apache Hadoop is how to select appropriate hardware for their new Hadoop clusters.
Although Hadoop is designed to run on industry-standard hardware, recommending an ideal cluster configuration is not as easy as delivering a list of hardware specifications. Selecting hardware that provides the best balance of performance and economy for a given workload requires testing and validation. (For example, users with IO-intensive workloads will invest in more spindles per core.)