This is a guest blog by Joshua Thorp, Searcher Seismic, VP of Data & Analytics
There are no silver bullets to solve your data issues in this Big Data world.
This truism became repeatedly apparent in Searcher Seismic’s journey towards better data management. We realized that every choice has compromises associated with it, and a robust solution will necessarily integrate many individual pieces of technology and business processes.
Having said that, there are some solutions that are undeniably better than others. The key is having a very clear understanding of your functional requirements upfront so you pick the vendors and contractors best suited to your needs.
Accessing and exploring our data more efficiently
Like many industries, oil and gas is adopting machine learning and digitalization initiatives. Successful implementation depends on high quality, well-organized data, held on a data platform that supports these new technologies.
Searcher holds a wealth of global seismic data that our clients use to make mission-critical decisions when planning new oil and gas exploration sites.
Traditionally, the process of compiling data sets for clients was labor-intensive and slow, involving a team of three or four Searcher staff pulling huge amounts of information from a range of sources. A typical timeframe was several weeks before the information could be delivered via post.
With our new system, it now takes one person at Searcher only three or four clicks of a mouse to provide our clients with direct access to the data they need, which they can either view via the Saismic™ portal or transfer to their own analytics systems.
Flexibility to grow, no compromise on performance
Before we embarked on our data management project, we did an extensive amount of research to provide clarity around our requirements and existing solutions.
We looked at how data management was being done both in the oil and gas industry and beyond. From there we determined that we needed a data platform that utilized open-source software, particularly open-source distributed computing software such as Hadoop and Spark.
We found that Cloudera (then Hortonworks) had the only fully open-source distribution of the Hadoop ecosystem, now known as the Cloudera Data Platform (CDP). This was incredibly attractive as it supported technologies, such as Apache Spark, that were already being used by many industries which enabled us to fast track our success without having to build everything from scratch. Their distribution also allowed for flexibility in deployment, where we could deploy in a public cloud, private cloud, or a hybrid cloud solution.
Another key vendor, Pure Storage, provided the storage component for our data platform. We looked to leverage the capabilities of separated compute and storage of the Cloudera platform to integrate the incredibly high performing Pure Storage Flashblade with the robust data tools of the Cloudera toolkit for the best of both worlds.
Our software teams worked with the Pure Storage sales engineers and Cloudera Professional Services team to build an integrated hybrid cloud solution that ultimately allowed us to have a data platform that can grow in a very flexible manner with no compromise on performance. Our TCO analysis on this deployment model showed that it was between two to ten times less than a pure public cloud deployment while allowing for higher overall performance.
Integrated teams providing tailored solutions
We found there was significant value in having access to the Professional Services team at Cloudera. This allowed us to have access to world-class data engineers who could work with our domain experts to build a data platform that was best suited for how our business operated and what we wanted to achieve. In this way, we could offer our company early successes while transitioning the knowledge base of the platform to our internal development as we ramped up our efforts.
Working with Cloudera we found that they were eager to understand our business and data, and then create tailored solutions based on our requirements and budget. Furthermore, they worked inclusively with our staff, helping to identify risks and resolve issues as they arose in the project. This helped us focus on the project outcome and build the best data platform possible.
Challenging but rewarding
This has been the biggest technology project that Searcher has ever undertaken, so there were plenty of learnings as we progressed but we are delighted with the outcome and excited about the ongoing development of our data platform. With the support of the Cloudera Professional Services team, we’re building key technology differentiators such as the ability to apply large scale deep learning training and predictions within our data platform and for our clients.
Software development isn’t easy if you haven’t done it before, so doing your homework is essential. It’s also essential to choose the right project vendors and partners. If your vendors don’t understand your problems and can’t help you achieve your goals, it can break a project even if they have great products or technology.
Learn more about Searcher Seismic’s story
Editor's Choice
