You did it. You have machine learning capabilities up and running in your organization. Success!
What started as a few nascent experiments (and maybe a few failures) are now carefully constructed models racing along in full production—with the ability to scale into the hundreds or thousands of productional models in sight. Assembling your expert team of data scientists and custodians seems like a distant memory. Now you’re looking ahead to the future—growth, innovation, revenue!
Machine learning and AI are projected to create $2.6 trillion in new business value for marketing and sales alone by 2022. Plus another $2 trillion for manufacturing and supply chain planning. With that much money on the line, you don’t want to risk having the models you’ve worked so hard on to fail you.
When it comes to making machine learning operational for your business – and sustaining it – there are many factors to consider. One of the most important being the human touch.
Remember: models are programmed by humans.
Also remember: humans aren’t perfect.
Your models will require consistent vigilance and maintenance over the course of their lifecycle. Sometimes data progresses out of a range and it confuses the model. Occasionally, a bad actor can sneak in and breach the integrity of an environment. Oftentimes, a simple human error can result in complete disruption.
This is the point where your platform can make or, quite literally, break your efforts. Your production environment should be built on a robust, intuitive architecture that can keep your models secure – and alert you when they are not.
While your policies and procedures will vary based on your organization and industry, these are the baseline factors you should enforce through continuous monitoring and governance policies:
- Maintain visibility into each model’s data lineage
- Control who can access and administer changes to models or environments
- Monitor the technical, mathematical, and business performance of every single model in production
This level of visibility into production environments will help your teams navigate the inevitable errors (human or otherwise) and even empower them to find new solutions or opportunities for future models.
Where human accuracy needs to shine
Mistakes can happen and, with the right platform, quickly be remediated. Where there can be no room for error, however, is in communicating how machine learning initiatives are impacting your business.
At any given point, any given team member must be able to understand and articulate why a model is producing the results that it is. Your data scientists need to explain the outcomes of machine learning models to your business teams, who in turn need to be able to explain the resulting predictions and business decisions to your customers or shareholders. You can (and should) lean on a platform to help with this; prioritizing interpretability capabilities will make automated, visualized reports possible and increase your ability to manage and report results.
As an exercise, see if you can take an outcome determined by a model over the past few months and explain how it came to the decision it did. If not, it might be time for a meeting with your lead data scientist. If so, great! That is the standard that should be set forth for all team members.
In that spirit, reporting to C-level executives is important for demonstrating success and keeping investments into your initiatives a priority. Early on, define what a “win” and “failure” looks like. Remember, this can go beyond the performance of a model and be based on what was learned through the model’s recommendations. Then, establish a cadence for reporting to executives and make sure to utilize analytics and outcomes in them.
It’s all about visibility
Machine learning models only stand to improve from rigorous examination, and clear communication is key to ensure you’re maximizing visibility into them. So keep monitoring and talking as the models keep crunching. Discover more how your team can monitor and maintain your models for optimal performance in Step 9 of our white paper. Access it here: 10 Steps to Making MachineLearning Operational.
Editor's Choice
