Machine Learning Initiatives Across Industries: Practical Lessons from IT Executives
14.06.2017M-Brain- White papers & e-books
In a recent project led by M-Brain, leading industry IT Executives shared their best practices implementing machine learning (ML) projects at their organizations.
Meet The Authors
Anna Räder
Anna, Senior Consultant at M-Brain Toronto, is responsible
for new business development in California as well as managing and
executing strategic analysis and advisory projects.
Email: anna.rader@m-brain.com
Twitter: https://twitter.com/AniaRader
LinkedIn: linkedin.com/in/annarader/
Google+: https://plus.google.com/+AnnaRaderProfile
Irida Jano
Irida, Consultant at M-Brain Toronto, manages and
executes strategic analysis and advisory projects.
Email: irida.jano@m-brain.com
LinkedIn: linkedin.com/in/irida-jano-1a355b48
Machine learning is a practice which is growing exponentially within North American and Global organizations. Organizations across a number of industries, including Healthcare, Financial Services, Manufacturing, and Retail have been inspired to undertake such initiatives due to various business needs, predominantly improving predictive analytics capabilities and automation. Despite the differences that exist in use cases and business needs across not only different industries, but also between organizations within specific industries, there is one key takeaway shared by all – the implementation of machine learning into organizations has proven to have a direct (positive) impact on earnings and profits.
In addition, machine learning accomplishes thousands of industry and company-specific goals, such as automating services, modeling pricing and services, improving customer service and predictive aspects such as fraud, threats, and even temperature control in manufacturing facilities. IT executives, who are the key stakeholders involved in machine learning initiatives, have shared a number of lessons learned and best practices. The insights gathered in this project show that with a visionary, forward-thinking leadership team and trustworthy, knowledgeable partner, there is no limit to the potential of machine learning in any business or industry setting.
Recent M-Brain research included in-depth interviews with IT management of twenty large companies (1000+ employees) across verticals including financial services, manufacturing, healthcare, retail, and media & gaming (see Exhibit 1). CIOs, CTOs, VPs of IT and IT Directors from the interviewed organizations shared their experiences related to the tactical implementation of machine learning initiatives.
This article outlines a summary of key findings from this research, including key lessons learned, examples of use cases, business needs that have lead to the initiation of machine learning projects, involvement of different stakeholder groups, insights on outsourcing preferences and external vendors of technology and human work, key steps of typical machine learning projects, as well as the evaluation of key benefits.
INTERVIEWED RESPONDENTS
Exhibit 1. Interviewed Respondents, by Vertical (n = 20)
Lessons Learned
CIOs and IT decision makers shared an extensive list of best practices that they would recommend to the organizations planning to implement machine learning projects.
Planning:
• Define the need and objectives, put a framework into place and create a solid plan; Define ROI being sought; Go into the process with an open mind
• Have a forward-thinking mentality and build a framework that is effective to manage in the present, but is also agile and resilient to future modification
• Do research up front; Figure out key requirements and let those drive the project
Stakeholders & Teams:
• Have a visionary leadership team involved to support the process, as it takes time to realize value and is expensive
• Avoid thinking it is a one-man job; Ensure that the ML initiative is collaborative
• Consider the opinions of multiple stakeholders
• Keep the team involved and updated throughout the process
Vendors:
Evaluate ML vendors extensively and choose the proper tools to implement; Top requirements include:
• Reputable and trustworthy (50% of respondents who discussed the topic of vendors)
• Has client’s industry expertise (38%)
• Strong technology, including a proven business case on how vendor implements ML, IoT, etc. (12%)
Data:
• Monitor and validate data and ensure it is error free
• Clean up the data beforehand to save time during project
Other:
• Consider compliance and regulations, especially in certain industries like healthcare
• Do not be intimidated, e.g. by thinking technology is too complicated or difficult to learn, or afraid to ask questions
• Break process tasks down into smaller projects
Use Cases
There is a wide variety of application areas for machine learning technology. During interviews, use cases varied not only between the analyzed verticals, but also within each vertical (as illustrated by Exhibit 2).
Show more
show less
Exhibit 2. Machine Learning Use Cases, by Vertical
Healthcare
• Insurance membership plans
• Data models for HIV, cancer, oncology, asthma, congestive heart failure (CHF)
• Wearable devices
• Pulmonary, lung cancer research
• Predicting and understanding customer behavior
Financial Services
• Evaluating new lines of business (win/loss analysis)
• Insurance modeling
• Pricing strategies
• Facilities management
• Customer service analytics
• Fraud analytics
• Threat analysis
• Automatic account approvals
Manufacturing
• Humidity and climate control
• CRM system
• Invoice scanning
• RFID tags
• Predict and monitor competitive threats
• Monitor materials to avoid shortfalls
• Automation, smart devices
• Improve IT operations
Retail
• Credit risk assessment
• Predicting product demand to improve supply chain
• Materials replenishment
• Fraud detection
• Customer behavior
• Pricing, costing
Media/Gaming
• Make company more agile, especially from customer service perspective
• Content delivery (recommendation engine)
• Forecasting (e-commerce)
• Identify trends and issues in customer service
Business Needs
Identified business needs that inspired machine learning projects varied significantly between the analyzed companies (see Exhibit 3). However, some common themes were discovered as well.
Show more
show less
BUSINESS NEEDS
Exhibit 3. Top Business Needs That Lead to Implementation of Machine Learning Projects, by Vertical
Healthcare
• Grow patient care
• Introduce predictive modeling
• Automate processes to free up resources and improve staffing
• Improve/streamline processes and data utilization
• Better understand customer behavior to help drive strategic planning
Financial Services
• Improve/streamline predictive analytics
• Reduce reliance on human intervention and manual review
• Provide improved service to customers to gain retention and wallet share
• Reduce support costs
Manufacturing
• Improve facilities management (humidity and climate control)
• Improve/reduce manual processes through automated capabilities as part of a natural industry evolution
• Monitor market and consumer trends
• Improve service level
• Organize data
Retail
• Develop solutions for credit risk assessment
• Improve supply chain processes (i.e. demand and planning)
• Generate more sales
• Become more technologically innovative (i.e. through automation)
Media/Gaming
• Gain competitive advantage
• Eliminate manual efforts
• Collect and streamline data utilization
• Improve recommendation engine
Exhibit 4. Summary of the Top Business Needs Across Verticals
Stakeholders
The IT department is the most involved in machine learning projects, based on the insights received from the respondents.
Show more
show less
STAKEHOLDERS
Exhibit 5. Key Stakeholders Involved in Machine Learning Projects (n = 20)
Note 1: 15% of the respondents did not provide an answer to this question
Note 2: Department Heads refers to leaders within key departments involved in ML initiatives; This differs by industry – examples include Strategy, Cardiology, Wealth Manager, Design Director, etc.
Note 3: Respondents could list several stakeholder groups
Outsourcing & Vendors
65% of respondents outsourced at least part of human work in machine learning projects to external vendors, as shown in Exhibit 6.
Show more
Companies that kept all work in-house argue that this decision helped them to keep all the knowledge gained during the project in-house.
As illustrated in Exhibit 8, financial services and manufacturing companies were more likely to keep the human work in machine learning projects in-house, compared to other verticals. All retail industry respondents used external vendors for more that 50% of the work.
With regards to the software and tools used for machine learning, 100% of respondents across all analyzed verticals said that they were using external software and tools in their machine learning projects. Microsoft (60%) and IBM (50%) were the most frequently selected vendors of machine learning software, followed by Google (30%). The specific names of the software used are listed in Exhibit 9.
60% of the interviewed companies relied on in-house data scientists for machine learning projects, 25% decided to outsource all data science work, and 15% used a hybrid model. Manufacturing was the only industry vertical that relied completely on their internal in-house data scientists (see exhibits 10 and 11). Companies that decided to keep the data science work in-house justified their decision by the fact that they already had sufficient in-house data scientist resources. Some of them may still decide to outsource for short-turnaround projects where more resources are required. When necessary, employees underwent special training to learn the required skills.
show less
VENDORS
Exhibit 6. Share of Respondents Outsourcing Human Work to External Vendors (n = 20)
Note 1: Software is not included in this analysis; see next page for an overview of external tools used.
Note 2: Interpretation of share of work outsourced is based on analysis of qualitative answers
Exhibit 7. Vendors Used for Outsourced Human Work and Consulting (n = 13)
Note: Respondents could list several vendors
Exhibit 8. Share of Respondents Outsourcing Human Work to External Vendors, by Vertical (n = 20)
SOFTWARE/TOOLS
Exhibit 9. Software / Tools Used in Machine Learning Projects (n = 20)
Note: Respondents could list several vendors / tools
DATA SCIENTISTS
Exhibit 10. Presence of Data Scientists in Machine Learning Projects (n = 20)
Exhibit 11. Presence of Data Scientists in Machine Learning Projects, by Vertical (n = 20)
Process
Key steps in a typical machine learning project included the identification of the need and key stakeholders, building a business case around expected benefits, planning the project and clarification of key objectives, evaluation or vendor offerings, ending with approval and implementation.
Show more
Respondents felt that the most difficult step in machine learning projects was data collection and processing (25%), as well as finding the right vendor (20%), as shown in Exhibits 15 and 16. Healthcare and financial services companies were also concerned with compliance and regulatory requirements.
A quarter (25%) of the interviewed companies said that seeing the results and the final launch was the most successful and rewarding step in their project. Other successful steps mentioned included the initial project phases, data collection, and overall team collaboration during the project (see Exhibits 18 and 19).
The majority of interviewed respondents (55%) stored the data for the discussed projects in the public cloud. Several respondents already had data in the cloud before the project started. 20% chose to store their data on-premises, with majority of these companies being in financial services sector, as illustrated in Exhibits 19 and 20.
show less
MACHINE LEARNING PROJECT PROCESS
Exhibit 12. Typical Steps of a Machine Learning Project
Note 1: The key steps are not necessarily identical in each company; this is a summary of identified similarities
Note 2: *Refers to vendors of software or human work
Exhibit 13. Implementation of ML Projects: From Needs Identification to Finalizing the Implementation (n = 20)
Note 1: 5% of the respondents did not provide an answer to this question
Note 2: Figures are based of the higher end of the provided ranges
Exhibit 14. Time Required for Pre-Work (n = 20)
Note 1: 35% of the respondents did not provide an answer to this question
Note 2: Figures are based of the higher end of the provided ranges
Exhibit 15. Most Challenging Steps in Machine Learning Projects, by Vertical
Healthcare
Vendor selection:
• Vendor lacking industry expertise in the medical field
• Selecting the most reputable provider for healthcare industry
Compliance:
• Meeting compliance requirements, incl. cloud security
Balancing efforts:
• Balancing between data mining efforts and actionable results
Financial Services
Data:
• Data cleaning
• Selecting tools for data collection and storage
• Data collection and model creation
Regulatory:
• Challenges with meeting legal requirements at the project launch
Framework:
• Creating a global framework that works across the organization
Manufacturing
Vendors:
• Getting vendors to be completely transparent, incl. the downsides
Data:
• Data collection and cleaning
Tools:
• Selecting the right toolsets
Priorities:
• Prioritizing the work
• Getting alignment from stakeholders
Retail
Vendor selection:
• Selecting the best vendor / solution
Technical skills:
• Debugging the code
• Having the technical expertise needed
Internal buy-in:
• Getting buy-in from stakeholders, selling to management
Media/Gaming
Understanding ML capabilities:
• Refining the need, understanding what the systems can do
Data:
• Data sourcing, aggregation, cleaning, building out the model
Exhibit 16. Summary of the Most Challenging Steps in Machine Learning Projects
Note: Unprompted; based on analysis on qualitative responses
Exhibit 17. Most Successful Steps in Machine Learning Projects, by Vertical
Healthcare
Results / launch:
• Seeing the results
• Delivering the product to end-users
• Showcasing the successful implementation to other departments
Collaboration:
• Cross-functional collaboration during the project
Financial Services
Pilot phase:
• Deploying a demo / pilot before full implementation
Data:
• Data collection
Global practices:
• Building a global standardized set of practices
Manufacturing
Mechanical steps:
• Connecting the APIs
Data:
• Collecting data
Tools:
• Finding simple tools that are reasonably priced
Collaboration:
• Voting on best solution / vendor as a cross-functional team
• Developing the internal process, bringing together a cohesive team
Retail
Early steps:
• Completing work before the debugging phase
Results / launch:
• Launching, seeing the solution work in reality
Media/Gaming
Data:
• Collecting data
Testing:
• Testing and validating of a viable solution
Exhibit 18. Summary of the Most Successful Steps in Machine Learning Projects (n = 20)
Note 1: 10% of the respondents did not provide an answer to this question
Note 2: Unprompted; based on analysis on qualitative responses
DATA STORAGE
Exhibit 19. Data Storage for Discussed Machine Learning Projects (n = 20)
Exhibit 20. Data Storage for Discussed Machine Learning Projects, by Vertical (n = 20)
Evaluation
All interviewed respondents said they have found their machine learning project initiatives successful.
Show more
As shown in Exhibit 23, 75% of respondents consider data science, analytics and machine learning high priorities in their organizations. This is due to the following factors:
• Data is needed to justify costs and operations, and often drives pricing and decision making at companies
• Data science, analytics, and machine learning enable company evolution, modernization, and digital transformation
• Business processes become more efficient (e.g. reducing margins of error and doubts, seeing trends through predictive analytics and understanding
customers better, etc.)
• Freeing up resources, resulting in saved money
15% of respondents consider data science, analytics, and machine learning low priority due to several reasons, including:
• The nature of the industry (e.g. mortgage servicing focuses on minimizing costs, and budget holders may not see value in such initiatives, especially
if current processes are working well)
• Benefits are not easily and quickly seen
• Key stakeholders may be narrow-minded and lack imagination
show less
BENEFITS
Exhibit 21. Key Benefits of the Implemented Machine Learning Projects, by Vertical
Healthcare
Automation:
• Automation of: Infrastructure, monitoring of medical equipment, monitoring of patients
Improved service:
• Improving the quality of care for patients
• Improving medical treatments
Analytics:
• Improving the quality of the analytics
Financial Services
Time savings:
• Time required to analyze the data reduced dramatically
Cost savings:
• Standardized business model
Automation:
• Automation of data collection and analytics
Risk management:
• Avoiding potential incidents
Manufacturing
Time savings:
• Faster data collection
• Faster response time
Cost savings:
• In procurement, when purchasing energy
Increased revenues:
• New stream of revenue generation
Data utilization:
• More actionable utilization of data
Risk management:
• Stopped certain issues from recurring
Retail
Increased revenues:
• Increased sales at higher prices; increased upselling
• More customers
Inventory:
• Better consumer trends insights that help to predict the inventory needs
Risk management:
• Increased accuracy of credit risk assessment
Media/Gaming
Analytics:
• More focused analysis
• More in-depth analysis of customer behavior
Exhibit 22. Summary of Key Benefits of the Implemented Machine Learning Projects, by Vertical (n = 20)
Note: Respondents could list multiple benefits
Exhibit 23. Priority of Data Science, Analytics and Machine Learning at Respondent Companies (n = 20)
Are you looking for answers to specific research questions that will help you become more competitive?
Call us at M-Brain. We can help.
