In this article we introduce you to our latest study in the field of Machine Learning, authored by Matthias Laube. Here you can find a sneak preview with the most important results.
Management Summary
The goal of these evaluation guidelines is to provide an expert-driven and practice-oriented framework to critically evaluate the business case behind machine learning projects in the financial services industry. For this, I have interviewed machine learning domain experts, of which most have implementation experience in financial services.
The centerpiece of this paper is the Business case assessment forms, which consist of four tables with a total of 22 distinct criteria. The first table is to determine the machine learning category. The second table deals with potential showstoppers & promoters – the big topics which need to be dealt with before diving deeper into the business case assessment. The third table considers costs & benefits, and the fourth table completes the assessment by looking at additional success factors. All of this is condensed on 2 pages. These four tables are then followed by in-depth background for each criterion with all the critical thoughts to be considered and enriched with some real-world examples. To round everything off, the last chapter walks you through an exemplary case.
While I have initially intended to make these guidelines something a person without machine learning background can apply, this endeavor was only partially successful. This is because some evaluation criteria do require a basic understanding of machine learning in order to enable critical thinking. However, this basic understanding is something a consultant should be able to acquire within 1-2 days. Armed with these guidelines, a machine learning beginner should at least be able to critically think through a machine learning business case and, if necessary, challenge it. However, due to the complex nature of the field, hundreds of already existing algorithms and the fast progress being made on developing new solutions and improving old ones, some of the criterions will require a data scientist to assess them in a serious manner. Such a person should of course be available if a machine learning project is seriously considered.
Last but not least, it is also of key importance to include your business stakeholders early on. Understand the desired business outcome and, if necessary, show them cost-benefit tradeoffs to manage expectations and negotiate towards a positive return on investment.
