Trust the machine! Enhancing cooperation of AI and human planners
Supply chain planners have the opportunity to change forecasts produced by the self-learning systems using artificial intelligence. Does this add value? And which forecast can be relied on more - the software's or the human's? In a state of the art study, KLU researchers Naghmeh Khosrowabadi, Prof. Kai Hoberg and Prof. Christina Imdahl (Eindhoven University of Technology)are getting to the bottom of these questions. According to their study, human intervention – on average – did not increase the accuracy of the forecasts.
For the study, the team analyzed data of 30 million forecasts for each unit of product, per store on each day from a leading AI provider and major European grocery retailer.
Planners’ optimism leads to more inaccurate forecasts
The results show that planners – on average – do not contribute to the accuracy of the forecast. “Instead, planners even tend to overcompensate effects like weather or a discount that have already been considered by the AI system”, says Khosrowabadi. In the study, only 50 percent of human interventions led to improved results.
A closer look at the data additionally reveals that around five percent of AI-generated forecasts were adjusted by supply chain planners. “We wanted to know why planners decided to adjust the AI-generated forecasts”, explains Naghmeh Khosrowabadi. “Our results show that characteristic of the product like price, freshness or discounts are key drivers for the frequency of planners’ adjustments to AI forecasts.”
If, for example, the AI system is giving a forecast for a very expensive product, planners tend to pay more attention and are more likely to intervene, e.g. by adjusting the forecast. “Besides, our results show that large increases from the AI forecast, for example when the human forecast for items to be sold on a given day in a specific store is two times greater than the AI forecast, are more frequent but also often inaccurate. Too much optimism on the side of the planners seems to be an issue here”, says Prof. Kai Hoberg. Decreases from the AI-forecast, on the other hand, were less likely but more accurate.
Improving the cooperation of human planners and AI
“Human planners will rightful continue to play an important role in AI-enabled forecast processes”, says Prof. Hoberg, “In certain cases human planners have knowledge that is not accessible to an AI system, e.g. local events or competitor actions, that enables them to increase the odds for a better forecast to more than 70%. That is why we need to enhance the cooperation of planners and AI.” For this, the team recommends more exchanges between retailers and AI providers: the better planners understand how the system makes its forecasts, the easier it is for them to decide when to intervene. “Using the results of our study, companies may save money and time”, promises Khosrowabadi, “The key is to help planners to decide when they need to intervene – and when the system is fine on its own and they may concentrate on other tasks.”
The study involved Naghmeh Khosrowabadi as part of her doctoral thesis, Prof. Dr. Kai Hoberg and Prof. Dr. Christina Imdahl from KLU. They analyzed data from 30 million SKU store-day level forecasts from a leading AI provider and a major European retailer. Data on additional variables such as products, weather or holidays were also considered.
Kühne Logistics University – Wissenschaftliche Hochschule für Logistik und Unternehmensführung (KLU) – is a private university located in Hamburg’s HafenCity. The independent, state-certified university’s major research areas are Sustainability, Digital Transformation and Value Creation in the fields of Transport, Global Logistics, and Supply Chain Management.
KLU is one of very few private universities in Germany entitled to confer their own PhDs. The 2021 ranking of the Wirtschaftswoche identifies KLU as one of the strongest research universities in the field of Business Administration in Germany, Austria and Switzerland. Comparing the research output per professor among all participating universities, KLU ranked 4th in Germany and 8th in the DACH region. In the latest CHE university ranking, KLU obtained the highest marks for all major criteria.
With one BSc and three MSc degree programs, a structured doctoral program, and a part-time Executive MBA, KLU offers its 400 full-time students a high level of specialization and excellent learning conditions. KLU has an international team of 27 professors who teach in English. In open, tailor-made management seminar series, industry specialists and managers alike benefit from the application of academic findings to practical issues.
▪ Follow us on LinkedIn and Twitter (@THE_KLU).
▪ KLU research, events & executive education: KLU Business Newsletter (https://www.the-klu.org/landingpages/newsletter/)
▪ More Information: www.the-klu.org
Naghmeh Khosrowabadi: firstname.lastname@example.org
Prof. Dr. Kai Hoberg: email@example.com
Naghmeh Khosrowabadi, Kai Hoberg, Christina Imdahl, Evaluating Human Behaviour in Response to AI Recommendations for Judgemental Forecasting, European Journal of Operational Research (2022), doi: https://doi.org/10.1016/j.ejor.2022.03.017