Professor André Calitz, NMU

In-between the hype cycle surrounding generative AI, a news story about ChatGPT passing an MBA exam went viral. In the whitepaper from Wharton School in the University of Pennsylvania, a researcher named Christian Terwiesch used ChatGPT 3 to complete a typical MBA course – in this case, operations management. There was a lot of excitement, but for Professor André Calitz from Nelson Mandela University (NMU), something wasn’t quite right. “As an academic, the headline was misleading. An MBA consists of a number of courses, not one,” he says.

Calitz embarked on a world-first research project to take an entire MBA programme and put it through ChatGPT. He approached various business schools in South Africa to participate in his study, getting lecturers from around the country to send him 2022 final-year exam papers. With four university business schools on board and 12 MBA exams uploaded to the large language model (LLM), ChatGPT took around nine minutes to complete an examination paper and was awarded a total average score of 51%. (However, when you consider the LLM received 4% for Management Accounting, the average goes up to 56%.)