An Artificial Intelligence system has outperformed numerous forecasting enthusiasts, including several professionals, in a competition aimed at predicting events such as conflicts between Donald Trump and Elon Musk, as well as Kemi Badenoch’s removal from the Conservative party leadership.
A British AI startup, co-founded by a former researcher from Google DeepMind, has secured a position in the top 10 of an international forecasting contest that challenges participants to predict the likelihood of 60 events throughout the summer.
ManticAI achieved eighth place in the Metaculus Cup, organized by a forecasting firm based in San Francisco that aims to forecast future events for investment funds and corporations.
While AI’s performance still falls short of the top human forecasters, it has led some to believe that AI could surpass human capabilities sooner than anticipated.
“It’s certainly a strange feeling to be surpassed by several bots at this stage,” remarked Ben Shindel, one of the professional forecasters who found himself trailing behind AI during the competition before ultimately finishing ahead of Mantic. “We’ve made significant progress compared to a year ago when the best bot was ranked around 300.”
Questions posed in the Metaculus Cup included which party would secure the most seats in the Samoan general election and the number of acres in the US that would be consumed by fires from January to August. Contestants were evaluated based on their predictions as of September 1.
“What Mantic has accomplished is remarkable,” stated Deger Turan, the CEO of Metaculus.
Turan projected that AI would match or exceed the performance of the best human forecasters by 2029, but noted that “currently, human forecasters are outperforming AI forecasters.”
In complex forecasts that depend on predicting interconnected events, AI systems can still face challenges in performing logic verification checks when converting knowledge into a final prediction, he explained.
Mantic dissects a forecasting challenge into various tasks and allocates them to a roster.
AI Startup Wins Worldwide Forecasting Contest Against Humans
