The world of artificial intelligence (AI) benchmark tests like Chatbot Arena are crucial for comparing the performance of different AI models. However, a recent study has cast doubt on the integrity of one of the most popular platforms, Chatbot Arena, operated by LM Arena. The study suggests that LM Arena may have provided preferential treatment to certain top-tier AI labs, potentially allowing them to manipulate rankings on the leaderboard. The allegations against LM Arena stem from a new research paper published by researchers at Cohere, Stanford University, MIT, and Ai2, which alleges that LM Arena’s practices favor select industry giants. These practices are being referred to as ‘gamification’ of the benchmark process.