{"id":47400,"date":"2025-04-14T13:38:04","date_gmt":"2025-04-14T17:38:04","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=47400"},"modified":"2025-07-24T11:50:26","modified_gmt":"2025-07-24T15:50:26","slug":"llama-gaming-ai-benchmarks","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2025\/04\/14\/llama-gaming-ai-benchmarks\/","title":{"rendered":"LLaMA Gaming AI Benchmarks"},"content":{"rendered":"

Kylie Robison<\/a> (via Hacker News<\/a>, Slashdot<\/a>):<\/p>\n

Over the weekend, Meta dropped two new Llama 4 models<\/a>: a smaller model named Scout, and Maverick, a mid-size model that the company claims can beat GPT-4o and Gemini 2.0 Flash “across a broad range of widely reported benchmarks.”<\/p>

[…]<\/p>

The achievement seemed to position Meta’s open-weight Llama 4 as a serious challenger to the state-of-the-art, closed models from OpenAI, Anthropic, and Google. Then, AI researchers digging through Meta’s documentation discovered something unusual.<\/p>

In fine print, Meta acknowledges that the version of Maverick tested on LMArena isn’t the same as what’s available to the public. According to Meta’s own materials, it deployed an “experimental chat version”<\/a> of Maverick to LMArena that was specifically “optimized for conversationality,” TechCrunch<\/em> first reported<\/a>.<\/p><\/blockquote>\n

First Google<\/a> demo shenanigans, then Apple<\/a>, and now Meta.<\/p>\n\n

Previously:<\/p>\n