What did we learn from the LLM poker challenge?

Liv Boeree, Doug Polk, and Igor Kurganov come together for the final analysis of the recently completed LLM poker challenge, trying to answer the question if these chat bots are any good at poker and if there is any rhyme or reason to their decision-making process.

While it was a fun challenge to watch, I feel like any type of serious analysis falls short here if we know that LLMs struggle to read board textures and their hole cards. Can we really judge the quality of one’s poker strategy if their entire reasoning for a particular line is based on a belief that they only need four cards of the same suit to make a flush?

But make no mistake – LLMs are not representative of Artificial Intelligence, and AI bots developed specifically to play poker are way stronger. Take this analysis with a big grain of salt!