About the Algorithm
The first computer program to outplay human professionals at
heads-up no-limit Hold'em poker
In a study completed December 2024 9️⃣ and involving 44,000
hands of poker, DeepStack defeated 11 professional poker players with only one outside
the margin of statistical 9️⃣ significance. Over all games played, DeepStack won 49 big
blinds/100 (always folding would only lose 75 bb/100), over four standard 9️⃣ deviations
from zero, making it the first computer program to beat professional poker players in
heads-up no-limit Texas hold'em poker.
Games 9️⃣ are serious business
Don’t let the name
fool you, “games” of imperfect information provide a general mathematical model that
describes how 9️⃣ decision-makers interact. AI research has a long history of using parlour
games to study these models, but attention has been 9️⃣ focused primarily on perfect
information games, like checkers, chess or go. Poker is the quintessential game of
imperfect information, where 9️⃣ you and your opponent hold information that each other
doesn't have (your cards).
Until now, competitive AI approaches in imperfect
information 9️⃣ games have typically reasoned about the entire game, producing a complete
strategy prior to play. However, to make this approach 9️⃣ feasible in heads-up no-limit
Texas hold’em—a game with vastly more unique situations than there are atoms in the
universe—a simplified 9️⃣ abstraction of the game is often needed.
A fundamentally
different approach
DeepStack is the first theoretically sound application of heuristic
search methods—which 9️⃣ have been famously successful in games like checkers, chess, and
Go—to imperfect information games.
At the heart of DeepStack is continual 9️⃣ re-solving, a
sound local strategy computation that only considers situations as they arise during
play. This lets DeepStack avoid computing 9️⃣ a complete strategy in advance, skirting the
need for explicit abstraction.
During re-solving, DeepStack doesn’t need to reason
about the entire 9️⃣ remainder of the game because it substitutes computation beyond a
certain depth with a fast approximate estimate, DeepStack’s "intuition" – 9️⃣ a gut feeling
of the value of holding any possible private cards in any possible poker
situation.
Finally, DeepStack’s intuition, much 9️⃣ like human intuition, needs to be
trained. We train it with deep learning using examples generated from random poker
situations.
DeepStack 9️⃣ is theoretically sound, produces strategies substantially more
difficult to exploit than abstraction-based techniques and defeats professional poker
players at heads-up 9️⃣ no-limit poker with statistical significance.
