About the Algorithm
The first computer program to outplay human professionals at
heads-up no-limit Hold'em poker
In a study completed December 2024 💋 and involving 44,000
hands of poker, DeepStack defeated 11 professional poker players with only one outside
the margin of statistical 💋 significance. Over all games played, DeepStack won 49 big
blinds/100 (always folding would only lose 75 bb/100), over four standard 💋 deviations
from zero, making it the first computer program to beat professional poker players in
heads-up no-limit Texas hold'em poker.
Games 💋 are serious business
Don’t let the name
fool you, “games” of imperfect information provide a general mathematical model that
describes how 💋 decision-makers interact. AI research has a long history of using parlour
games to study these models, but attention has been 💋 focused primarily on perfect
information games, like checkers, chess or go. Poker is the quintessential game of
imperfect information, where 💋 you and your opponent hold information that each other
doesn't have (your cards).
Until now, competitive AI approaches in imperfect
information 💋 games have typically reasoned about the entire game, producing a complete
strategy prior to play. However, to make this approach 💋 feasible in heads-up no-limit
Texas hold’em—a game with vastly more unique situations than there are atoms in the
universe—a simplified 💋 abstraction of the game is often needed.
A fundamentally
different approach
DeepStack is the first theoretically sound application of heuristic
search methods—which 💋 have been famously successful in games like checkers, chess, and
Go—to imperfect information games.
At the heart of DeepStack is continual 💋 re-solving, a
sound local strategy computation that only considers situations as they arise during
play. This lets DeepStack avoid computing 💋 a complete strategy in advance, skirting the
need for explicit abstraction.
During re-solving, DeepStack doesn’t need to reason
about the entire 💋 remainder of the game because it substitutes computation beyond a
certain depth with a fast approximate estimate, DeepStack’s "intuition" – 💋 a gut feeling
of the value of holding any possible private cards in any possible poker
situation.
Finally, DeepStack’s intuition, much 💋 like human intuition, needs to be
trained. We train it with deep learning using examples generated from random poker
situations.
DeepStack 💋 is theoretically sound, produces strategies substantially more
difficult to exploit than abstraction-based techniques and defeats professional poker
players at heads-up 💋 no-limit poker with statistical significance.
