About the Algorithm
The first computer program to outplay human professionals at
heads-up no-limit Hold'em poker
In a study completed December 2024 and involving 44,000
hands of poker, DeepStack defeated 11 professional poker players with only one outside
the margin of statistical significance. Over all games played, DeepStack won 49 big
blinds/100 (always folding would only lose 75 bb/100), over four standard deviations
from zero, making it the first computer program to beat professional poker players in
heads-up no-limit Texas hold'em poker.
Games are serious business
Don’t let the name
fool you, “games” of imperfect information provide a general mathematical model that
describes how decision-makers interact. AI research has a long history of using parlour
games to study these models, but attention has been focused primarily on perfect
information games, like checkers, chess or go. Poker is the quintessential game of
imperfect information, where you and your opponent hold information that each other
doesn't have (your cards).
Until now, competitive AI approaches in imperfect
information games have typically reasoned about the entire game, producing a complete
strategy prior to play. However, to make this approach feasible in heads-up no-limit
Texas hold’em—a game with vastly more unique situations than there are atoms in the
universe—a simplified abstraction of the game is often needed.
A fundamentally
different approach
DeepStack is the first theoretically sound application of heuristic
search methods—which have been famously successful in games like checkers, chess, and
Go—to imperfect information games.
At the heart of DeepStack is continual re-solving, a
sound local strategy computation that only considers situations as they arise during
play. This lets DeepStack avoid computing a complete strategy in advance, skirting the
need for explicit abstraction.
During re-solving, DeepStack doesn’t need to reason
about the entire remainder of the game because it substitutes computation beyond a
certain depth with a fast approximate estimate, DeepStack’s "intuition" – a gut feeling
of the value of holding any possible private cards in any possible poker
situation.
Finally, DeepStack’s intuition, much like human intuition, needs to be
trained. We train it with deep learning using examples generated from random poker
situations.
DeepStack is theoretically sound, produces strategies substantially more
difficult to exploit than abstraction-based techniques and defeats professional poker
players at heads-up no-limit poker with statistical significance.
