About the Algorithm
The first computer program to outplay human professionals at
heads-up no-limit Hold'em poker
In a study completed December 2024 👏 and involving 44,000
hands of poker, DeepStack defeated 11 professional poker players with only one outside
the margin of statistical 👏 significance. Over all games played, DeepStack won 49 big
blinds/100 (always folding would only lose 75 bb/100), over four standard 👏 deviations
from zero, making it the first computer program to beat professional poker players in
heads-up no-limit Texas hold'em poker.
Games 👏 are serious business
Don’t let the name
fool you, “games” of imperfect information provide a general mathematical model that
describes how 👏 decision-makers interact. AI research has a long history of using parlour
games to study these models, but attention has been 👏 focused primarily on perfect
information games, like checkers, chess or go. Poker is the quintessential game of
imperfect information, where 👏 you and your opponent hold information that each other
doesn't have (your cards).
Until now, competitive AI approaches in imperfect
information 👏 games have typically reasoned about the entire game, producing a complete
strategy prior to play. However, to make this approach 👏 feasible in heads-up no-limit
Texas hold’em—a game with vastly more unique situations than there are atoms in the
universe—a simplified 👏 abstraction of the game is often needed.
A fundamentally
different approach
DeepStack is the first theoretically sound application of heuristic
search methods—which 👏 have been famously successful in games like checkers, chess, and
Go—to imperfect information games.
At the heart of DeepStack is continual 👏 re-solving, a
sound local strategy computation that only considers situations as they arise during
play. This lets DeepStack avoid computing 👏 a complete strategy in advance, skirting the
need for explicit abstraction.
During re-solving, DeepStack doesn’t need to reason
about the entire 👏 remainder of the game because it substitutes computation beyond a
certain depth with a fast approximate estimate, DeepStack’s "intuition" – 👏 a gut feeling
of the value of holding any possible private cards in any possible poker
situation.
Finally, DeepStack’s intuition, much 👏 like human intuition, needs to be
trained. We train it with deep learning using examples generated from random poker
situations.
DeepStack 👏 is theoretically sound, produces strategies substantially more
difficult to exploit than abstraction-based techniques and defeats professional poker
players at heads-up 👏 no-limit poker with statistical significance.
