Abstract:

This study compares the performance of Random Forest (RF) and Deep Q-Network (DQN) models in No-Limit Texas Hold’em poker. Using a simulated environment, RF agents trained on labeled data consistently outperformed DQN agents trained via reinforcement learning, achieving higher profits and greater stability. DQN agents suffered from high variance and sensitivity to hyperparameters. Results suggest that supervised models are more reliable in this domain, and a hybrid approach—pretraining with RF and fine-tuning with RL—may offer the best balance of performance and adaptability.