An Application of Reinforcement Learning in Rocket League

Abstract

This paper presents the development of a reinforcement learning (RL) agent for Rocket League, aiming to achieve competitive, human-like gameplay. Building upon existing RL frameworks and Proximal Policy Optimization (PPO), we address limitations of prior agents by implementing a refined reward structure that balances offensive and defensive strategies, discrete action spaces for improved control precision, and enhanced observation processing for better spatial awareness. We utilize RLGym and RLBot frameworks for training and interaction, respectively. Our agent demonstrates superior performance against human players, achieving significant score disparities in controlled matches, showcasing advanced ball control, strategic decisionmaking, and effective execution of ground-based maneuvers. We discuss the agent’s architecture, training methodology, and performance metrics, highlighting its strengths in dribbling, flicking, and kickoffs. Limitations, such as the lack of opponent diversity during training and challenges with advanced aerial maneuvers, are also addressed. Future work focuses on enhancing reward functions, exploring alternative learning architectures, and optimizing environment interaction to further improve the agent’s competitive performance and strategic adaptability