Solving Path of Exile Item Crafting with Reinforcement Learning

This article discusses using Reinforcement Learning to tackle item crafting complexity in the game Path of Exile (PoE). PoE involves modifying item attributes through crafting currencies to create powerful gear. Crafting involves changing modifiers on items through various actions to achieve desired outcomes. The article explores the challenge of finding the optimal sequence of actions to craft a target item efficiently. Traditional game tree search algorithms like Minimax are not suitable due to the cyclic nature of crafting actions and the large branching factor of possible outcomes. Monte Carlo Tree Search (MCTS) is proposed as an alternative to explore crafting strategies. The problem is formalized as a Markov Decision Process (MDP) in Reinforcement Learning, where a policy is learned to maximize rewards over crafting trajectories. Different reward functions, such as minimizing steps or costs, can guide the crafting process. While the dynamics of the crafting environment are known theoretically, the large distribution of possible outcomes makes direct modeling challenging. Sampling transitions through simulations is suggested as a practical approach to navigate the crafting process in PoE.