About The Project

This project implements a Reinforcement Learning (RL) agent designed to play the original Diablo game.
The primary goal is to train an AI to navigate the early dungeon levels, interact with the environment, engage in combat, and ultimately defeat the Butcher.

Prerequisites

1. Conda: Miniconda or Anaconda installed.
   • Note: It is recommended to stick with conda terminal rather than windows CMD/PowerShell
2. Modified DevilutionX:
   • Clone the required fork: git clone https://github.com/lciesielski/devilutionX.git
   • Follow the build instructions in that repository's README to compile and configure it.
   • Note: This specific fork is required because it contains necessary API for the agent.

Setup Instructions

1. Setup environment and dependencies:

• Open up conda terminal
• Run conda create -n deepdungeon python=3.10.11 -y
• Run conda activate deepdungeon
• Run cd to swtich to repository root folder eg. cd f:\Repos\Internal\DeepDungeon\
• Run pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu118
• Run pip install -r requirements-stable.txt

2. Setup DevilutionX paths:

• Run: generate_properties.bat
• Provide the full path to your DevilutionX build root directory (the one with devilutionx.exe file).
• Provide the number of clients to run, the more powerful PC, the more clients you can run.

How to Run Training

• Run following command python src\DiabloRunner.py "X:\path\to\repository\root\src\Agent.py" from repository root
• Note: Replace second argument with your actual path to Agent.py file

How to Run Evaluation

• Run following command python src\Evaluate.py 2 from repository root
• Note: To run evaluation you need to have training data inside repository root. You can download it from Release 1.0.0

FAQ

Can I increase client count in d1Runner.properties?

Yes. You must ensure that there are save folders with save files, by deafult initial_setup.bat file in DevilutionX fork creates 50 folders.
Since I do not have most powerfull rig in the world (i9-9900k, Asus Rog Strix 3090, 64GB DDR4) running 14 clients put my processor workload at 80%.

If you have more powerfull PC feel free to increase number of clients.

What is the purpose of the NativeHelper library?

The goal was to send keyboard/mouse input directly to a specific Diablo game process, identified by its Process ID.
Implementing this reliably using only Python libraries proved challenging.

Since I have more experience with input handling in C# and Microsoft Spy++, I opted to create this helper library.
The source code for the library is included in the project, along with a link to its Virustotal Scan.

Why does the warrior have overpowered gear?

Providing the warrior with powerful gear was a shortcut to simplify the training process.
The primary focus was on teaching the agent dungeon navigation, effective combat strategies, and ultimately, how to defeat the Butcher.

Incorporating other complex game mechanics like inventory management, health/mana management, character point allocation, and trading would significantly increase the difficulty of the learning task.
The gear used by the agent was spawned using developer commands but consists of items that are legitimately obtainable within the game through finding or trading.

Why does the warrior have the Apocalypse skill?

I added the Apocalypse skill primarily for debugging purposes during development.
While examining the code, you might notice the monsters array in Utils.py, which tracks enemies.

I used the Apocalypse spell to quickly eliminate groups of monsters simultaneously to verify that this array was updating correctly.
The agent itself is not given an action that allows it to cast this spell; it's purely a leftover development tool and not part of the agent's learned abilities or action space.

Media

Evaluation run from Release 1.0.0 checkpoint

evaluation_hevc.mp4

Training run from Release 1.0.0 checkpoint

training_hevc.mp4

Note: If videos do not play or you wish to check them out in higher quality then Evaluation and Training

Credits

Special thanks to the following people for making this project possible:

• DevilutionX For the incredible modern port of Diablo 1, which this project builds upon.
• Peter Whidden For the inspiration drawn from his Pokémon Red experiments and his invaluable training explanation video (highly recommended).
• Stable Baselines3 - Contrib For their MaskablePPO implementation.
• David Brevik & Blizzard North For developing the legendary game that started it all.
• Everyone who contributed to this repository