The History of Machine Learning in Trackmania
Trackmania, a racing game, inspires machine learning projects aiming for superhuman performance. Players like Rottaca and TMRL use supervised and unsupervised learning. Innovations include track lookahead and diverse learning algorithms. Programs show progress but not human-level skills. The goal is a top-performing AI on a desktop PC, blending gaming and machine learning for innovation.
Read original article
Trackmania, a competitive racing game, has sparked interest in machine learning applications. The goal is to train a neural network to achieve superhuman performance in the game's Cup of the Day event. Various players and groups have developed autonomous programs to play Trackmania, such as Rottaca's supervised learning approach and TMRL's unsupervised learning pipeline. Laurens Neinders improved performance by incorporating track curvature lookahead, while AndrejGobeX experimented with different learning algorithms. Results varied, with some programs showing promising advancements but still falling short of human-level performance. The ultimate aim is to create a program that can excel in Trackmania using machine learning techniques on a single desktop PC. The intersection of gaming and machine learning presents exciting challenges and opportunities for innovation in the field.
Related
AMD MI300x GPUs with GEMM tuning improves throughput and latency by up to 7.2x
Nscale explores AI model optimization through GEMM tuning, leveraging rocBLAS and hipBLASlt for AMD MI300x GPUs. Results show up to 7.2x throughput increase and reduced latency, benefiting large models and enhancing processing efficiency.
AMD MI300x GPUs with GEMM tuning improves throughput and latency by up to 7.2x
Nscale explores GEMM tuning impact on AI model optimization, emphasizing throughput and latency benefits. Fine-tuning parameters and algorithms significantly boost speed and efficiency, especially on AMD GPUs, showcasing up to 7.2x throughput improvement.
Analysing 16,625 papers to figure out where AI is headed next (2019)
MIT Technology Review analyzed 16,625 AI papers, noting deep learning's potential decline. Trends include shifts to machine learning, neural networks' rise, and reinforcement learning growth. AI techniques cycle, with future dominance uncertain.
My Python code is a neural network
Neural networks are explored for identifying program code in engineering messages. Manual rules and a Python classifier are discussed, with a suggestion to use a recurrent neural network for automated detection.
Getting the World Record in Hatetris (2022)
David and Felipe set a world record in HATETRIS, a tough Tetris version. They used Rust, MCTS, and AlphaZero concepts to enhance gameplay, achieving a score of 66 points in 2021.
10 commentsJust wanted to provide some perspective here on how many things those projects need to take care of in order to get some training setup going.
I'm the developer behind TMInterface [1] mentioned in this post, which is a TAS tool for the older TrackMania game (Nations Forever). For Linesight (last project in this post), I recently ended up working with its developers to provide them the APIs they need to access from the game. There's a lot of things RL projects usually want to do: speed up the game (one of the most important), deterministically control the vehicle, get simulation information, navigate menus, skip cut scenes, make save states, capture screenshots etc. Having each of those things implemented natively greatly impacts the stability and performance of training/inference in a RL agent, e.g. for the latest version the project uses a direct capture of the surface that's rendered to the game window, instead of using an external Python library (DxCam). This is faster, doesn't require any additional setup and also allows for training even if the game window is completely occluded by other windows.
There are also many other smaller annoying things: many games throttle FPS if the window is unfocused which is also the case here, and the tool patches out this behaviour for the project, and there's a lot more things like this. The newest release of Linesight V3 [2] can reliably approach world records and it's being trained & experimented with by quite a few people. The developers made it easy to setup and documented a lot of the process [3].
[1] https://donadigo.com/tminterface/
> Thus far, every attempt at training a Trackmania-playing program has trained the program on one map at a time. As a result, no matter how well the network did on one track, it would have to be retrained - probably significantly retrained
This is a crucial aspect when talking about RL. Most of the Trackmania AI attempts focuses on a track at a time, which is not really a problem since they want to, given an individual track, outperform the best human racers.
However, it is this nuance that a lot of more business oriented users don't get when being sold on some fancy new RL project. In the real world (think self-driving cars), we typically want agents to be way more able to generalize.
Most of the RL techniques we have do rather well in these kinds of constrained environments (in a sense they eventually start overfitting on the given environment), but making them behave well in more varied environments is way harder. A lot of beginner RL tutorials also fail to make this very explicit, and will e.g. show how to train an agent to find the exit in a maze without ever trying it on a newly generated maze :).
Tangentially related, is anybody besides the autonomous car folks developing games or virtual environments designed from the ground up for exporting machine learning APIs? By this I mean exporting game state and accepting game controls through the network without going through adapter contortions.
I think that point is an important one. ML algorithms work better when they are given better context. Especially in programming, it is clear the models are trained on code, rather than repositories. They know about files and repositories, but i always get the impression that they are totally clueless about whole programs.
What could be done better in code, is provide in training more data about where each function is located in the project, some other files where similar functions are defined or called and so on. In general before each code is fed into the training, to do a little bit of data mining in the project like the tree-hugger project [1] enables. Tree-hugger however is a little bit older code, and tree-sitter has advanced a lot the last 4 years.
In my opinion 5x to 10x in code, is within reach, with no need to increase GPU compute or electricity.
I do love Trackmania. I'd like to play 2020 but alas I do not have a compatible computer. I mostly play the Wii version.
It literally doesn't matter if it's 1px or 100 meters to the wall, just learn to not hit it.
Instead, measure from the _camera_. That's all that matters. That's what humans do when we play.
Bonus: with this added perspective you'll be able to drive maps with hills and jumps. Not just flat maps.
E.g. can you improve or replicate missile intercept algorithms
Related
AMD MI300x GPUs with GEMM tuning improves throughput and latency by up to 7.2x
Nscale explores AI model optimization through GEMM tuning, leveraging rocBLAS and hipBLASlt for AMD MI300x GPUs. Results show up to 7.2x throughput increase and reduced latency, benefiting large models and enhancing processing efficiency.
AMD MI300x GPUs with GEMM tuning improves throughput and latency by up to 7.2x
Nscale explores GEMM tuning impact on AI model optimization, emphasizing throughput and latency benefits. Fine-tuning parameters and algorithms significantly boost speed and efficiency, especially on AMD GPUs, showcasing up to 7.2x throughput improvement.
Analysing 16,625 papers to figure out where AI is headed next (2019)
MIT Technology Review analyzed 16,625 AI papers, noting deep learning's potential decline. Trends include shifts to machine learning, neural networks' rise, and reinforcement learning growth. AI techniques cycle, with future dominance uncertain.
My Python code is a neural network
Neural networks are explored for identifying program code in engineering messages. Manual rules and a Python classifier are discussed, with a suggestion to use a recurrent neural network for automated detection.
Getting the World Record in Hatetris (2022)
David and Felipe set a world record in HATETRIS, a tough Tetris version. They used Rust, MCTS, and AlphaZero concepts to enhance gameplay, achieving a score of 66 points in 2021.