Using Image Recognition to Automate More Mario Minigames

Introduction

Last year, I embarked on an exciting project where I developed Python scripts that utilized image recognition technology to automatically play a few mini-games from New Super Mario Brothers on the Nintendo DS. Inspired by my previous success, I've decided to tackle the mini-games from Mario Party DS this year. With many mini-games yet to be automated, I thought it would be fun to experiment once again, bringing in some crazier ideas along the way. So, without further ado, let's dive right in!

Automating “Whack-A-Monty”

Our first target is a mini-game called Whack-a-Monty. As the name suggests, players must whack as many Monty moles as possible within a 30-second time limit. The game begins easily, but as time progresses, it becomes nearly impossible to keep up with the sheer number of Monty moles appearing on the screen. I believed this mini-game would be a perfect candidate for image recognition to get started.

To tackle this mini-game, I created a new Python script, leveraging the project I developed last year, which helped streamline the process. The first step was to load an image of a Monty mole so the program could click on them upon detection. I wrote code to locate the Monty mole texture on the DS’s bottom screen. After some testing, I programmed the mouse to move to the center of each detected Monty mole, and it worked flawlessly.

After pushing the score to 57, I wasn't satisfied. I sought to optimize the script further. I decided to search for only one Monty mole at a time, which significantly increased the score to 85. Finally, by fine-tuning the speed of each mouse click to the absolute minimum required for the game to recognize it, I reached an impressive score over 100. With this success, it was time to move on to the next mini-game.

Automating “Snowball Slalom”

Next up is Snowball Slalom. The objective here is to roll a snowball down a slippery track filled with obstacles while avoiding rocks and penguins that slow down the snowball. The key to a high score is swiftly dodging these obstacles, so I started by optimizing the cursor movement for maximum speed.

However, I encountered challenges avoiding obstacles without risking collisions with the sides of the track. My initial approach involved locating each rock to swerve away from them. Despite some success, the strategy proved flawed. Instead, I pivoted to an idea rooted in my previous work with neural networks where I utilized evolutionary algorithms.

My new strategy involved directing the snowball’s movement forward until it hit an obstacle and then randomly adjusting its path left or right to discover a more effective route. By tracking the snowball's position relative to the distance meter, I built in a mechanism that reset whenever the snowball got stuck for a few frames. The results were impressive, showing a quick completion of the track!

Automating “Danger Bomb”

The final mini-game I automated is Danger Bomb. Here, the goal is to protect a red bomb from being ignited by fireballs and Bowser's breath. I began by coding the detection of fireballs and employed trigonometry to calculate the angles between each fireball and our bomb player. The mouse movement was then programmed to steer away from the closest fireball.

While this approach initially worked, the bomb often got trapped at the screen's edges. To resolve this, I treated the map's edges like fireballs, allowing for appropriate avoidance of corners. Additionally, I coded a detection method for Bowser's breath when his eyes opened, enhancing the bomb's protection. The outcome was a successful run that, while not a super high score, demonstrated satisfactory automation.

Conclusion

With these mini-games now fully automated, I'm thrilled with the results and the innovative ideas that emerged from combining image recognition and AI. The project was complex but rewarding, and it’s clear there’s more to explore in this area.

---

Keyword

• Image Recognition
• Python Scripts
• Mario Party DS
• Automated Gaming
• Whack-a-Monty
• Snowball Slalom
• Danger Bomb
• Neural Networks
• AI Algorithms

FAQ

Q1: What is image recognition?
A1: Image recognition is a technology that allows a program to identify objects or patterns in images, commonly used in various fields including automation and AI.

Q2: How were the scores in the mini-games measured?
A2: Scores were determined by the number of successful actions executed within the game, such as hitting Monty moles, avoiding obstacles, or protecting the bomb.

Q3: What programming language was used for automation?
A3: Python was the programming language used to write the scripts for automating the mini-games.

Q4: Did you use any AI methods in your automation?
A4: Yes, AI algorithms were used, especially evolutionary algorithms, to optimize the snowball's trajectory in the Snowball Slalom mini-game.

Q5: Can we expect more automated mini-games in the future?
A5: Definitely! There are still many mini-games left to automate, and feedback from viewers could spark more ideas for future projects.