Programming an AI-based 2048 game (Part 4: Neural Network and Wrapping Up)

This fourth blog post concludes my personal project of developing an AI-based 2048 video game in Python 3. Feel free to check the first blog post of this series out if you haven’t read it yet.

If you want more details, the source code of the project is also available on Github.

This fourth and last post will focus on implementing the Neural Network itself and show a basic train/play cycle. Initially, I also wanted to demonstrate how this AI-based 2048 was (almost) always performing better than a Human but, well, you will see that it did not go as good as planned… 😀

What is an artificial Neural Network?

Simply explained, a Neural Network (also known as artificial Neural Network) is a collection of connected nodes that is able to generate output(s) according to some input(s), based on the connections between its different nodes.

The figure below shows a common representation of a Neural Network with three layers (Input, Hidden, Output) and multiple nodes (or neurons, in reference to the biological brain).

Although each neuron seems to be connected to all nodes of the subsequent layer, it is a bit more complicated than that. In fact, just like in a biological brain, neurons communicate with each other thanks to synapses (the edges in the figure). Each neuron has a certain sensitivity level (often modeled thanks to an activation function and a threshold), which prevents a neuron to be activated if the stimulus was not strong enough.

Regarding Neural Networks, the whole point of the learning process is to determine how strong should the signal be (i.e. the stimulus) passing through each connection. For that reason, the Neural Network has to be appropriately trained on a representative training set (the more representative of the future inputs, the better).

I won’t go in more details about Neural Networks but you can check the two following websites if if you want to learn more:

• https://mlfromscratch.com/neural-networks-explained/
• https://blog.zhaytam.com/2018/08/15/implement-neural-network-backpropagation/

How did I apply Neural Network to the 2048 video game?

The Neural Network that I defined was quite (too much?) simple. It was composed of:

• 1 Input layer with 16 neurons (one for each grid cell);
• 1 Hidden layer with 4 neurons;
• 1 Output layer with 4 neurons (one for each direction – Up, Down, Left, Right).

The 4×4=16 grid tiles represented the Neural Network inputs. The directions to be played next (Up, Down, Left, Right) were the Neural Network outputs.

I trained my Neural Network on three 2048 games where I achieved to get the 2048, 1024 and 512 tiles, respectively.

As a reminder, below is an example of a 2048 game log history:

...
749 16124 1024 8 4 4 512 32 0 0 256 128 2 4 128 128 16 0 Left [0]
750 16388 1024 8 8 0 512 32 0 0 256 128 2 4 256 16 2 0 Up [0]
751 16904 1024 8 8 4 512 32 4 4 512 128 0 0 0 16 0 0 Up [0]
752 17936 1024 8 8 8 1024 32 4 0 0 128 0 4 0 16 0 0 Up [0]
753 19984 2048 8 8 8 4 32 4 4 0 128 0 0 0 16 0 0 WIN [-1]

Did it work?

Well, yes and no. Yes because my program was able to play an entire 2048 game autonomously until it was winning or loosing. And no because it didn’t perform as good as expected (spoiler alert: it was only loosing).

A simple example being worth a thousand words, below are three GIFs showing the start of three different 2048 games:

• The first GIF shows the start of a 2048 game played by a Human (i.e. me);
• The second GIF shows the start of a 2048 game played by my “trained” Neural Network;
• The last GIF shows the start of a 2048 game where a direction (Up, Down, Left, Right) is randomly picked by the program at each round.

By taking these three different 2048 games, after 8 rounds, we can notice the following scores:

Human	40 points
“Trained” Neural Network	20 points
Random mode	24 points

The fact that one playing randomly could achieve better results than a trained Neural Network could be a bit surprising. In reality, it just means that my Neural Network was not trained in a proper way and/or was just not suitable for this type of problem.

Possible improvements and closing remarks

Many possible improvements could be imagined in order to improve the average score that my AI-based 2048 program was getting. For instance:

• A better and more diverse training set, with 2048 games played by many players and not just by me;
• A more complex Neural Network structure (more layers, different layer compositions, etc.);
• More advanced Neural Network with “memory”, such as LSTM networks, in order to predict future outputs by taking into account sequences of past states;
• Simpler algorithms such as MinMax could also work, given the Grid size and therefore the search space.

That’s all folks, I hope that you enjoyed this series of blog posts on 2048 game, AI and Python. I honestly didn’t think that writing these four blog posts would take me that long. It is one thing to develop a program/software/algorithm but it is really another exercise to explain why and how we did it.

Stay tuned for more content and do not hesitate to drop a comment if you want to share something 🙂