Let's talk about AI. Rather, let's talk about this project, which is a neural network built from scratch (no tensorflow, no pytorch, nothing).
So far, I have used this neural network to successfully complete two tasks:
- Train on and find a defined function (for example, y < sin(6x))
- Train on a dataset of MNIST handwritten digits and detect handwritten digits through a pygame interface
For installation, download this GitHub repo and run any of the files!
The custom neural network (NN) takes a learning rate (float), training data (arr[DataPoints]), and a structure (arr). The NN then uses a backpropagation algorithm, utilizing gradient descent, in order to train. In the NN file, there are several activation functions to choose from depending on your data (LeakyRELU is recommended as a starting point).
For training on a specific mathematical function, the NN file uses the method 'gradient_function' in order to generate any number of starting training values. The NN then trains on this data, providing debug plots as shown below:
Scatter plot showing actual NN results in 2D space
Scatter plot showing rounded NN results in 2D space
Scatter plot showing NN accuracy (green = accurate prediction, red = inaccurate prediction)
Line plot showing cost vs epochs
As you can see, the NN very accurately found the sin(6x) function, albeit given a large number of epochs.
For training on the MNIST handwritten digits, the mnist_train file uses the NN and NN Trainer class in order to train from 60,000 28x28 pixel monochrome images of handwritten digits. The file an image modification algorithm randomly resizes and rotates each image, in addition to adding salt and pepper noise, in an effort to avoid overfitting. Example handwritten digits after image modification algorithm:
Five handwritten digits resized, rotated, and with salt and pepper noise applied
The mnist_test file displays an accuracy rating of the model in the terminal for both training and test data. It also displays 10 example images of where the model made mistakes to help debug. Currently, my model has an accuracy of 93.05%.
Ten mistakenly classified digits, labeled the model identifications
Finally, the mnist_drawing_test file opens a pygame window to allow users to draw their own digits. These are identified by the NN in real time and a % ranking is displayed on the side for each possible classification.
A drawing of the number six with the NN identifying it with a 95% likelihood
While this outlet is not perfect, as it does not accurately represent the training images, it is more applicable to real life and also more fun.
Here are some of the resources I used while creating this project
- This project is based upon this video by Sebastian Lague
- I used ChatGPT and JebrainsAI in a limited fashion on this project