As I delved into the world of AI and machine learning, I wanted to ensure I had a solid grasp of the underlying math and structure. So, I embarked on a journey to create an MNIST model from scratch using JavaScript. However, I soon found myself stuck with a dismal 1% accuracy. I knew something was amiss, but I couldn’t quite put my finger on the problem.
If you’re like me, struggling to get your MNIST model off the ground, this post is for you. I’ll walk you through my thought process, the mistakes I made, and the lessons I learned along the way.
First, let’s talk about the importance of understanding the fundamentals. When building a model from scratch, it’s essential to have a clear understanding of the math behind it. I wanted to see how the training process worked mathematically on an example-by-example basis, so I opted to pass training examples as single vectors rather than matrices containing multiple examples.
Now, let’s dive into the code. I’ve included the entire code snippet below, but I’ll highlight some key takeaways. One crucial aspect is the initialization of weights and biases with small random values. I used the mathjs library to generate these values.
Another critical component is the backpropagation function, which calculates the gradients for each parameter. I used the chain rule to derive these gradients, and I’ll explain the process in more detail later.
So, what went wrong? After reviewing my code, I realized that I was making a few critical mistakes. First, I was averaging the gradients across all training examples, but I wasn’t updating the model parameters accordingly. Second, I wasn’t using a batch size, which led to inefficient computation.
To fix these issues, I revised my learn function to update the model parameters after each batch and implemented a batch size to improve efficiency.
The moral of the story? Building a model from scratch can be challenging, but it’s an invaluable learning experience. Don’t be afraid to ask for help, and don’t be discouraged by setbacks. With persistence and patience, you’ll get there.
If you’re interested in exploring more, I recommend checking out the LWiAI Podcast (#217) for a detailed explanation of the MNIST model and its applications.