Deep neural network trained on MNIST dataset

Below is my attempt to implement a rudimentary deep neural network in Javascript and make it interactive on browser.

I followed the Deep Learning chapter of the Data Science from Scratch book and trained a MNIST data classifier. This is the model:

const model = neural.newSequential([
  ["dense1", neural.newDense(784, 30, neural.xavier, neural.newTanh())],
  ["dropout1", neural.newDropout(0.2)],
  ["dense2", neural.newDense(30, 10, neural.xavier, neural.newTanh())],
  ["dropout2", neural.newDropout(0.2)],
  ["dense3", neural.newDense(10, 10, neural.xavier, null)],
]);

Draw a shape of a number (0~9) on the area below and see the classification results and also the output from each layer.

The parameters were trained offline with Deno and the model got 0.926 accuracy on test data, but it appears that the accuracy is not great on this interactive toy. I think it is because I am not doing any centering in the preprocessing step so the model does not work well with translational variantions in inputs. TODO: improve => done adding centering to the preprocessing. It was the lack of centering that reduced the accuracy greatly.

Sources

• https://yann.lecun.com/exdb/mnist/