Networks

Introduction This is actually a sort of back-to-the-future post because it’s related to something I completed one year ago: I built this Elman network and it learnt to count. What I shame, I forgot it, now it’s kind of its first birthday so let’s celebrate :D This is Elman, the best in class in adding int32 numbers. For everybody who already knows what I will talk about (what?!), here’s the Github repo. I’m sorry for the name, it’s still go-perceptron-go but that repo contains my GoLang ANN. ...

History We can date the birth of artificial neural networks in 1958, with the introduction of Perceptron 1 by Frank Rosenblatt. It was the first algorithm created to reproduce the biological neuron. Conceptually, the easier perceptron that you might think of is made of a single neuron: when it’s exposed to a stimulus, it provides a binary response, just as would a biological neuron. This model differs greatly from the neural network involving billions of neurons in a biological brain. Shortly after his birth, the researchers showed the world the problems of Perceptron: in fact, it was quickly proved that perceptrons could not be trained to recognize many classes of input patterns. To get a more powerful network, it was necessary to take advantage of multiple level of units and create a multilayers perceptron, with more intermediates neurons used to solve linearly separable2 subproblems, whose outputs were combined together by the final level to provide a concrete response to original input problem. Even though the Perceptron was just a simple but severely limited binary classifier, it introduced a great innovation: the idea to simulate the basic computational unit of a complex biological system that exists in nature. ...