Deep L-layer Neural Network In this post we will make a Neural Network overview. We will see what is the simplest representation of a Neural Network and how deep representation of a Neural Network looks like. You may have heard that the perceptron is the simplest version of a Neural Network. The perceptron is a one layer Neural Network with the \(step\) activation function . In the previous posts we have defined a Logistic Regression…
Read more
In this post we will see how to implement gradient descent for one hidden layer Neural Network as presented in the picture below. One hidden layer Neural Network Parameters for one hidden layer Neural Network are \(\textbf{W}^{[1]} \), \(b^{[1]} \), \(\textbf{W}^{[2]} \) and \(b^{[2]} \). Number of unitis in each layer are: input of a Neural Network is feature vector ,so the length of “zero” layer \(a^{[0]} \) is the size of an input feature…
Read more
Activation functions When we build a neural network, one of the choices we have to make is what activation functions to use in the hidden layers as well as at the output unit of the Neural Network. So far, we’ve just been using the sigmoid activation function but sometimes other choices can work much better. Let’s take a look at some of the options. \(sigmoid \) activation function In the forward propagation steps for Neural Network there are two steps…
Read more
Vectorizing Across Multiple Training Examples In this post we will see how to vectorize across multiple training examples. The outcome will be similar to what we saw in Logistic Regression. Equations we defined in previous post are these: \(z^{[1]} =\textbf{W}^{[1]} x + b ^{[1]} \) \(a^{[1]} = \sigma ( z^{[1]} ) \) \(z^{[2]} = \textbf{W}^{[2]} a^{[1]} + b ^{[2]} \) \(a^{[2]} = \sigma ( z^{[2]} ) \) These equations tell us how, when given an…
Read more
A quick overview In previous posts we had talked about Logistic Regression and we saw how this model corresponds to the following computation graph: We have a feature vector \(x \) , parameters \(w \) and \(b \) as the inputs to the computation graph. That allows us to compute \(z \) which is then used to compute \(a \) and we use \(a \) interchangeably with the output \(\hat{y} \). Finally, we can compute a…
Read more
