In this post we will see how to implement gradient descent for one hidden layer Neural Network as presented in the picture below. One hidden layer Neural Network   Parameters for one hidden layer Neural Network are \(\textbf{W}^{[1]} \), \(b^{[1]} \), \(\textbf{W}^{[2]} \) and \(b^{[2]} \). Number of unitis in each layer are:  input of a Neural Network is feature vector ,so the length of “zero” layer \(a^{[0]} \) is the size of an input feature…
Read more

Vectorizing Across Multiple Training Examples In this post we will see how to vectorize across multiple training examples. The outcome will be similar to what we saw in Logistic Regression. Equations we defined in previous post are these: \(z^{[1]} =\textbf{W}^{[1]} x + b ^{[1]} \) \(a^{[1]} = \sigma ( z^{[1]} ) \) \(z^{[2]} = \textbf{W}^{[2]} a^{[1]} + b ^{[2]} \) \(a^{[2]} = \sigma ( z^{[2]} ) \) These equations tell us how, when given an…
Read more

Fast Logistic Regression When we are programming Logistic Regression or Neural Networks we should avoid explicit \(for \) loops. It’s not always possible, but when we can, we should use built-in functions or find some other ways to compute it. Vectorizing the implementation of Logistic Regression  makes the code highly efficient. In this post we will see how we can use this technique to compute gradient descent without using even a single \(for \) loop.…
Read more

Gradient Descent on m Training Examples In this post we will talk about applying gradient descent on \(m\) training examples. Now the question is how we can define what gradient descent is? A gradient descent is an efficient optimization algorithm that attempts to find a global minimum of a function. It also enables a model to calculate the gradient or direction that the model should take to reduce errors (differences between actual \(y\) and predicted \(\hat{y}\)).  Now…
Read more