Building blocks of a Deep Neural Network In this post we will see what are the building blocks of a Deep Neural Network. We will pick one layer, for example layer \(l \) of a deep neural network and we will focus on computatons for that layer. For layer \(l \) we have parameters \(\textbf{W}^{[l]} \) and . Calculation of the forward pass for layer we get as we input activations from the previous layer…
Read more

  In this post we will see how to implement gradient descent for one hidden layer Neural Network as presented in the picture below. One hidden layer Neural Network   Parameters for one hidden layer Neural Network are \(\textbf{W}^{[1]} \), \(b^{[1]} \), \(\textbf{W}^{[2]} \) and \(b^{[2]} \). Number of unitis in each layer are:  input of a Neural Network is feature vector ,so the length of “zero” layer \(a^{[0]} \) is the size of an input feature…
Read more

Vectorizing Across Multiple Training Examples In this post we will see how to vectorize across multiple training examples. The outcome will be similar to what we saw in Logistic Regression. Equations we defined in previous post are these: \(z^{[1]} =\textbf{W}^{[1]} x + b ^{[1]} \) \(a^{[1]} = \sigma ( z^{[1]} ) \) \(z^{[2]} = \textbf{W}^{[2]} a^{[1]} + b ^{[2]} \) \(a^{[2]} = \sigma ( z^{[2]} ) \) These equations tell us how, when given an…
Read more

Fast Logistic Regression When we are programming Logistic Regression or Neural Networks we should avoid explicit \(for \) loops. It’s not always possible, but when we can, we should use built-in functions or find some other ways to compute it. Vectorizing the implementation of Logistic Regression  makes the code highly efficient. In this post we will see how we can use this technique to compute gradient descent without using even a single \(for \) loop.…
Read more