Category: Other

#010A Gradient Descent for Neural Networks

  In this post we will see how to implement gradient descent for one hidden layer Neural Network as presented in the picture below. One hidden layer Neural Network   Parameters for one hidden layer Neural Network are \(\textbf{W}^{[1]} \), \(b^{[1]} \), \(\textbf{W}^{[2]} \) and \(b^{[2]} \). Number of unitis in each layer are:  input of a Neural Network is feature vector ,so the length of “zero” layer \(a^{[0]} \) is the size of an input feature…
Read more

#009 Activation functions and their derivatives

Activation functions When we build a neural network, one of the choices we have to make is what activation functions to use in the hidden layers as well as at the output unit of the  Neural Network. So far, we’ve just been using the sigmoid activation function but sometimes other choices can work much better.  Let’s take a look at some of the  options. \(sigmoid \) activation function In the forward propagation steps for Neural Network there are two steps…
Read more

#008 Shallow Neural Network

Vectorizing Across Multiple Training Examples In this post we will see how to vectorize across multiple training examples. The outcome will be similar to what we saw in Logistic Regression. Equations we defined in previous post are these: \(z^{[1]} =\textbf{W}^{[1]} x + b ^{[1]} \) \(a^{[1]} = \sigma ( z^{[1]} ) \) \(z^{[2]} = \textbf{W}^{[2]} a^{[1]} + b ^{[2]} \) \(a^{[2]} = \sigma ( z^{[2]} ) \) These equations tell us how, when given an…
Read more

#006A Fast Logistic Regression

Fast Logistic Regression When we are programming Logistic Regression or Neural Networks we should avoid explicit \(for \) loops. It’s not always possible, but when we can, we should use built-in functions or find some other ways to compute it. Vectorizing the implementation of Logistic Regression  makes the code highly efficient. In this post we will see how we can use this technique to compute gradient descent without using even a single \(for \) loop.…
Read more

#005A Logistic Regression from scratch

Gradient Descent on m Training Examples In this post we will talk about applying gradient descent on \(m\) training examples. Now the question is how we can define what gradient descent is? A gradient descent is an efficient optimization algorithm that attempts to find a global minimum of a function. It also enables a model to calculate the gradient or direction that the model should take to reduce errors (differences between actual \(y\) and predicted \(\hat{y}\)).  Now…
Read more