Vectorizing Across Multiple Training Examples In this post we will see how to vectorize across multiple training examples. The outcome will be similar to what we saw in Logistic Regression. Equations we defined in previous post are these: \(z^{[1]} =\textbf{W}^{[1]} x + b ^{[1]} \) \(a^{[1]} = \sigma ( z^{[1]} ) \) \(z^{[2]} = \textbf{W}^{[2]} a^{[1]} + b ^{[2]} \) \(a^{[2]} = \sigma ( z^{[2]} ) \) These equations tell us how, when given an…
Read more

Fast Logistic Regression When we are programming Logistic Regression or Neural Networks we should avoid explicit \(for \) loops. It’s not always possible, but when we can, we should use built-in functions or find some other ways to compute it. Vectorizing the implementation of Logistic Regression  makes the code highly efficient. In this post we will see how we can use this technique to compute gradient descent without using even a single \(for \) loop.…
Read more

Gradient Descent on m Training Examples In this post we will talk about applying gradient descent on \(m\) training examples. Now the question is how we can define what gradient descent is? A gradient descent is an efficient optimization algorithm that attempts to find a global minimum of a function. It also enables a model to calculate the gradient or direction that the model should take to reduce errors (differences between actual \(y\) and predicted \(\hat{y}\)).  Now…
Read more

Logistic Regression – the Computation Graph Why do we need a computation graph? To answer this question, we have to check how the computation for our neural network is organized. There are two important principles in neural network computation: Forward pass or forward propagation step Backward pass or backpropagation step During NN’s forward propagation step we compute the output of our neural network. In a binary classification case, our neural network output is defined by…
Read more