Fast Logistic Regression When we are programming Logistic Regression or Neural Networks we should avoid explicit \(for \) loops. It’s not always possible, but when we can, we should use built-in functions or find some other ways to compute it. Vectorizing the implementation of Logistic Regression  makes the code highly efficient. In this post we will see how we can use this technique to compute gradient descent without using even a single \(for \) loop.…
Read more

Gradient Descent on m Training Examples In this post we will talk about applying gradient descent on \(m\) training examples. Now the question is how we can define what gradient descent is? A gradient descent is an efficient optimization algorithm that attempts to find a global minimum of a function. It also enables a model to calculate the gradient or direction that the model should take to reduce errors (differences between actual \(y\) and predicted \(\hat{y}\)).  Now…
Read more

Logistic Regression – the Computation Graph Why do we need a computation graph? To answer this question, we have to check how the computation for our neural network is organized. There are two important principles in neural network computation: Forward pass or forward propagation step Backward pass or backpropagation step During NN’s forward propagation step we compute the output of our neural network. In a binary classification case, our neural network output is defined by…
Read more

 Logistic Regression – Cost Function Optimization First, to train parameters \(w \) and \(b \)  of a logistic regression model we need to define a cost function. Given a training set of \(m\) training examples, we want to find parameters \(w\) and \(b \), so that \(\hat{y}\) is as close to \(y \) (ground truth). Here, we will use \((i) \) superscript to index different training examples. Henceforth, we will use loss (error) function \(\mathcal{L}\) to measure how well our algorithm is doing. The loss…
Read more

Binary Classification Binary classification is the task of classifying elements of a given set into two groups. Logistic regression is an algorithm for binary classification. Example of a binary classification problem: We have an input image \(x\) and the output \(y\) is a label to recognize the image. 1 means cat is on an image, 0 means that a non-cat object is in an image. In binary classification, our goal is to learn a classifier that can…
Read more