Gradient Descent on m Training Examples In this post we will talk about applying gradient descent on \(m\) training examples. Now the question is how we can define what gradient descent is? A gradient descent is an efficient optimization algorithm that attempts to find a global minimum of a function. It also enables a model to calculate the gradient or direction that the model should take to reduce errors (differences between actual \(y\) and predicted \(\hat{y}\)).  Now…
Read more

Logistic Regression – the Computation Graph Why do we need a computation graph? To answer this question, we have to check how the computation for our neural network is organized. There are two important principles in neural network computation: Forward pass or forward propagation step Backward pass or backpropagation step During NN’s forward propagation step we compute the output of our neural network. In a binary classification case, our neural network output is defined by…
Read more

 Logistic Regression – Cost Function Optimization First, to train parameters \(w \) and \(b \)  of a logistic regression model we need to define a cost function. Given a training set of \(m\) training examples, we want to find parameters \(w\) and \(b \), so that \(\hat{y}\) is as close to \(y \) (ground truth). Here, we will use \((i) \) superscript to index different training examples. Henceforth, we will use loss (error) function \(\mathcal{L}\) to measure how well our algorithm is doing. The loss…
Read more

Binary Classification Binary classification is the task of classifying elements of a given set into two groups. Logistic regression is an algorithm for binary classification. Example of a binary classification problem: We have an input image \(x\) and the output \(y\) is a label to recognize the image. 1 means cat is on an image, 0 means that a non-cat object is in an image. In binary classification, our goal is to learn a classifier that can…
Read more