In this post, we will see how to implement a deep Neural Network in Python from scratch. It isn’t something that we will do often in practice, but it is a good way to understand the inner workings of Deep Learning.  First, we will import the libraries we will use in the following code. In the following code, we will define activation functions: \(sigmoid \), \(ReLU \), and \(tanh \) we will also save values…
Read more

The Computation graph – Example Let’s say that we’re trying to compute a function \(J\), which is a function of three variables \(a\), \(b\), and \(c\) and let’s say that function \(J\) is \(3(a+bc)\). Computation of this function has actually three distinct steps: Compute  and store it in the variable \(u\), so \(u = bc \) Compute \(v = a + u \), Output \(J \) is \(3v\). Let’s summarize: $$ J(a, b, c) =…
Read more

Gradient Descent in Python We will first import libraries as NumPy, matplotlib, pyplot and derivative function. Then with a NumPy function – linspace() we define our variable \(w \) domain between 1.0 and 5.0 and 100 points. Also we define alpha which will represent learning rate. Next, we will define our \(y \) ( in our case \(J(w) \)) and plot to see a convex function, we will use \((w-3)^2  \). So we can see that…
Read more

Gradient Descent A term “Deep learning” refers to training neural networks and sometimes very large neural networks. They are massively used for problems of classification and regression. The main goal is to optimize parameters  and  ( both in logistic regression and in neural networks ). Gradient Descent achieved amazing results in solving these optimization tasks. Not so new, developed in the 1940s, 1950s and 1960s. Hence, Gradient Descent is an algorithm that tries to minimize the error (cost)…
Read more

Image representation in a computer The computer stores 3 separate matrices corresponding to the red, green and blue (RGB) color channels of the image. If the input image is 64 by 64 pixels, then we would have three 64 by 64 matrices corresponding to the red, green and blue pixel intensity values for our image. For a 64 by 64 image – the total dimension of this vector will be =64*64*3=12288. Notation that we will follow…
Read more