Highlight: In this OpenCV with Python post we are going to talk about morphological transformations. Fundamentally, there are two basic morphological transformations and they are called dilation and erosion. They are present in image processing in different applications. They are used for the removal of noise or for finding the bumps or holes in images. In addition, these operations can also be used to calculate gradients of images. Moreover, once we learn two basic morphological operations we can combine them to create additional operations like opening and closing. So, let’s explore this very interesting topic.

Tutorial Overview:

1. What are dilation and erosion in the image processing?
2. What is opening and closing?
3. Morphological transformations with different types of kernels

1. What are dilation and erosion in the image processing?

What is dilation?

Morphological transformations of images consist of two basic operations: dilation and erosion. Dilation is exactly what it sounds like. It is an addition (expansion) of bright pixels of the object in a given image. It is illustrated in the image below.

So, how can we dilate or expand the image? We just need to perform the convolution on our input image with the kernel. The kernel can be of any shape, but usually, it is a square. One important thing to remember is that the kernel is defined with respect to the anchor point which is usually placed at the central pixel.

Now, in the Figure below, we have our input image (left image) and the kernel (middle image). The next step is to perform a dilation. We are going to take our kernel and run it across the whole image in order to calculate a local maximum for each position of the kernel. This local maximum we will store in the output image. So, the process of dilation can be seen as finding a local maximum of the current overlap between the image and the kernel. Note that the overlap with an image is defined as a set of pixels where the kernel has white pixels. An overlap of the image pixels with black pixels from the kernel we ignore. We can see the result of the dilation in the following GIF animation (right image). In our example we use binary images, so a white area is 1, and a black area is 0.

In the previous example we can see how the process of dilation actually looks like. We have a binary image on the left, and \(3\times 3 \) kernel in the middle with the anchor point at the central pixel. We moved our kernel across the image and obtained the output image on the right. In each position of the kernel we calculated a local maximum. In other words, if there was at least one value of 1 (white pixels) of the kernel that landed on the 1 (white pixel) in the input image, the result was 1. On the other hand, if the 1s of the kernel are overlapped only with 0s, our output was 0. We can clearly see that for dilation the white area in the output image became larger.

We can also use a different, let’s say \(3\times 3 \), kernel with only 1s. The boundaries of the object in the output image will grow by a certain additional layer. In the image below you can see an example of performing dilation on the original image with \(3\times 3 \) kernel that consists of all 1s. In addition, you can see the difference between original and dilated image. The difference explains how the newly created white pixels are distributed. Once, again, note that if we have applied a different kernel, our result would be different.

What is erosion?

In addition to dilation, we also have a complementary operation that is called an erosion. This operation is complete inversion of the dilation. The kernel is scanning the image, and it looks for the overlapping interval between the kernel and the image pixel. However, opposed to the dilation, here we are computing the local minimum. That means that only if 1s of the kernel are overlapped with only 1s of the image, the result will be 1 (white pixel). On the other hand, in all other cases the local minimum will be equal to 0 (black pixel). Once we process our original image with the kernel, the white area will shrink. The process of erosion is illustrated in the following GIF animation.

Python experiments with dilation and erosion

So, let’s see how we can implement this in OpenCV. There are two steps. First of all, we need to import our image. Then, we need to define our kernel.

As the first step, we will load our input image and we need to threshold it in order to create a binary image. In our previous post we have already explained a thresholding in more detail. Feel free to look at this post if you need to refresh your knowledge.

# Necessary imports
import numpy as np
import cv2
import matplotlib.pyplot as plt
from google.colab.patches import cv2_imshow

# Loading an input image and performing thresholding
img = cv2.imread('Billiards balls 1.jpg', cv2.IMREAD_GRAYSCALE)
_, mask=cv2.threshold(img, 230, 255, cv2.THRESH_BINARY_INV)

We will store the output of a thresholding operator in a variable that we will call a mask. We also have the output parameter that we will not store, and therefore, we will use the underscore line in the code (_). Additional parameters that we use are threshold intensity pixel values, which in our case are set to values of 230 and 255. This means that all values larger than 230 will be set to the value of 255. The last parameter that we use is a type of thresholding algorithm THRESH_BINARY_INV which will simply invert our values (all values smaller than 230 will become white, and all values larger than 230 will become black).

titles=["image","mask"]
images=[img, mask]

# Plotting the image and the mask
for i in range(2):
  plt.subplot(1, 2, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show  

Output:

You can see that colors close to white became black, whereas gray and black areas became white. 

Now, we will create a dilated image and for that we will use a function cv2.dilate(). An input to this function will be a mask image and a kernel. Here, we can define a kernel as a small square of ones with \(3\times 3 \) pixels size. It is important to note that the type of this image matrix should be an unsigned integer unit8.

# Creating a 3x3 kernel
kernel=np.ones((3,3), np.uint8)
# Performing dilation on the mask
dilation=cv2.dilate(mask, kernel)

for i in range(3):
  plt.subplot(1, 3, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show[]

Output:

We got the results of dilation and we can see that the white area has been extended. Note that these tiny black dots shrunk in the dilated image.

Maybe you have asked if it is possible to apply dilation more than once. Yes, it is possible. For that, we can use the parameter called the number of iterations. For instance, we can set this parameter to 10. This means that the process of dilation will be repeated 10 times consecutively. In the resulting image we can see that much more black regions disappeared from the image, whereas the white area expanded.

# Performing a dilation with a 3x3 kernel and the mask 10 times in a row
kernel=np.ones((3,3), np.uint8)
dilation=cv2.dilate(mask, kernel, iterations=10)

titles=["image","mask","dilation"]
images=[img, mask, dilation]

for i in range(3):
  plt.subplot(1, 3, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show 

Output:

It is good to remember that if you want to boost the enlargement of white areas, you can use a larger kernel. For instance, making use of a “all 1s” kernel of size \(5\times 5 \), will expand white area even more in the dilated image.

In a similar way, we apply an erosion operation by using a function cv2.erode(). This function has the same parameters as the cv2.dilate() function.

# Performing an erosion with a 3x3 kernel and the mask
kernel=np.ones((3,3), np.uint8)
dilation=cv2.dilate(mask, kernel)
erosion=cv2.erode(mask, kernel)

titles=["image","mask","dilation","erosion"]
images=[img, mask, dilation, erosion]

for i in range(4):
  plt.subplot(2, 2, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show 

Output:

We can see that the black areas increased with the application of cv2.erode() function. Note that in the erosion we have used a kernel that has only values of one.

2. What is opening and closing?

Another operator closely related to dilation and erosion is called an opening. It is actually an operation that consists of an erosion followed by a dilation. Pretty simple, right. For this, we will use the function cv2.morphologyEx(). This function can be used for several operations, so we will need to add a parameter to specify which one we want to use. Similarly, we have a closing operation which is an inverse of the opening. This means that we will apply dilation first, which is then followed by an erosion.

kernel=np.ones((3,3), np.uint8)
dilation=cv2.dilate(mask, kernel, iterations=1)
erosion=cv2.erode(mask, kernel, iterations=1)
opening=cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
closing=cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)

titles=["image","mask","dilation","erosion","opening","closing"]
images=[img, mask, dilation, erosion, opening,closing]

for i in range(6):
  plt.subplot(2, 3, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show 

Output:

Morphological gradient and Top hat operations

Now, lets have a look at two additional operations. The first one is called a morphological gradient. It is the difference between the dilation and the erosion of the image. The second one is a Top hat, and it is the difference between the image and the opening of the image.

kernel=np.ones((3,3), np.uint8)
dilation=cv2.dilate(mask, kernel, iterations=1)
erosion=cv2.erode(mask, kernel, iterations=1)
opening=cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
closing=cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
mg=cv2.morphologyEx(mask, cv2.MORPH_GRADIENT, kernel)
th=cv2.morphologyEx(mask, cv2.MORPH_TOPHAT, kernel)

titles=["image","mask","dilation","erosion","opening","closing","mg","th"]
images=[img, mask, dilation, erosion, opening, closing, mg, th]

for i in range(8):
  plt.subplot(2, 4, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show 

Output:

3. Morphological transformations with different types of kernels

Finally, in order to better visualize these operations, we can create a set of images processed with different types of kernels. First, we will create three different kernels. Then, we will process our image while performing several morphological operations. You can see the results in the following section.

img = np.zeros ( (30 , 30), np.uint8, )
# Creating 3x3 kernels of different shapes
kernel_1 =np.array ( [  [0,1,0], [1,1,1], [0,1,0] ]  )
kernel_2 =np.array ( [  [0,0,1], [0,0,1], [0,0,1] ]  )
kernel_3 =np.array ( [  [1,1,1], [0,0,0], [0,0,0] ]  )

kernel_1 = kernel_1.astype('uint8')
kernel_2 = kernel_2.astype('uint8')
kernel_3 = kernel_3.astype('uint8')

# we add a white letter 'D' on the black image - img
font = cv2.FONT_HERSHEY_PLAIN
cv2.putText(img, 'D', (5,25), font, 2, (255,255,255), 3);

titles=["kernel_1","kernel_2","kernel_3"]
images=[ kernel_1, kernel_2, kernel_3]

for i in range(3):
  plt.subplot(1, 3, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show 

Output:

# Performing dilation and erosion 
dilation_1 = cv2.dilate(img, kernel_1)
dilation_2 = cv2.dilate(img, kernel_2)
dilation_3 = cv2.dilate(img, kernel_3)
erosion_1 = cv2.erode(img, kernel_1)
erosion_2 = cv2.erode(img, kernel_2)
erosion_3 = cv2.erode(img, kernel_3)

titles=["dilation_1","dilation_2","dilation_3","erosion_1","erosion_2",
        "erosion_3"]
images=[ dilation_1, dilation_2, dilation_3, erosion_1, erosion_2,
        erosion_3]

for i in range(6):
  plt.subplot(2, 3, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show 

# Performing opening and closing  
opening_1 = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel_1)
opening_2 = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel_2)
opening_3 = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel_3)
closing_1 = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel_1)
closing_2 = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel_2)
closing_3 = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel_3)

titles=["opening_1","opening_2","opening_3","closing_1","closing_2",
        "closing_3"]
images=[ opening_1, opening_2, opening_3, closing_1, closing_2,
        closing_3]

for i in range(6):
  plt.subplot(2, 3, i+1), plt.imshow(images[i], "gray")
  plt.title(titles[i])
  plt.xticks([]),plt.yticks([])
plt.show 

Output:

In the following example you can see the difference between the original image and all of the processed images. We can achieve such an effect by subtracting the original image from the processed one.

plt.subplot(2,3,1)
plt.imshow(dilation_1-img, "gray")
plt.subplot(2,3,2)
plt.imshow(dilation_2-img, "gray")
plt.subplot(2,3,3)
plt.imshow(dilation_3-img, "gray")
plt.subplot(2,3,4)
plt.imshow(erosion_1-img, "gray")
plt.subplot(2,3,5)
plt.imshow(erosion_2-img, "gray")
plt.subplot(2,3,6)
plt.imshow(erosion_3-img, "gray")

plt.subplot(2,3,1)
plt.imshow(img-opening_1, "gray")
plt.subplot(2,3,2)
plt.imshow(img-opening_2, "gray")
plt.subplot(2,3,3)
plt.imshow(img-opening_3, "gray")
plt.subplot(2,3,4)
plt.imshow(closing_1-img, "gray")
plt.subplot(2,3,5)
plt.imshow(closing_2-img, "gray")
plt.subplot(2,3,6)
plt.imshow(closing_3-img, "gray")

Output:

Summary

In summary, this blog post explained dilation and erosion transformations. We have explained what they are and also how we can use them for image processing with OpenCV. We do hope that you loved our GIFs 🙂 In addition, we have learned how to combine a dilation and an erosion to create more advanced operations like opening and closing. In the next post, we will talk about a machine learning algorithm called k-Means clustering and how it can be applied in the color quantisation process.

References:

[1] OpenCV Python Tutorial For Beginners 17 – Morphological Transformations

[2] Practical Python and OpenCV by Adrian Rosebrock