Image Semantic Segmentation

 What is Semantic Segmentation?

Semantic Segmentation is one of the most famous computer vision tasks and it is widely used in autonomous driving, medical image analysis, and aerial image processing.

What is Semantic Segmentation?

Semantic Segmentation is labeling each pixel of an image with a corresponding class of what is being represented, which means that semantic segmentation can be thought of as a classification of images at a pixel level.

Example of Semantic Segmentation

Therefore, it is a dense prediction which predicts every pixel in the image.

Unlike instance segmentation, semantic segmentation does not separate instances of the same class.

For example, if an input image has two objects in the same class, the segmentation map does not inherently distinguish these as separate objects. 

How does Semantic Segmentation work?

Semantic Segmentation follows three steps:

Classifying: Classifying a certain object in the image. (Classification)

Localizing: Finding the object and drawing a bounding box around it. (Object Detection)

Segmentation: Grouping the pixels in a localized image by creating a segmentation mask. (Finally Segmentation)

Semantic segmentation generates a segmentation map where the pixel value (from 0 to 255) of the input image is transformed into a class label value (0, 1, 2, … n).

Semantic Segmentation often requires the extraction of features and representations, which can derive meaningful correlation of the input image, essentially removing the noise.

The convolutional neural network (CNN) extracts features and representations, which can derive a meaningful correlation of the input image and remove the noise.

Mostly, Segmentation models, have CNN in the front that consists of a convolutional layer, a pooling layer, and a non-linear activation function.

For the Loss function, the cross-entropy loss is widely used.

Each pixel of the output of the network is compared with the corresponding pixel in the ground truth segmentation image.

Problem:

One problem with using Segmentation in real life is that it is slower than object detection or image classification.

References:

https://www.jeremyjordan.me/semantic-segmentation/

https://www.v7labs.com/blog/semantic-segmentation-guide

https://divamgupta.com/image-segmentation/2019/06/06/deep-learning-semantic-segmentation-keras.html