In HALCON 18.05, deep learning classification has been extended in the way that
- the image size can be set by the user and
- the deep learning inference can also be performed on CPUs.
Using both features, one can achieve a deep-learning-based pixel segmentation. The idea is to crop small images of the same size around the pixels of a larger input image and to feed these so-called patches into a trained deep learning network. The class assignments to the corresponding pixels make up the respective class regions.
To provide the deep learning network with training data, an appropriate amount of labeled patches (at least 500) are then selected at random positions inside each class region.
Fig. 1: Training patches
The patch size should be chosen such that the classes can visually be distinguished. Larger sizes give a better separation, smaller sizes are processed faster. For training the deep learning network, the classifier needs to be set to the patch size via
set_dl_classifier_param (DLClassifierHandle, ‘image_dimensions’, \
[PatchSize, PatchSize, NumChannels])
and the retraining of the deep learning network can be done as usual.
The segmentation is achieved by cropping a patch around each pixel of the input image. Each patch is then classified by the inference of the trained deep learning network. All classification results sum up to the class regions.
Fig. 2: Segmentation
While the deep learning training needs to be processed on a compatible NVIDIA GPU, one can perform the segmentation using the deep learning inference on CPU via
set_dl_classifier_param (DLClassifierHandle, ‘runtime’, ‘cpu’)
Processing every pixel on large images can make the segmentation quite slow. By subsampling, however, one can make the pixel segmentation considerably faster at moderate loss of resolution. Each subsampling step will reduce the processing time in quadratic order.
Fig. 3: Segmentation result N = 4
Fig. 4: Segmentation result N = 16
A simple program that shows the method can be downloaded here.