In our previous Support item we described the preliminary steps for training a deep learning (DL) classifier in HALCON. In this article, we focus on the consequent training steps and convenience procedures to analyse the classifier.
As a prerequisite for training a DL classifier, the preprocessed images need to be divided into three subsets. The largest subset (approx. 60-75% of the images) is used for the training itself, i.e., the optimisation of the classifier. The validation set (typically 10-20% of the images) is used to evaluate the classifier during and after training. The remaining images form the test dataset, which should only be used once the training shows satisfactory results (see below). In order to obtain meaningful results, the subsets should represent all conditions but should also be as independent of each other as possible. E.g., images of the same physical object should all be assigned to the same subset. If all images are independent of each other the procedure split_dl_classifier_data_set() can be used for random splitting.
To allow optimal customisation, the training process is not encapsulated in an operator. Instead, the user is advised to base his code on the HDevelop example ‘classify_pill_defects_deep_learning’.
Before starting a training, the so-called hyperparameters, i.e., the parameters that guide the training progress, need to be set using set_dl_classifier_param(). For an initial try, we recommend to use the default values, apart from the parameter ‘batch_size’, which should be chosen as large as possible depending on the GPU memory.
During training, the classification errors on the training and the validation sets are evaluated and plotted. This plot reflects the progress of the training and should always be analysed to see whether hyperparameters need to be adapted. A successful training progression is shown in Fig. 1.
Fig. 1: Training progress plot
Throughout the training, the best classifier is determined based on the calculated validation errors and saved. After the training, this classifier should be applied again to the full validation dataset and analysed thoroughly. HALCON offers several procedures to assist you with this task (see Fig. 2):
evaluate_dl_classifier() calculates measures such as the classification error.
gen_confusion_matrix() returns and displays the confusion matrix, which can help to identify classes that were not learned well.
get_dl_classifier_image_results() retrieves, e.g., erroneously classified images.
dev_display_dl_classifier_heatmap(), in previous versions gen_dl_classifier_heatmap(), displays the regions of a given image that influence the confidence of its class assignment the most.
If any shortcomings are identified, modification of the image preprocessing, adjustment of the hyperparameters and image augmentation during training, facilitated by augment_images(), should be considered.
Fig. 2: Analysis examples of a DL classifier using HALCON procedures: Confusion matrix, incorrectly classified image and heatmap.
Once a classifier that performs well on the validation data has been trained, it should be evaluated on the test data to examine the generalisability of the classifier to unknown data. Then, your classifier is ready to be used for inference (classification) in your application.