Introducing the new standard example two_camera_calibrated_mosaicking.hdev launched with HALCON 19.05
Sometimes one camera simply is not enough to cover the desired field of view (FOV) with the required resolution, for example if a wide production line needs to be monitored. In this case, one can employ multiple cameras side by side, thus increasing the FOV. Knowing the geometry, every camera image can be projected into the measurement “world” plane and the projected images of all cameras can be stitched together; this process is called calibrated mosaicing.
In our standard example two_camera_calibration.hdev, the calibration process is done with two rectangular calibration plates mounted on a rigid fastener. However, HALCON’s calibrated mosaicing is made easier with our calibration plates with hexagonally arranged marks (introduced with HALCON 12). Thus, we show the simplified workflow in the new standard example two_camera_calibrated_mosaicking.hdev in HALCON 19.05.
In the following paragraphs, we would like to showcase the most important steps of the new standard example. Let’s start with the calibration.
Calibration
For calibrated mosaicing it is essential to know the setup’s geometry, i.e., the relative pose between all cameras and the world coordinate system, which we use for measuring. Unlike with classical stereo setups, we only have small overlapping image parts of adjacent cameras. This makes the calibration quite difficult when using calibration plates which need to be fully visible in every image. However, with our calibration plates with hexagonally arranged marks, the only requirement is to have one of the five finder patterns visible in the image. This eases the calibration process significantly.
For the stereo calibration, the calibration plate is placed at various positions in the common FOV of two neighboring cameras. Note that with the HALCON calibration data model it is possible to do a simultaneous calibration of all internal and external camera parameters. This is done in one operator call: calibrate_cameras.
Mosaicing
After the calibration, the pose of each camera is known with respect to one common coordinate system. By using the operator image_to_world_plane, we can now map all images into this common world coordinate system. Here, we take some technical details into account:
- The pose of the world coordinate system usually corresponds to the first pose of the calibration plate. To make sure that all pixels in the final image have positive row and column values, the world coordinate system has to be translated accordingly in x and y direction. Furthermore, we correct the z value of the reference pose by the calibration plate thickness. Like this, our world plane corresponds to the surface of our table.
- We choose appropriate image dimensions for the target image. The resolution of the result image should be comparable to the original images.
- For mapping the images, we correct the camera poses by the height of the observed objects.
Now, the images of both cameras are mapped to the world plane and the mapped images are stitched together.
Stitched result image looks like this:
