Part 5: Imaging Processing Software

A machine vision, or computer vision solution is heavily reliant on software that is used to process and inspect images, document and store data, as well as provide feedback to PLC’s for machine system control.

When developing a vision solution there are three main software categories to consider. Which approach you use will depend on the vision inspection requirements and the skill set of the developer. Other important factors to consider are timescales and budget for the solution.

Here we are focusing on interfacing to a camera and inspecting those images. System software for controlling devices such as PLC’s and robots etc, all have similar software choices available but are not referred to here.


Camera/Frame grabber Software Development Kit’s (SDK’s)

These are a library of functions for interacting, configuring and grabbing images from the camera (or frame grabber). They are essentially vendor specific although with standards like GenICam there is a set of common features and naming conventions which makes it easier to create a software application that supports multi-vendors.

It requires the developer to have software skills in a programming language and be familiar with likes of Microsoft Visual Studio.

The best SDK’s offer easy installation including device drivers and are also fully documented. In the best case, just a few lines of code are all that’s needed to configure the camera, capture images and save them to the PC’s hard drive. Extensive documentation and a wealth of sample programs for all typical camera applications in all supported programming languages (C, C++, C#, VB.Net, etc) provides a critical boost into application development. Basler’s Pylon is a great example of this, offering a modern comprehensive software suite that includes a camera viewer for live image capture and camera configuration. (Fig.1)

One important note – these camera/frame grabber SDK’s do not include image processing operators that are used to inspect the image. They are therefore the ideal choice for applications that only require image visualisation, notation and/or storage for traceability or documentation purposes.

Vendor SDK’s are usual free of charge and do not require the purchase of a run-time-licence.


All-in-one (point ‘n’ click) image processing software

Ideal choice for creating vision software solutions without the need to have software programming skills. They are also perfect for situations that are under strict time constraints where time-to-market is paramount.

These products are able to accomplish this by providing an extensive graphical user interface for point ‘n’ click configuration of the camera interface and the processing tools. Many also provide the ability to output inspection results via COM/Ethernet etc.

Once the inspection ‘tasks’ have been designed you then enter an execution mode – without the need of development environments such as Microsoft Visual Studio (C++,C# etc). In execution mode an end user graphical interface is presented for both application control and image visualisation – again without any coding being required. Time-to-market is optimised.

All-in-one products tend to have a heavy visual presence which means it is easier to understand the vision task created than it is say reading native C++,C# etc. This has the added benefit of future proofing the solution where those other than the original creator of the task will be able to understand the flow of the image processing solution and amend if necessary. This is not so straightforward when a solution has been written using programming languages.

MVTec’s MERLIC is a prime example of all-in-one software, created to build a machine vision application without programming a single line of code (Fig.2). You can simply step through the application and rely on the many standard machine vision tools for creating the solution. MVTec have taken this even one step further with an outstanding, innovative, and unique feature in MERLIC, the easyTouch concept (Fig.3). It helps to guide the user interactively towards the solution. For example, simply hover the mouse pointer over an image and easyTouch recognises and marks objects to be identified.

From a cost point of view, all-in-one software typically requires the purchase of a full licence for every system deployed.


Image Processing Libraries

This is a set of image processing operators or functions that requires the developer to have good software skills in programming languages as well as an understanding of how to design software solutions from the ground up. The most successful developers also have excellent knowledge of image processing terminology, techniques and algorithms. It is clear how this differs from the all-in-one approach mentioned previously.

There are open source libraries available as well as fully supported and documented libraries such as Euresys Open eVision and MVTec’s HALCON. While open source may be free of charge, there is no official documentation or support provided and no control over version release. This can be a significant downside for customers that require a known manageable state of the system software. It’s a cost trade off that needs to be taken on the merits of the customer’s situation.

Some libraries also have an application that can run to allow the developer to try functions before committing to code. This is very useful when a library can have several operators offering similar processing techniques and the developer wants to quickly try them as well as tweaking the parameters. Once happy, the developer then takes that function and writes it in native code. Hence, the cycle continues until a solution is created at which point the developer also must ensure an appropriate graphical user interface design for the end customer to use.

MVTec’s HALCON goes one step further and has an integrated development environment (HDevelop) that is used worldwide. It enables cost savings and improved time to market. (Fig.4)

Aside from open source, there is usually a one off cost for the initial development licence and then ongoing costs (fraction of initial development cost) for the run-time-licences.

Part 4: 3D Imaging