Why should you implement 3D into your Automation?
Automation is becoming more widely accepted with more complex applications targeted in many industry areas. With tasks becoming more advanced and the increased use of robotics, 3D vision is becoming more important. 3D vision technology has improved significantly in recent years and is now able to overcome previous limitations.
How does 3D vision improve automation?
BECOM ToF, Photoneo PhoXi, SmartRay ECCO
There are many different 3D techniques and they all use different types of 3D camera technology. These techniques include Stereo Vision, Laser Triangulation, Time of Flight (ToF), Structured Light and most recently Parallel Structured Light. All these technologies capture the 3D data differently and have different strengths/advantages. The combination of these methods is pushing automation forward by making it increasingly applicable to a broad range of scenarios.
HALCON Progress
Real-world environments do not typically fit into neat parameters. There will always be varying textures, non-uniform object heights, object type variation and other elements that rarely remain constant. 3D vision accommodates these variations surprisingly well compared to previous methods and often enables robots & machinery to adapt to random environments and become more adept at new tasks.
In Automation, it is common to work with several robotic technologies simultaneously, to get work done faster with fewer errors. Using Machine Vision (MV) technology in combination with the robots copes with differences in the environment and improves safety overall. Many industry-leading companies now have fully automated production and inspection lines, the use of vision on these lines eliminates human error, frees up time and saves on costs.
Data from 3D cameras can be processed in different ways. Image processing libraries offer a choice of tools to allow the incoming data to be interpreted in different ways to produce the result. For example, if 3D vision is being used on an Autonomous Mobile Robot (AMR), the data is processed in a different way to a fixed setup on an inspection line. Different applications require different tool-sets for image processing, 3D Matching, Surface inspection, object processing, and reconstruction are examples of methods used.
Common robot applications deployed using 3D vision
Not all robotic applications use vision, it depends on the application and what is required. Vision is a necessity if the robot needs ‘eyes’, to perform tasks such as ‘pick and place’ or ‘bin picking’. As mentioned above, vision also offers a level of safety for collaborative work when robots are working alongside humans, to avoid collisions with operators. Vision also drastically reduces the need for operator oversight and intervention. This often enables operators to manage multiple production processes increasing throughput and efficiency.
Common places you will find robotic applications using vision include manufacturing, agriculture, food and beverage, and logistics. Some use cases of applications where 3D vision and automation are used together are shown below.
Bin Picking, Pick and Place, Palletizing
Manufacturing
3D vision is widely used in manufacturing, covering a wide variety of application types. A typical use cases for 3D would be to inspect objects and match them against a CAD model, other cases require robots to assemble or place objects as part of a line process. 3D provides the information needed to correctly guide the robot.
Below: The Photoneo PhoXi 3D scanner and Bin Picking Studio are automating the assembly of spikes on shoes.
Medical
Vision is used in multiple areas in the medical industry including operating theatres and patient monitoring. 3D reconstruction and analysis of chronic wounds is one of the latest use cases.
Chronic wounds heal very slowly, and the healing process can be further prolonged if treated incorrectly. Scans/images from multiple viewpoints are taken of the wounds and the data is reconstructed into a 3D model, enabling the healing process to be monitored very accurately and allowing treatment to be revised if ineffective.
3D Scan of Chronic Wound
Food and Beverage
‘Pick and place’ applications use robots combined with 3D vision. The example below shows how 3D vision can be used for bin picking things as random and non-uniform as croissants!
Not only are all croissants slightly different shapes, sizes and colours, they are produced in batches with random orientation and large variability. They are non-geometric shapes and this offers huge challenges for robot picking. This application requires a 3D ‘Deep Learning’ classifier, to accommodate the variation whist still being able to localize the product in all orientations and pass those coordinates to the robot.
Logistics
Palletizing and depalletizing is another area where 3D vision is commonly used. 3D vision systems use imaging cameras to locate/calculate the coordinates and dimensions of the boxes that are being palletized or depalletized. Some systems like the one below, also calculate a collision-free trajectory path for the robot.
Automation challenges
Cobot handling Computer Components
2D vision systems cannot provide depth information and are only suitable for simpler robotic/ vision tasks such as barcode reading, OCR/OCV, blob analysis, verification, and so on.
As challenges in automation get more complex, traditional vision approaches with standard area scan and smart cameras are sometimes no longer enough. 3D technologies remain on the edge of what is technologically possible, as advances are made, more application areas are opening up.