Rise of the Cobots

Robots are nothing new. Indeed, the first industrial robot was first developed some 70 years ago in 1954 for the automotive industry. The increasing adoption of robots promises vast potential benefits to productivity and efficiency not just in manufacturing but across a wide range of industries.

However, a major issue with robots is trust, and particularly when it comes to safety. The number of accidents involving robots is in fact relatively small when compared to other types of industrial equipment. In the USA, there were 41 fatalities recorded in the 25 years from 1992-2017, with the vast majority of incidents occurring while the robot was being maintained, rather than in everyday use. Each of these incidents is of course a tragedy, and every step must be taken to prevent them wherever possible. Even so, the breathless media coverage given to each incident, coupled with a wider wariness among the general public of the growing influence of machines and robots, contributes to a narrative that robots are fundamentally unsafe.

The use of robots undoubtedly introduces a new hazard into any workplace. However, thanks to recent innovations in robotic technology, which in turn build on decades of progress in the field, today’s robots are safer than ever. Indeed, collaborative robots, or “cobots”, are now increasingly gaining popularity across a wide range of industries and applications. These are robots that are designed specifically to work with and/or in the presence of human workers. This greatly widens the possible applications in which robots can be deployed, while also improving efficiency and productivity across the whole production line.

Given the wider perceptions of robots and their safety, the widespread adoption of cobots is an extremely high stakes endeavour for any robot or cobot manufacturer, who naturally want to prevent any incident that potentially feeds into this narrative. This is why vast amounts of resource, research and testing have been dedicated to ensuring that any robot operating in the vicinity of a human is as safe as it can possibly be.

Technologies for Object Detection

Effective object detection is crucial for the safe operation of cobots. There are currently two major sensing technologies for detecting objects and humans in an environment: ultrasonic, and optical based systems. Ultrasonics, although useful in some applications, have several drawbacks, namely a limited range and accuracy that is affected by soft materials.

Optical based systems are often based on LiDAR – Light Detection And Ranging or Laser Detection And Ranging. This can come in two forms, either 2D or 3D. 2D LiDAR suffers from the drawback that, because it uses only a single beam of light bounced off a single surface, it can miss certain objects that are out of the scannable area of the laser beam. For example, 2D LiDAR can potentially miss boxes or other items on the floor as well as possible hazards such as steps. This can be especially dangerous for mobile robots. For instance, if the robot is mounted on an Automated Guided Vehicle (AGV) then it will potentially need to detect and comprehend moving targets when in motion itself, whilst also avoiding any obstacles.

In contrast, 3D LiDAR offers a greater detection envelope by using multiple beams of light simultaneously to create a 3D image of the surrounding area. While it is generally considered the superior technology compared to 2D LiDAR with a greater accuracy and detection distance, it is also typically much more expensive. The additional sensing equipment required for 3D LiDAR can also mean that machines must be larger, bulkier, and less portable.

Time-of-Flight

An emerging optical method that mitigates many of these drawbacks is the time-of-flight camera (ToF camera), also known as a time-of-flight sensor (ToF sensor). This is a range imaging camera system that measures the distance to each point of the image based on time-of-flight: the round trip time of an artificial light signal. As scannerless systems, ToF devices can capture an entire scene with a single pulse of light from a laser or an LED. Capturing multiple scenes continuously can enable the machine to understand the nature of the objects around it; what the objects are, whether the objects are stationary are moving, and the direction and speed at which they are travelling.

ToF systems are typically much more compact compared to 3D LiDAR, with the illuminating light or laser placed close to the detection lens. The alternative would be to use stereo vision systems, however these need a certain minimum base line to function effectively. Another advantage compared to scanning systems is that no mechanical moving parts are required.

The amount of computing power required is also minimised, with only simple algorithms required to extract the distance information from the output signals of the ToF sensor. Stereo vision, in contrast, requires the implementation of complex correlation algorithms for a cobot to build an accurate picture of the world around it. 

Click image to enlarge

Figure 2 – Cobots move quickly and are very flexible, so effective object detection is crucial for their safe operation

ToF cameras can measure the distances within a complete scene with a single shot, turning captured images into 3D images. With speeds of up to 20 frames per second, they are ideally suited for use in real-time applications and tracking the movements of objects in three dimensions.

ToF sensors can also outperform 2D and 3D cameras in situations of poor illumination, or when for instance there are issues with the colours. 3D cameras can find it difficult to distinguish objects if the colours are very similar, or in particularly bright environments. White objects can pose additional problems, as white contains all wavelengths. Indeed, for a white coloured object on a white background, a conventional camera may struggle to accurately identify the edges and shape of objects, particularly if there are several similarly sized and coloured objects within the same scene. Because it uses artificial light signals, ToF sensors are less affected by ambient light.

Superior Sensing

Omron’s B5L ToF sensor illustrates how the technology can be successfully deployed. It utilises a ToF system to provide real-time 3D sensing of the distance to humans or objects with a resolution of 320x240 pixels. The B5L offers an ambient light immunity equivalent to 100,000 lux. This powerful ambient light immunity ensures a stable detection performance that is free from saturation even in bright environments.

Designed for a measurement distance of between 0.5 and 4m, the B5L has a detection resolution of 0.3 degrees, and a detection accuracy of +/-2%, equivalent to 4cm or less at 2m from the object, with a repeating accuracy of only 1%. The device also outputs compensated signals to minimise the need for processing by the robot’s computer. Omron’s proprietary circuit design, along with an innovative heat emission design, and the adoption of LEDs for the emission elements, ensure an expected life of five years.

Click image to enlarge

Figure 3 - Omron’s B5L ToF sensor utilises a ToF system to provide real-time 3D sensing of the distance to humans or objects with a resolution of 320x240 pixels­

The optical design technology of the B5L also contributes to the accuracy and quality of the image detection and distance measuring. The lens used is designed to correspond to the wavelength of the emitter LEDs, while the arrangement of emitters and receivers minimises the effect of suspended particles of dust. The B5L also incorporates interference prevention. This allows up to 17 B5L units to be used simultaneously without interfering with one another.

Skeleton Detection

Detecting objects is one thing, but detecting humans requires an additional layer of sophistication. Any mistake or oversight here can potentially result in tragedy, and so any technology used must be completely effective and reliable. To achieve this, the B5L incorporates skeleton detection software, which uses AI to recognise patterns of human movement, and from this infer precisely what the human is doing.

Skeleton detection is not purely a safety feature, and as such should be combined with additional safety functionality. However, anything that can enhance the robot’s understanding of its surroundings can contribute to improved safety, for instance by recognising when a human worker is moving towards a cobot or away from it. Additional machine learning can allow the skeleton detection function to learn the typical actions and patterns of the people it works with, allowing it to further optimise the timing and speed of its movements, for instance by slowing down as a human approaches.

Summary

The more a robot can detect of the world around it, the more effectively it can collaborate. Cobots offer vast potential to improve the efficiency of industrial processes, but this cannot come at the expense of safety. New innovations in the field of cobot safety, and specifically in object detection and human detection, are helping to make these productivity gains more accessible and cost-effective, while also opening up new and innovative applications for their deployment.

Omron Electronic Components Europe B.V.