Whitepaper: Enabling the next generation of smarter self-navigating robots using Hybrid ToF

Download the full Hybrid ToF Whitepaper

Older robot vacuum cleaners used a random, inefficient, and slow cleaning pattern. Smart robots generate a map of their environment using sensors and simultaneous localization and mapping (SLAM) to localize, providing users with a floorplan from which areas can be selected for cleaning, or restricted from access. This whitepaper describes designing slimmer robots with new features using an approach with novel time-of-flight cameras.

Visual-SLAM (vSLAM) uses>1 cameras and requires high computational power to extract depth from the 2D images captured. SLAM uses depth-sensor time-of-flight (ToF) cameras providing true hi-res 3D images. Depth cameras have leaner and more efficient SLAM implementations compatible with embedded processing platforms for applications such as cleaning robots, warehouse robots, tracking drones, etc.

REAL3™ hybrid Time-of-Flight (hToF) combines a single ToF image sensor with two illumination types addressing:

  • Provide precise long-distance 3D point-cloud data enabling SLAM
  • Depth data with a high lateral resolution for advanced obstacle avoidance
  • Accurate close-range data with high resolution to allow cliff detection

Infineon, pmd and partners developed hToF as a cost-efficient depth sensing technology for consumer products. Particularly for consumer robotics, smaller hToF sensors help design slimmer robots than laser-distance-sensor (LDS) tower robots. This whitepaper implements hToF sensors for consumer robots, presenting excellent map accuracies achieved with open-source SLAM implementations, as well as the reduced processing resources needed.

The unique hToF technology uses Infineon’s and pmd's high-resolution REAL3™ ToF sensor, a homogenous flood illumination, and a powerful spot grid illumination offering a cost-efficient solution. The hToF camera’s broad, 110° horizontal field-of-view (FoV) is ideal for both SLAM and obstacle avoidance. hToF works in all lighting conditions (complete darkness to bright sunlight) across a large variety of furniture and floor reflectivity and textures. This is a huge advantage over structured light (SL) that struggles in bright sunlight, and also over stereo vision (SV) that struggles in darkness and with repetitive textures. Older robots were routinely immobilized under low-clearance furniture objects. Smart robots with hToF technology navigate better even under lower-clearance furniture, thanks to the reduced height and also the additional height-clearance information from the hToF sensor.

In an evaluation study, an R&D robot with two hToF cameras has been used in various office and home environments. The hTof sensor solution enables the robot to generate precise and consistent maps with excellent accuracy even for challenging environments with glass walls, non-rectangular rooms, dark floor material, and cluttered furniture objects. 3D depth image data by hToF handles glass walls properly, as the frame is detected as a wall.

The computational effort for hToF data’s depth processing using pmd’s library, and SLAM usingthe open-source Google Cartographer implementation, is benchmarked on three embedded SoC platforms used in robotics: NVIDIA Jetson Nano, Qualcomm RB5, and Raspberry Pi 3B/4B. We show that depth processing for hToF and corresponding SLAM algorithms can be executed on a single A53 (Raspberry 3B), in stark contrast to vSLAM systems that usually need >2 cores for their processing due to the high computational load ofthe required correspondence search.

In summary, the hToF technology with Infineon’s REAL3™ sensors along with pmd’s depth processing offer a powerful solution for consumer robots simultaneously supporting SLAM, advanced obstacle avoidance, and cliff detection. All this while reducing overall BOM cost, compute demand, and system complexity. The maps generated by open-source SLAM algorithms are accurate and reliable and prove the fidelity ofthe hToF spot-illuminated depth data. The solution is computationally lean, requiring only one core for depth processing and SLAM computation.


This might interest you.

#pmd News, #cases, #technology

isual sensing technology enables robots to observe, understand, and interact with their environment, much like humans. In a busy logistics warehouse, autonomous mobile robots (AMRs) swiftly transport goods, avoiding obstacles and calculating optimal paths.

#pmd News, #products, #technology

In collaboration with device manufacturer OMS and Infineon Technologies AG (FSE: IFX / OTCQX: IFNNY), we have developed a new high-resolution camera solution that enables enhanced depth sensing and 3D scene understanding for next-generation smart consumer robots. The new hybrid Time of Flight solution combines two depth sensing concepts and helps significantly reduce maintenance effort and costs for smart robots.

Group Photo

#3D News, #News, #pmd News, #tradeshows, #technology, #3D News, #News

At Hanover Messe 2023, German Chancellor Olaf Scholz and Indonesian President Joko Widodo visited the booth of ifm and pmd.

#pmd News, #tradeshows, #products, #technology

Our new i-ToF imager enables smallest 3D camera systems with improved quantum efficiency at optimized cost