A7-[FreePerspective] - Details

Real-time Aggregation of Spatial Perceptions into a Renderable 3D Representation

 

With this project we want to develop methods for perceiving space with a range-camera (a so-called "time-of-flight camera"). Time-of-flight cameras send out modulated light into the scene and measure the phase and amplitude of the reflected light. In principle, range-cameras are ideal sensors for this task, measuring the distance (which directly corresponds to the phase) to the next object in every pixel. However, these measurements are heavily perturbed by various strange disturbances, caused by multi-path reflections and lens-scattering. The difficulty lies in interpreting the data despite these flaws.

We will model those disturbances probabilistically and formulate the problem as estimation of the most likely map, given the sensor-data. A well chosen prior will enforce smooth maps unless there is contrary evidence and thereby prevent artifacts at the brink of the camera range. An estimation algorithm will be designed that computes the most likely map based on this model. The underlying map representation is a voxel-grid with a Gaussian in each cell. The Gaussian can model different phenomena with covariance matrices of different rank. These range from wires (rank 1), over a (roughly) planar surface patch (rank 2), to spurious reflections or irregular geometry (rank 3). This makes it an ideal continuous representation, suited for maximization algorithms, such as relaxation. This approach also overcomes the independence assumption commonly underlying grid-maps, because it optimizes over all past observations.

The application we address is urban search and rescue (USAR), where an actuated endoscope ("snake robot") searches for entombed victims in a collapsed building. Casper and Murphy analyzed robot-supported rescue missions in the World Trade Center. They reported that operators have enormous difficulties understanding the images of an on-board camera. We will support the operator with a 3D-map, viewed from arbitrary perspectives, such as the "Guardian Angel" perspective above/behind the camera.

We take the USAR application serious, beyond a loose motivation. We will therefore collect requirements and environment characteristics from USAR professionals (THW), base our algorithmic developments on these findings, and evaluate the usability of our system in a realistic mockup environment.