Learning maps of the environment with mobile robots has been studied intensively in the past. Most of the approaches presented so far, however, are purely based on the perception of the three-dimensional structure. In this phase of the SFB/TR8, we study the problem of learning 3D representations from a new perspective. The key idea is to learn such representations by the interpretation of human activity. We observe the individual activities of a human user that is wearing an XSens Moven data suit. This device consists of various sensors attached to a suit that return their positions in space and allow to reconstruct the pose and movement of the user. With this data, we develop a system that is able to generate a virtual environment of typical indoor environments with multiple floors and containing several relevant objects including doors, walls, tables, and chairs.

We investigate the problem of recognizing activities from high-dimensional data suit information to associate activities or gestures with objects to be reconstructed. On the one hand side, we consider recognized gestures and a known mapping from activities to objects, for instance 'sitting down' could be associated with a chair, and other reconstruction operations such as constraint insertion for walls or tables. On the other hand, we implicitly reconstruct features of the environment from natural human activities, for instance we can insert doors when recognizing door-handling activities and staircases when detecting stair climbing activities.

To represent the virtual environment, we investigate appropriate data structures for the reconstructed objects. These structures need to provide efficient insertion and modification operations. Moreover, as we include the user into the virtual environment, they provide appropriate techniques for interaction with the user, including collision detection, for example. To be efficient in terms of memory, we investigate approaches to detect symmetries and similarities in reconstructed objects. We furthermore develop techniques to detect similar features of partially reconstructed objects and the objects learned thus far to allow for an effective completion.

Additionally, we keep track of the human in the reconstructed environment to be able to check whether the current activity of the human is consistent with the reconstruction and to perform update operations in the case of inconsistencies. We will furthermore investigate approaches allowing the human to directly modify the scene by manually grabbing, replacing, and reshaping objects.