Number of cases campus Koblenz: 1 (warning level Gelb until 02.12.2020) Action plan

Optimisation of environment perception and advanced autonomy-functions

Optimisation of environment perception and advanced autonomy-functions
Period: Jul 01, 2018 - Nov 30, 2020
Status: laufend

The main goal of this study is to enhance the environment perception in the field of outdoor robotics.

The interaction of a robot with its environment is a challenging task which requires deep knowledge of the robots surrounding. In this project it is assumed that the robot operates in an unstructured outdoor environment.

This makes tasks such as path planning even more challenging as they are in structured environments such as indoors or in urban areas. Data from sensors such as 3D-LiDARs can be used to create geometrical maps of the environment but they lack the ability to capture the semantics and properties of the surrounding surfaces. This can result in slippage in sand, emergency stops in front of traversable objects like high grass or bushes or even in getting stuck in marshy areas.


To overcome those problems the active vision group researches how semantics can be extracted from hyperspectral image data using classical spectral analysis methods (see Fig. 1), but novel deep learning methods as well . Those are fused with data of LiDARs and other sensor modalities to create semantically enriched 3-dimensional environment maps. which allow precise planning of the robots interaction with its surroundings.



Figure 1: Semantic segmentation of hyperspectral images in rural and

urban  area  using  fully  convolutional  networks.  The left side shows

grayscale  representations  of  the  scenes. On the right side predicted 

labels are overlaid in different colors. 


Participating organizational units



Theisen, Nick; Paulus, Dietrich (2019): Verbessern der temporalen {K}ohärenz in semantisch segmentierten, multispektralen {B}ildsequenzen. In: Püschel, Frank; Stanke, Gerd; Pochanke, Michael: Tagungsband zum 24. Workshop Farbbildverarbeitung. Berlin, Germany: Gesellschaft zur Förderung angewandter Informatik e.V.. Bd. 24. S. 75-85.

Winkens, Christian; Paulus, Dietrich (2018): Context aware hyperspectral scene analysis. In: Electronic Imaging. Springfield, USA: Society for Imaging Science and Technology. S. 346.1--346.7.