At the Institute of Measurement, Control and Microtechnology, methods of machine learning are used, among other things, for situational analysis, classification and situational prediction. In this way, patterns and relations are found from observations of reality using statistical learning methods.

Precise sensory recording of the vehicle's surroundings is essential for automated driving. Camera data as well as measurements of lidar and radar data are used for this purpose. Machine learning methods now dominate signal processing, as the recognition performance is significantly higher than the deterministic approaches used to date. The neural networks used differ depending on the data. While for images so-called Convolutional Neural Networks (CNNs) dominate, for lidar and radar data PointNets have established themselves.  In the meantime, however, numerous derivatives of these are being discussed and further researched. A method suitable for the classification of image data is the so-called Scene Labeling or Pixel Classification method. Here, an image is divided into object classes (typically 10-20), which are used to understand the scene. Typical classes that are distinguished are road, footpath, lane markings, traffic lights, pedestrians, vehicles, bicycles and some more. The video on the right shows an example from a trip of our test vehicle through Ulm. Each pixel in the input image is assigned a probability of the respective class using a trained CNN, which is signaled by the different class-specific coloring. The network was trained using manually labeled images. If the pixels of the same class are now combined into objects, this is called panoptic segmentation.

The prediction of the situation aims at the prediction of a future scene development. Since the future development can have a variety of manifestations, the prediction result is rather uncertain. The possible scene evolution, based on a known past, is therefore determined probabilistically.
An example of a method investigated at the institute is the Variational Bayesian Bernoulli Gaussian Mixture Model (VBBGMM). The method is able to learn the classification procedure independently with the help of training data. The gained probability model allows the integration of a priori knowledge. This enables the model to be adapted at a later stage by the integration of additional training data without a consistent storage of the previous training data.