Colloquium Cognitive Systems

Neurodynamic models of visual action recognition and
its interaction with action execution

Prof. Dr. Martin A. Giese,
Section Computational Sensomotorics, CIN & HIH, Department of Cognitive Neurology, University Clinic Tübingen

Abstract. Action perception and execution are intrinsically linked in the human brain. Consequently, visual action processing involves a whole spectrum of cortical functions, ranging from the visual processing of shape and motion, spatio-temporal relationships and semantic aspects of actions, to the interaction of visual representations with motor programs. The talk presents a neural theory that has been developed in close connection to neural and behavioral data. It provides a unifying account for a variety of experimental observation on visual action recognition and its interaction with motor execution. The core of the model is a physiologically-inspired neural hierarchy (‘deep architecture’) that mimics properties of neurons in the visual pathway and associated motor areas. The framework embeds neural field models for the representation of temporal sequences, response selection, and the implementation of flexible couplings between different representations. For the processing of goal-directed actions, the basic hierarchical model has to be extended by special mechanisms for the processing of spatial relationships between effectors and objects. In order to account for the interaction between action perception and motor execution the theory includes also neural representations for motor programs, which dynamically interact with visual representations. It is shown that such models account in a unifying manner for experimental results obtained with a variety of different methods, including single-cell physiology, behavioral studies and fMRI experiments. In addition, the theory helped to predict neural mechanisms at the single cell level that seem to be involved in the visual perception of causality. 

Acknowledgements: Supported by EC Fp7-PEOPLE-2011-ITN PITN-GA-011-290011 (ABC), FP7-ICT-2013-FET-F/604102 (HBP), BMBF, FKZ: 01GQ1002A, DFG GI 305/4-1 + KA 1258/15-1, and HFSP RGP0036/2016.

Bio. Martin A. Giese (PhD in Electrical Engineering) was a postdoc at CBCL (M.I.T.), and then headed the HONDA Cambridge Research Laboratory. In 2001, he founded the Laboratory for Action Representation and Learning at the Hertie Institute for Clinical Brain Research in Tübingen (Germany).

In 2007 he became Senior Lecturer at the Department of Psychology at the University of Bangor (UK), and since 2008 he is Professor for Computational Sensomotorics at the Hertie Institute and the Centre for Integrative Neuroscience at the University Clinic Tübingen, Germany. His research addresses the role of learning in action recognition and control, underlying neural mechanisms, and related biomedical applications in neurology.