Colloquium Cognitive Systems

Active Oberservers and Visual Intelligence

Prof. Dr. John Tsotsos (York University, Canada)

Abstract:  It would be easy to think that the AI community has made major progress in computer vision. After all, there are so many successes. However, this is too casual a conclusion; the successes have been in very narrow domains. In this presentation, I will show you three examples of human visuospatial problem solving that motivate us to look deeper. First, these examples demonstrate what we all understand: humans are active observers.  Humans have the ability to actively find the information they need to solve a task; they are not passive receivers of information. When we search for something, we do not wait motionless, eyes fixated on one spatial location, for our target to perhaps walk into our line of sight (i.e., as would a surveillance camera system, or the vision system of a barnacle). Second, through detailed experiments, we show how humans exhibit a variety of visual problem-solving strategies whose breadth and complexity are surprising and not easily handled by current methodologies. It is apparent that human solution methods are dynamically composed by hypothesizing sequences of actions, testing them, and if they fail, trying different ones. They are not learned by massive amount of training data. The presentation ends by speculating on novel methods for embodying these behaviors into artificial agents.

Bio: John Tsotsos is Distinguished Research Professor of Vision Science and Director of the Centre for Innovation in Computing at Lassonde at York University, with Adjunct Professorships in Computer Science and in Ophthalmology and Vision Sciences at the University of Toronto. He received his doctorate in Computer Science from the University of Toronto in 1980 developing the first computer system to interpret visual motion depicted in digital image sequences, with application to heart motion analysis. After a postdoctoral fellowship in Cardiology at Toronto General Hospital, he joined the University of Toronto on faculty in Computer Science and in Medicine. In 1980 he founded the highly respected Computer Vision Group at the University of Toronto, which he led for 20 years. He was recruited to move to York University in 2000 as Director of the Centre for Vision Research. Under his directorship, the centre was ranked in the top six interdisciplinary vision research organizations in the world. He has been a Canadian Heart Foundation Research Scholar (1981-83), a Fellow of the Canadian Institute for Advanced Research (1985-90), CP-Unitel Fellow of the Canadian Institute for Advanced Research (1990-95) and currently holds the NSERC Tier I Canada Research Chair in Computational Vision (2003-2024). He has received many awards and honours including 10 conference best paper distinctions, among them a 1987 inaugural Marr Prize citation, the 1997 CITO Innovation Award for Leadership in Product Development, the 2006 Canadian Image Processing and Pattern Recognition Society Award for Research Excellence and Service, was named Distinguished Research Professor of Vision Science at York University in 2008, the 1st President’s Research Excellence Award by York University on the occasion of the University’s 50th anniversary in 2009, the 2011 Geoffrey J. Burton Memorial Lectureship jointly from the United Kingdom's Applied Vision Association and the British Machine Vision Association for significant contribution to vision science, has been an ACM Distinguished Speaker, and is an IEEE Fellow. He was elected as Fellow of the Royal Society of Canada in 2010, in the Division of Mathematical and Physical Sciences of the Academy of Science, and was awarded its 2015 Sir John William Dawson Medal for sustained excellence in multidisciplinary research, the first computer scientist to be so honoured.  He was awarded a 2020  CS-Can|Info-Can Lifetime Achievement Award in Computer Science, the first computer vision researcher to receive this honour. He has co-founded or been a principal of 6 companies, holds a number of patents and technology transfer licences, and has consulted widely to industry in Canada and USA. Visiting positions were held at: University of Hamburg, Germany; Polytechnical University of Crete, Greece; Center for Advanced Studies at IBM Canada; INRIA Sophia-Antipolis, France; and, the Massachusetts Institute of Technology, USA. In his academic career, he has mentored over 180 trainees, many going on to faculty positions around the world, to major industries such as Google, IBM, Oracle, Apple, Amazon and more, and over a dozen starting their own companies. His current research focuses on a comprehensive theory of visual attention in humans. A practical outlet for this theory forms a second focus, embodying elements of the theory into the vision systems of mobile robots.