Patterns of thought, for a cognitive system that interprets a scene or performs a task, consist of perceptual, motor and linguistic representations of the world, controlled by an attention mechanism. The role of the attention mechanism is to facilitate parsing of the different representations according to a grammar of action. In this talk I will introduce, for the first time, an action grammar. It is really a disguised version of Chomsky’s Universal Grammar adapted in a minimalist framework. Parsing requires an attention mechanism comprising visual human and object filters, active segmentation routines and a context implementing system utilizing language. I will describe our developments in learning visual filters, achieving robust segmentation techniques and integrating vision with language and our implementations of these ideas in the field of Humanoid Robotics and the general area of video interpretation.