A important application goal of the EU-FP7 project SEARISE was the detection of security-relevant events in large crowds. The detection of such events is a difficult vision problem.
We investigated differences in visual search of dangerous events between security experts and naive observers during the observation of large scenes, typically encountered on the grandstand of stadiums during soccer matches. To overcome the scarcity and legal issues associcated with real footage, we designed a new algorithm for the synthesis of crowd scenes with well-controlled statistical properties, which we call the "Tübingen hooligan simulator". Subjects were eye-tracked during the observation of the synthesized scenes. Detection rates, fixation rates and times were assessed from 20 naive subjects and an expert observer.

We characterize the relative importance of saliency and expert knowledge for the generation of correct detections and the visual search strategies for both types of observers.

We found that during the first few seconds of this search task, experts and naive observers look at the scenes in a similar fashion, but experts see more. This suggests that the fixation behavior of both observers types is driven by (low-level) saliency, whereas event classification performance is strongly influenced by expert training.

We compare the results with theoretical models for saliency and event classification:

   1. an approach for saliency computation based on low-level features (Bruce and Tsotsos, 2009),
   2. a non-parametric graphical Bayesian recognition model that was trained with expert knowledge derived from scenes containing security-relevant events, exploiting optic flow features extracted with a neurally plausible algorithm (Beck et al, 2007) and the Bayesian optic flow from (Simoncelli 1998).

We show that the recognition model can deliver reasonable classification/detection performance even when operating under real-time constraints. When real-time performance is not a concern, performance can be improved further by allowing the model to grow.