Our visual system is capable of recognizing objects under different viewing angles despite drastically different retinal images, and it achieves a stable representation of visual space independent of eye movements. We investigated how such invariant representations could be learned based on the spatio-temporal statistics of visual inputs under natural viewing conditions. Our approach combines temporal correlation-based learning with the concept of self-organizing maps in a recurrent network of spiking neurons with realistic conductances and Hebbian plasticity (Michler et al 2009, J Neurophysiol 102:953-964). When trained with stimulus sequences that mimicked the fixation of objects during self-motion, the network learned topographic maps of object views, similar to the topography of object-selective neurons found in the inferotemporal cortex. Neurons in the output layer of the network showed selectivities for objects invariant of viewing angle. With an extended input layer to include signals representing gaze-direction that combined nonlinearly with the retinal signals, the network achieved invariant representations of object positions. After training with stimuli that mimicked the exploration of visual scenes by saccadic eye movements, neurons in the network exhibited gain-field properties similar to neurons in parietal cortex. Neurons in the output layer showed selectivities for object positions in head-centered visual space, invariant to gaze direction. Our results suggest that invariant representations of visual objects and visual space can be learned in an unsupervised way using biologically plausible mechanisms, and that the stimulus statistics under natural viewing behavior may play a significant role in this process.