Abstract

We investigated audiovisual interactions in motion perception by behavioral experiments testing both, the influence of visual stimuli on auditory apparent motion and the influence of auditory stimuli on visual apparent motion perception. A set of loudspeakers with an LED mounted in the middle of each speaker cone was arranged in a semicircle. Apparent motion streams were presented for each modality alone in the unimodal conditions. In the bimodal conditions, stimuli of the second modality were added to fill the temporal and spatial gaps of the sampled trajectory of the reference stream. The participants' task was to observe the quasi-naturalistic stimulus sequences and to perform a standard classification. The addition of stimuli of the second modality indeed facilitated apparent motion perception. Bimodal presentation increased the upper temporal interval up to which the stimuli could be separated in time while still being perceived as continuous motion. We interpret these results as evidence for an ecologically advantageous audiovisual motion integration mechanism which operates beyond the constraints of strict spatiotemporal coincidence. Functional considerations suggest that this mechanism may represent an amodal stage suited for the processing of both unimodal and bimodal signals.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

This paper derives a particle filter algorithm within the Dempster-Shafer framework. Particle filtering is a well-established Bayesian Monte Carlo technique for estimating the current state of a hidden Markov process using a fixed number of samples. When dealing with incomplete information or qualitative assessments of uncertainty, however, Dempster-Shafer models with their explicit representation of ignorance often turn out to be more appropriate than Bayesian models. The contribution of this paper is twofold. First, the Dempster-Shafer formalism is applied to the problem of maintaining a belief distribution over the state space of a hidden Markov process by deriving the corresponding recursive update equations, which turn out to be a strict generalization of Bayesian filtering. Second, it is shown how the solution of these equations can be efficiently approximated via particle filtering based on importance sampling, which makes the Dempster-Shafer approach tractable even for large state spaces. The performance of the resulting algorithm is compared to exact evidential as well as Bayesian inference.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

Based on the neurobiological and cognitive principles of human information processing, we develop a system for the automatic visual identification and exploration of scenes. The system architecture consists of three layers: a bottom-up feature extraction stage, a top-down object identification stage and knowledge from a domain ontology for scene analysis. The uncertainty in the latter two stages is managed by Dempster–Shafer belief measures. The system sequentially selects “informative” image regions, identifies the local structure in these regions, and uses this information for drawing efficient conclusions about an object in the scene. The selection process involves low-level, bottom-up processes for sensory feature extraction, and cognitive top-down processes for the generation of active motor commands that control the positioning of the sensors towards the most informative regions. Both processing levels have to deal with uncertain data, and have to take into account learned statistical knowledge. For bottom-up feature extraction this is achieved by integrating a nonlinear filtering stage modeled after the neural computations performed in the early stages of the visual system. The top-down cognitive reasoning strategy operates in an adaptive fashion on a belief distribution. The resulting object hypotheses in combination with knowledge from the domain ontology in the third layer are used for generating a scene hypothesis.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

We investigate the relation between the physical world and its mental representation in the ‘cognitive map,’ and test if this representation is image-like and complies with the laws of Euclidean geometry. We have developed a new experimental technique using ‘impossible’ virtual environments (VE) to directly influence the representational development. Subjects explore a number of VEs — some ’normal,’ others with severe violations of Euclidean metrics or planar topology. We check if these manipulated properties cause problems in navigation performance. A consistent VE should be easily represented mentally in a map-like fashion, while a VE with severe violations should prove difficult. Surprisingly, we found no substantial influence of the impossible VEs on navigation performance, and forced-choice tests showed little evidence that subjects were aware of manipulations. This suggests that the representation does not resemble a two-dimensional image-like map. Alternatives to consider are sensorimotor and graph-like representations.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

We investigate a hybrid system for autonomous exploration and navigation, and implement it in a virtual mobile agent, which operates in virtual spatial environments. The system is based on several distinguishing properties. The representation is not map-like, but based on sensorimotor features, i.e. on combinations of sensory features and motor actions. The system has a hybrid architecture, which integrates a bottom-up processing of sensorimotor features with a top-down, knowledge-based reasoning strategy. This strategy selects the optimal motor action in each step according to the principle of maximum information gain. Two sensorimotor levels with different behavioural granularity are implemented, a macro-level, which controls the movements of the agent in space, and a micro-level, which controls its eye movements. At each level, the same type of hybrid architecture and the same principle of information gain are used for sensorimotor control. The localisation performance of the system is tested with large sets of virtual rooms containing different mixtures of unique and non-unique objects. The results demonstrate that the system efficiently performs those exploratory motor actions that yield a maximum amount of information about the current environment. Localisation is typically achieved within a few steps. Furthermore, the computational complexity of the underlying computations is limited, and the system is robust with respect to minor variations in the spatial environments.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

We investigated nonlinear multi-stage networks optimised to reduce statistical dependences in natural images. These networks serve as models for the neural information processing in the higher visual areas of primates (visual cortices V2, V4). We analysed the resulting units with regard to nonlinear selectivity and invariance properties. We show that the proposed network principle leads to units that are highly selective with respect to the input signal space and to units that are invariant with respect to certain stimulus classes. The stimulus selectivity is tested by a set of Cartesian, hyperbolic, and polar gratings as used in several physiological experiments. We compared the population histogram of the stimulus selectivity indices of the network units with the population histogram of macaque V2 units and show that the network can achieve the high selectivity observed in many V2 neurons. We also tested for nonlinear frequency interaction effects, and found substantial interactions for frequencies with the same orientation and for frequencies with different orientations. The former enable a selective encoding of edges and the latter are required for the selective encoding of intrinsically two-dimensional features like corners and terminators

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

Linear filtering is a basic concept in neural models of early sensory information processing. In particular the visual system has been described to perform a wavelet-like multi-channel decomposition by a set of independent spatial-frequency selective filter mechanisms. Here we suggest that this principle of linear filtering deserves a critical re-evaluation. We propose that an optimal adaptation to natural scene statistics would require AND-like nonlinear interactions between the frequency-selective filter channels. We describe how this hypothesis can be tested by predicted violations of the principle of linearity that should be observable if cortical neurons would actually implement the proposed nonlinearities. We further explain why these effects might have been easily overlooked in earlier tests of the linearity of neurons in primary visual cortex.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

We investigated the extent to which auditory and visual motion signals are combined when observers are asked to predict the location of a moving target. In condition 1, the unimodal and bimodal signals were noisy but the target object was continuously visible and audible; in condition 2, the moving object was hidden (invisible and inaudible) for a short period prior to its arrival at the target location. Our main finding is that the facilitation due to simultaneous visual and auditory input is very different for the two conditions: when the target is continuously visible and audible (condition 1), the bimodal performance is twice as good as the unimodal performances thus suggesting a very effective integration mechanism. On the other hand, if the object is hidden for a short period (condition 2) and the task therefore requires the extrapolation of motion speed over a temporal and spatial period, the facilitation due to both sensory inputs is almost absent and the bimodal performance is limited by the visual performance.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

The neural mechanisms of early vision can be explained in terms of an information-theoretic optimization of the neural processing with respect to the statistical properties of the natural environment. Recent applications of this approach have been successful in the prediction of the linear filtering properties of ganglion cells and simple cells, but the relations between the environmental statistics and cortical nonlinearities, like those of end-stopped or complex cells, are not yet fully understood. Here we present extensions of our previous investigations of the exploitation of higher-order statistics by nonlinear neurons. We use multivariate wavelet statistics to demonstrate that a strictly linear processing would inevitably leave substantial statistical dependencies between the outputs of the units. We then consider how the basic nonlinearities of cortical neurons--gain control and ON/OFF half-wave rectification--can exploit these higher-order statistical dependencies. We first show that gain control provides an adaptation to the polar separability of the multivariate probability density function (PDF), and, together with an output nonlinearity, enables an overcomplete sparse coding. We then consider how the remaining higher-order dependencies between different units can be exploited by a combination of basic ON/OFF point nonlinearities and subsequent weighted linear combinations. We consider two statistical optimization schemes for the computation of the optimal weights: principal component analysis (PCA) and independent component analysis (ICA). Since the intermediate nonlinearities transform some of the higher-order dependencies into second-order dependencies even the basic PCA approach is able to exploit part of the redundancies. ICA ignores this second-order structure, but can exploit higher-order dependencies. Both schemes yield a variety of nonlinear units which comprise the typical nonlinear processing properties, such as end-stopping, side-stopping, complex-cell properties and extra-classical receptive field properties, but the 'ideal' complex cells seem only to occur with PCA. Thus, a combination of ON/OFF nonlinearities with an integrated PCA-ICA strategy seems necessary to exploit the statistical properties of natural images.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

The classical approach in vision research--the derivation of basically linear filter models from experiments with simple artificial test stimuli--is currently undergoing a major revision. Instead of trying to keep the dirty environment out of our clean labs we put it now right into the focus of scientific exploration. An increasing number of scientists are using natural images in their experimental work, and concepts from statistics and information theory are employed for the theoretical modeling of the results. The new approach has a close relation to basic engineering strategies for electronic image processing, since its major concept is that biological sensory systems exploit the statistical redundancies of the environment by appropriate neural transformations. The standard engineering methods are not sufficient, however. Even such a basic biological feature as orientation selectivity requires the consideration of higher-order statistics, like multivariate wavelet statistics, cumulants, or polyspectra. Furthermore, there exists an abundance of nonlinear phenomena in biological vision, for example the phase invariance of complex cells, cortical gain control, end-stopping, and a variety of extra-classical receptive field properties. These amount to nonlinear combinations of linear wavelet filter outputs, which are required to exploit higher-order statistical dependencies, and make it necessary to consider unconventional modeling approaches like differential geometry or Volterra-Wiener systems. By use of such methods we cannot only gain a deeper understanding of the adaptation of the visual system to the complex natural environment, but we can also make the biological system an inspiring source for the design of novel strategies in electronic image processing.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

Based on an information theoretical approach, we investigate feature selection processes in saccadic object and scene analysis. Saccadic eye movements of human observers are recorded for a variety of natural and artificial test images. These experimental data are used for a statistical evaluation of the fixated image regions. Analysis of second-order statistics indicates that regions with higher spatial variance have a higher probability to be fixated, but no significant differences beyond these variance effects could be found at the level of power spectra. By contrast, an investigation with higher-order statistics, as reflected in the bispectral density, yielded clear structural differences between the image regions selected by saccadic eye movements as opposed to regions selected by a random process. These results indicate that nonredundant, intrinsically two-dimensional image features like curved lines and edges, occlusions, isolated spots, etc. play an important role in the saccadic selection process which must be integrated with top-down knowledge to fully predict object and scene analysis by human observers.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

The inherent structure of the encoding in early stages of the visual system is investigated from a combined information-theoretical, psychophysical, and neurophysiological perspective. We argue that the classical modeling in terms of linear spatial filters is equivalent to the assumption of a Cartesian organization of the feature space of early vision. We show that such a linear Cartesian feature space would be suboptimal for the exploitation of the statistical redundancies of natural images since these have a radially separable probability-density function. Therefore a more efficient representation can be obtained by a nonlinear encoding that yields a feature space with polar organization. This prediction of the information-theoretical approach regarding the organization of the feature space of early vision is confirmed by our psychophysical measurements of basic discrimination capabilities for elementary Gabor patches, and the necessary nonlinear operations are shown to be closely related to cortical gain control and to the phase invariance of complex cells. Finally, we point out some striking similarities between the polar representation in visual cortex and basic image-coding strategies pursued in shape-gain vector quantization schemes.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

We suggest that intrinsic-2D (i2D) features, computationally defined as the outputs of nonlinear operators which model the activity of end-stopped neurons, play a role in preattentive texture discrimination. We first show that for discriminable textures with identical power spectra the predictions of traditional models depend on the type of nonlinearity and fail for energy measures. We then argue that the concept of intrinsic dimensionality, and the existence of end-stopped neurons, can help to understand the role of the nonlinearities. Furthermore, we show examples where models without strong i2D selectivity fail to predict the correct ranking order of perceptual segregation. Our arguments regarding the importance of i2D-features resemble the arguments of Julesz and colleagues on behalf of textons like terminators and crossings. However, we provide a computational framework which identifies textons with the outputs of nonlinear operators which are selective to i2D features.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

The geometric concept of curvature can help to deal with some important aspects of information processing in natural and artificial vision systems. The paper briefly reviews earlier results regarding the relationship between the notions of curvature, the processing of information, and the modelling of end-stopped neurons in the visual cortex. It then focuses on the difference between Gaussian curvature and the Riemann tensor and reveals the corresponding logical structures. Furthermore, it is shown how multidimensional deviations from flatness can be measured by two-dimensional curvature operators. In the context of image-sequence analysis, a new relationship between the Riemann tensor components and the computation of the optical flow is found.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

Orientation selectivity is a basic property of neurones in the visual cortex of higher vertebrates. Such neurones can be seen to act as `feature detectors', which provide an efficient cortical representation of the outside world. More recently, the removal of correlations between the signals of cortical neurones has been suggested as suitable theoretical concept for explaining the development of receptive fields. Corresponding neural network simulations yielded oriented `receptive field' structures resembling those observed by neurophysiologists. The findings suggest that the `decorrelation approach' can provide a causal relationship between characteristics of the physical world and brain function. However, we were able to reveal a basic deficit of the decorrelation approach which we illustrate by the construction of two artificial `worlds', a `Gaussian' one and an `orientation-only' one. We show that, according to the decorrelation approach, oriented environmental features would be neither necessary nor sufficient for the development of oriented receptive fields. Thus the link between environmental structure and cortical orientation selectivity still awaits a theoretical explanation.

Abstract

Local intrinsic dimensionality is shown to be an elementary structural property of multidimensional signals that cannot be evaluated using linear filters. We derive a class of polynomial operators for the detection of intrinsically 2-D image features like curved edges and lines, junctions, line ends, etc. Although it is a deterministic concept, intrinsic dimensionality is closely related to signal redundancy since it measures how many of the degrees of freedom provided by a signal domain are in fact used by an actual signal. Furthermore, there is an intimate connection to multidimensional surface geometry and to the concept of `Gaussian curvature'. Nonlinear operators are inevitably required for the processing of intrinsic dimensionality since linear operators are, by the superposition principle, restricted to OR-combinations of their intrinsically 1-D eigenfunctions. The essential new feature provided by polynomial operators is their potential to act on multiplicative relations between frequency components. Therefore, such operators can provide the AND-combination of complex exponentials, which is required for the exploitation of intrinsic dimensionality. Using frequency design methods, we obtain a generalized class of quadratic Volterra operators that are selective to intrinsically 2-D signals. These operators can be adapted to the requirements of the signal processing task. For example, one can control the "curvature tuning" by adjusting the width of the stopband for intrinsically 1-D signals, or the operators can be provided in isotropic and in orientation-selective versions. We first derive the quadratic Volterra kernel involved in the computation of Gaussian curvature and then present examples of operators with other arrangements of stop and passbands. Some of the resulting operators show a close relationship to the end-stopped and dot-responsive neurons of the mammalian visual cortex

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

A new interband vector quantization of a human vision-based image representation is presented. The feature specific vector quantizer (FVQ) is suited for data compression beyond second-order decorrelation. The scheme is derived from statistical investigations of natural images and the processing principles of biological vision systems, the initial stage of the coding algorithm is a hierarchical, and orientation-selective, analytic bandpass decomposition, realized by even- and odd-symmetric filter pairs that are modeled after the simple cells of the visual cortex. The outputs of each even- and odd-symmetric filter pair are interpreted as real and imaginary parts of an analytic bandpass signal, which is transformed into a local amplitude and a local phase component according to the operation of cortical complex cells. Feature-specific multidimensional vector quantization is realized by combining the amplitude/phase samples of all orientation filters of one resolution layer. The resulting vectors are suited for a classification of the local image features with respect to their intrinsic dimensionality, and enable the exploitation of higher order statistical dependencies between the subbands. This final step is closely related to the operation of cortical hypercomplex or end-stopped cells. The codebook design is based on statistical as well as psychophysical and neurophysiological considerations, and avoids the common shortcomings of perceptually implausible mathematical error criteria. The resulting perceptual quality of compressed images is superior to that obtained with standard vector quantizers of comparable complexity

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

Human amblyopes display reduced contrast sensitivities, suffer from perceptual distortion, and their letter acuities are worse than is predicted from grating visibility. We sought the origin of these dysfunctions by measuring normal and amblyopic sensitivities to various forms of well-defined image distortion, namely band-limited phase quantization, phase quantization with additional amplitude modulation, and grey-scale modification. Our results prove the existence of an amblyopic quasi-blindness to image structure, that cannot be explained in terms of contrast detection. We discuss these findings within the computational scheme of image decomposition into local amplitude and local phase values. They are consistent with the assumption of amblyopic eyes being impaired in processing local phase but having the local amplitude (or energy, possibly at reduced gain) at their disposal. Phrased in physiological terms, we propose a scheme of complex-cells-only vision in amblyopia. We also provide a demonstration of how amblyopic eyes may see the test stimuli and natural images by generating local amplitude and phase representations at limited phase resolution.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

With a minimal set of assumptions resulting from considerations about the perception of temporal structure, we argue for the existence of a spatio-temporal memory established by the mapping of time into simultaneous physical properties. The important point of this model is the distinction between external, physical time and the internal representation of time. An immediate consequence of such a structure is the emergence of properties usually associated with the concept of iconic memory or informational persistence. Some of these properties may hence be regarded as epiphenomena produced by the testing of a spatio-temporal system with tachistoscopic spatial stimuli. The model can explain properties of the immediate memory span, the lack of effect of exposure duration on tachistoscopic report, the partial-report superiority, the decay of iconic memory, and effects of a backward mask. It does not only avoid the incompatibility problems of the frozen-image concept in dynamic vision, but also provides an adequate basis for the processing of time-varying scenes.

Abstract

In this paper we consider how the representation of images as surfaces, and their characterizations via surface differential forms, can be related to the concept of redundancy in the intensity signal. In contrast to common approaches, the basic surface types (planar, parabolic, elliptic/hyperbolic) are not seen as equal-priority classes, but as corresponding to different degrees of redundancy. This leads to a new approach to image representation and region labeling based upon generalized curvature measures. Furthermore, we employ different reconstruction algorithms to show that elliptic surface patches carry the significant information in natural images. Based upon deterministic and stochastic relaxation techniques, these algorithms allow one to reconstruct the original image from (i) "elliptic intensities" only and (ii) curvature measures which are zero for nonelliptic regions.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.

Abstract

Intrahemispheric learning in a paradigm of choice-reaction time shows multimodal response histograms where a later component gradually disappears, the first one being locked to a specific temporal window. Choice appears to be embedded in a temporally discrete process in non-automated situations.

Request publication as PDF

Enter contact information

Name:
Email:	Please provide an institutional email address.

I declare that I will use the publication for research purposes only.