Research Abstracts - 2006

Biophysical Models of Neural Computation: MAX and Normalization

Ulf Knoblich & Tomaso Poggio

The Problem

A model of object recognition via the feedforwardpath of the ventral visual stream in cortex [9, 10] is composed of a hierarchy of feed-forward layers of neuron-like units, performing one of two basic operations. Either (1) template matching with a Gaussian function to increase feature complexity or (2) nonlinear pooling with a maximum operation to increase translation and scale invariance. However, it is not known how these basic operations could be implemented in cortex.

Motivation

Neurons in different areas of visual cortex of primates and cats have been found to exhibit behavior resembling a soft-max operation [2, 3]. Unfortunately the biophysical mechanism underlying these responses is still unknown.

It has been shown that Gaussian-like bell-shaped tuning can be achieved by the combination of normalization and linear summation [5]. Thus, a mechanism for normalization would suffice to explain the bell-shaped tuning that is utilized in the standard model.

New insights into the interplay of excitatory and inhibitory synaptic inputs suggest an important role for the balance of these two. The finding that the ratio of inhibitory and excitatory is actively balanced in the dendrite hints towards the functional significance of this ratio [6]. The crucial importance of timing for the combination of excitatory and inhibitory effects [4] underlines the need for the step from static models to detailed dynamical models of the underlying biophysics of the operations under investigation.

Previous Work

Several circuit models have been proposed to compute either a maximum operation or a normalization or both. [1, 5, 7, 8, 11] Although this variety of models exist, it has not been possible to decide which one describes the actual implementation in cortex most accurately. Since none of these models has been implemented in a biophysical framework faithful to neuronal properties, it is not entirely clear how closely these mathematical models approximate the actual cortical circuits.

Approach

Using the NEURON simulation environment and other specialized systems of differential equations, we explore dynamical biophysically plausible implementations of the previously presented circuits as well as new models utilizing special biophysical properties of the neurons that can give rise to complex behavior, paying special attention to their dynamic behavior. Investigating these detailed dynamical models then provides a basis for predictions about single neurons and circuits in visual cortex. Using these model predictions, new experiments can be designed to specifically test and distinguish between the different models, determining which model is most accurate.

A biophysically implemented circuit as a) feedforward and b) feedback version that is able to perform both a maximum-type operation and a normalization. The two inputs (x) excite the y units which are also inhibited by the pooling interneuron and converge on the output unit z. These are only two of the variety of different biophysical models that are investigated and then tested experimentally.

Progress

In order to perform the investigated computations within 20ms, a limit set by the ability of cortex to process rapidly presented images, a single spiking neuron does not offer enough dynamic range. We investigate several possible solutions to this limitation. One possible mechanism involves nonspiking neurons and graded synaptic transmission. We successfully implemented and investigated several nonspiking circuit. The current focus is on spiking circuits with ensembles of neurons replacing single units in order to achieve the dynamic range that is needed for computation.

Impact

It is an open question how the two key nonlinear operations (maximum and tuning) could be implemented biophysically. Once possible circuits have been modeled biophysically and differential predictions have been tested experimentally, incorporating the most accurate model of the cortical circuits into our theory of object recognition will then introduce dynamics and establish a much closer correspondence with neurons, including their biophysical properties in cortex.

Future Work

This work is a part of a larger project of continuously validating and enhancing the standard model, which attempts to quantitatively summarize the computational architecture of the ventral pathway with a unifying framework. Future work includes incorporating an even greater extent of electrophysiological data about V1, V2, V4, and IT through close collaborations with experimental groups.

Research Support

This abstract describes research done at the Center for Biological & Computational Learning, which is in the McGovern Institute for Brain Research at MIT, as well as in the Dept. of Brain & Cognitive Sciences, and which is affiliated with the Computer Sciences & Artificial Intelligence Laboratory (CSAIL).

This research was sponsored by grants from: Office of Naval Research (DARPA) Contract No. MDA972- 04-1-0037, Office of Naval Research (DARPA) Contract No. N00014-02-1-0915, National Science Foundation- NIH (CRCNS) Contract No. EIA-0218506, and National Institutes of Health (Conte) Contract No. 1 P20 MH66239-01A1.

Additional support was provided by: Central Research Institute of Electric Power Industry (CRIEPI), Daimler-Chrysler AG, Eastman Kodak Company, Honda Research Institute USA, Inc., Komatsu Ltd., Merrill-Lynch, NEC Fund, Oxygen, Siemens Corporate Research, Inc., Sony, Sumitomo Metal Industries, Toyota Motor Corporation, and the Eugene McDermott Foundation.

References

[1] M. Carandini and D.J. Heeger. Summation and division by neurons in primate visual cortex. Science, 264:1333–1336, 1994.

[2] T. Gawne and J. Martin. Responses of primate visual cortical V4 neurons to simultaneously presented stimuli. Journal of Neurophysiology, 88:1128–1135, 2002.

[3] A.T. Gulledge and G.J. Stuart. Excitatory Actions of GABA in the Cortex. Neuron, 37:299–309, 2003.

[4] M. Kouh and T. Poggio. A General Mechanisms for Tuning: Gain Control Circuits and Synapses Underlie Tuning of Cortical Neurons. CBCL Paper #245/AI Memo #2004-031, 2004.

[5] I. Lampl, D. Ferster, T. Poggio, and M. Riesenhuber. Intracellular measurements of spatial integration and the MAX operation in complex cells of the cat primary visual cortex. Journal of Neurophysiology, 92:2704–2713, 2004.

[6] G. Liu. Local structural balance and functional interaction of excitatory and inhibitory synapses in hippocampal dendrites. Nature Neuroscience, 7(4):373–379, 2004.

[7] T. Poggio and E. Bizzi. Generalization in vision and motor control. Nature, 431:768–774, 2004.

[8] T. Poggio and W. Reichardt. Visual control of orientation behaviour in the fly. Part II. Towards the underlying neural interactions. Q R Bioph., 9(3):377–438, 1976.

[9] M. Riesenhuber and T. Poggio. Hierarchical models of object recognition in cortex. Nature Neuroscience, 2:1019–1025, 1999.

[10] T. Serre, M. Kouh, C. Cadieu, U. Knoblich, G. Kreiman, and T. Poggio. A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex. CBCL Paper #259/AI Memo #2005-036, 2004.

[11] A. Yu, M. Giese, and T. Poggio. Biophysiologically plausible implementations of the maximum operation. Neural Computation, 14(12):2857–2881, 2002.