Biological Neural Network Models for Data Compression and Classification
The backbone of a number of large scale systems is a communication network for transmitting information up and down the hierarchy. Our approach here is guided by experimental brain studies of multi-sensory convergence into the limbic system, short term integration and holding over time by the hippocampus. Motivated by studies of visual, auditory, olfactory and somatic cortices, we have hierarchical neural network models with levels of incommensurate frequency 2D coupled oscillator receptor arrays, with Hebbian learning, and chaotic attractors for encoding complex data patterns. Learning of new attractors and simulating compression of raw data are important features of these architectures.We are devising a mechanism for compression of signals from large arrays of transducers. Our data have been derived from the olfactory, visual, auditory, and somatic cortices, all operating with the same algorithms. Our model of a perceptual system has three levels in a hierarchy: Transduction and Pre-processing; Compression and Classification; and Readout and Choice. At Level 1 is a spatial array of receptors for the transduction of incident energy (odor, sound, light, or movement) into intensity values at points in time and space, coded in binary form of whether a threshold for a particular kind of energy has been exceeded. Coded inputs are gated in diastolic periods, and normalized to enhance sparse inputs and compress dense input patterns. At Level 2 is a 2-D array of coupled oscillators. Each oscillator has two point attractors, one stable at zero and the other stable at a nonzero value, and a limit cycle attractor. The rate coefficients that set the characteristic frequencies are distributed, so that there is a distribution of frequencies about an appropriate mean. Input to the system allows the array to go into limit cycle operation above an input threshold. Those nodes receiving input oscillate at the common frequency with the same phase, whereas those not receiving input have distributed phase values. A Hebbian algorithm in reinforcement learning is applied to the system to increase the connection strengths between the nodes receiving ``signal'' or foreground input on sequential trials, thus forming a cell assembly. A non-Hebbian algorithm is applied to connections of nodes receiving input on ``noise'' or background trials. Hebbian learning leads to high intensity oscillation in the gamma range (40 Hz), whereas habituation leads to enhancement of oscillation in the theta range (3-7 Hz), so that background input facilitates threshold crossing for desired input. Level 3 consists of a second layer of coupled oscillators with frequencies that are incommensurate with those in the first layer, and with distributed, delayed feedback, both positive and negative, to the first layer. The multiple feedback provides for global aperiodic solutions reflecting a global chaotic attractor of the combined arrays, called a KIII set. The formation of cell assemblies provides for wings of the global attractor. The output is read in the second layer after spatiotemporal transformation and integration, which places the second layer into a wing of the global attractor reading the class of the input. Its output is classified in R^n, where n is the number of oscillators, so the high dimensional input is reduced to the number of bits determined by the classification space.
Our aim in the proposed grant is to explore and develop our findings concerning chaos and noise, and to write the specifications to implement the system in analog hardware. We will optimize the system parameters to give 1/f spectra with aperiodic oscillations corresponding to EEGs, and stabilize it with a mix of external internal additive noise modeled on the brain. We will introduce distributed parametric variation as it exists in brains to make the solutions robust in respect to varying input intensities and patterns. We hypothesize that the 1/f noise inherent in analog amplifiers on LSI chips can be made to serve the same purpose of stabilization of chaotic activity, that the myriads of action potentials perform in the cerebral cortex. Our next step will be to test our KIV model of flexible and selective priming of information processing arrays to search for weak and varying signals, that are embedded in random background noise and transient coherent noise (``clutter''). We have detected the form, timing, and spectral characteristics of brain signals, that are transmitted to the primary sensory cortices prior to the arrival of an expected stimulus, and that facilitate the state transition of each cortex into the basin of attraction that signals a "hit", when the weak stimulus is present. We propose to construct a fourth layer for our system, which will simulate the operation of that part of the limbic system, which we believe is responsible for the implementation of attention. In future studies the problem we propose to address is that of temporal integration. How does the limbic system construct an on-going dynamical state, which reflects the sequence of input frames over the past several systoles, such that a prediction can be made for proper biasing of the several sensory Level 2 systems?