Leonid I. Perlovsky

Maximum likelihood neural networks for sensor fusion and adaptive classification

Abstract A maximum likelihood artificial neural system (MLANS) has been designed for problems which require an adaptive estimation of metrics in classification spaces. Examples of such problems are an XOR problem and most classification problems with multiple classes having complicated classifier boundaries. The metric estimation has the capability of achieving flexible classifier boundary shapes using a simple architecture without hidden layers. This neural network learns much more efficiently than other neural networks or classification algorithms, and it approaches the theoretical bounds on adaptive efficiency according to the Cramer-Rao theorem. It also provides for optimal fusing of all the available information, such as a priori and real-time information coming from a variety of sensors of the same or different types, and utilizes fuzzy classification variables to provide for the efficient utilization of incomplete or erroneous data, including numeric and symbolic data. This paper describes the neural network and presents examples of its performances in unsupervised, partially supervised, and environment-interrogation modes. We discuss the Cramer-Rao theory as it relates to neural networks, the relevance of the MLANS to biological and other neural networks, and issues for future work.

Conundrum of combinatorial complexity

This paper examines fundamental problems underlying difficulties encountered by pattern recognition algorithms, neural networks, and rule systems. These problems are manifested as combinatorial complexity of algorithms, of their computational or training requirements. The paper relates particular types of complexity problems to the roles of a priori knowledge and adaptive learning. Paradigms based on adaptive learning lead to the complexity of training procedures, while nonadaptive rule-based paradigms lead to complexity of rule systems. Model-based approaches to combining adaptivity with a priori knowledge lead to computational complexity. Arguments are presented for the Aristotelian logic being culpable for the difficulty of combining adaptivity and a priority. The potential role of the fuzzy logic in overcoming current difficulties is discussed. Current mathematical difficulties are related to philosophical debates of the past.

Musical emotions: Functions, origins, evolution

Theories of music origins and the role of musical emotions in the mind are reviewed. Most existing theories contradict each other, and cannot explain mechanisms or roles of musical emotions in workings of the mind, nor evolutionary reasons for music origins. Music seems to be an enigma. Nevertheless, a synthesis of cognitive science and mathematical models of the mind has been proposed describing a fundamental role of music in the functioning and evolution of the mind, consciousness, and cultures. The review considers ancient theories of music as well as contemporary theories advanced by leading authors in this field. It addresses one hypothesis that promises to unify the field and proposes a theory of musical origin based on a fundamental role of music in cognition and evolution of consciousness and culture. We consider a split in the vocalizations of proto-humans into two types: one less emotional and more concretely-semantic, evolving into language, and the other preserving emotional connections along with semantic ambiguity, evolving into music. The proposed hypothesis departs from other theories in considering specific mechanisms of the mind-brain, which required the evolution of music parallel with the evolution of cultures and languages. Arguments are reviewed that the evolution of language toward becoming the semantically powerful tool of today required emancipation from emotional encumbrances. The opposite, no less powerful mechanisms required a compensatory evolution of music toward more differentiated and refined emotionality. The need for refined music in the process of cultural evolution is grounded in fundamental mechanisms of the mind. This is why todays human mind and cultures cannot exist without todays music. The reviewed hypothesis gives a basis for future analysis of why different evolutionary paths of languages were paralleled by different evolutionary paths of music. Approaches toward experimental verification of this hypothesis in psychological and neuroimaging research are reviewed.

“Vague-to-Crisp” Neural Mechanism of Perception

This brief describes neural modeling fields (NMFs) for object perception, a bio-inspired paradigm. We discuss previous difficulties in object perception algorithms encountered since the 1950s, and describe how NMF overcomes these difficulties. NMF mechanisms are compared to recent experimental neuroimaging observations, which have demonstrated that initial top-down signals are vague and during perception they evolve into crisp representations matching the bottom-up signals from observed objects. Neural and mathematical mechanisms are described and future research directions outlined.

2009 Special Issue: Language and cognition

What is the role of language in cognition? Do we think with words, or do we use words to communicate made-up decisions? The paper briefly reviews ideas in this area since 1950s. Then we discuss mechanisms of cognition, recent neuroscience experiments, and corresponding mathematical models. These models are interpreted in terms of a biological drive for cognition. Based on the Grossberg-Levine theory of drives and emotions, we identify specific emotions associated with the need for cognition. We demonstrate an engineering application of the developed technique, which significantly improves detection of patterns in noise over the previous state-of-the-art. The developed mathematical models are extended toward language. Then we consider possible brain-mind mechanisms of interaction between language and cognition. A mathematical analysis imposes restrictions on possible mechanisms. The proposed model resolves some long-standing language-cognition issues: how the mind learns correct associations between words and objects among an astronomical number of possible associations; why kids can talk about almost everything, but cannot act like adults, what exactly are the brain-mind differences; why animals do not talk and think like people. Recent brain imaging experiments indicate support for the proposed model. We discuss future theoretical and experimental research.

Evolution of Languages, Consciousness and Cultures

The knowledge instinct is a fundamental mechanism of the mind that drives evolution of higher cognitive functions. Neural modeling fields and dynamic logic describe it mathematically and relate to language, concepts, emotions, and behavior. Perception and cognition, consciousness and unconsciousness, are described, while overcoming past mathematical difficulties of modeling intelligence. The two main aspects of the knowledge instinct determining evolution are differentiation and synthesis. Differentiation proceeds from and unconscious states to more crisp and conscious, from less knowledge to more knowledge; it separates concepts from emotions, Its main mechanism is language. Synthesis strives to achieve unity and meaning of knowledge; it is necessary for resolving contradictions, concentrating will and for purposeful actions. Synthesis connects language and cognition. Its main mechanisms are emotionality of languages and the hierarchy of the mind. Differentiation and synthesis are in complex relationship of symbiosis and opposition. This Leads to complex dynamics of evolution of consciousness and languages. Its mathematical modeling predicts evolution of cultures. We discuss existing evidence and future research directions.

Model-based neural network for target detection in SAR images

A controversial issue in the research of mathematics of intelligence has been that of the roles of a priori knowledge versus adaptive learning. After discussing mathematical difficulties of combining a priority with adaptivity encountered in the past, we introduce a concept of a model-based neural network, whose adaptive learning is based on a priori models. Applications to target detection in SAR images are discussed. We briefly overview the SAR principles, derive relatively simple physics-based models of SAR signals, and describe model-based neural networks that utilize these models. A number of real-world application examples are presented.

Curiosity and pleasure

We discuss the hypothesis that acquisition of knowledge is a deeply rooted psychological need. But so is the desire for fast decisions and for minimizing cognitive efforts. There is a controversy between maximizing knowledge rationally for decision making or using Tversky and Kahneman heuristic mechanisms. Here we explore a basic aspect of learning, does it bring pleasure? We report experimental results showing that acquisition of knowledge is hedonically pleasing. Thus, the satisfaction of curiosity through acquiring knowledge brings pleasure and could improve decision making. Such a mechanism would confirm the hypothesis that curiosity is a fundamental and ancient motivation.

Cognitive high level information fusion

Fusion of sensor and communication data currently can only be performed at a late processing stage after sensor and textual information are formulated as logical statements at appropriately high level of abstraction. Contrary to this it seems, the human mind integrates sensor and language signals seamlessly, before signals are understood, at pre-conceptual level. Learning of conceptual contents of the surrounding world depends on language and vice versa. The paper describes a mathematical technique for such integration. It combines fuzzy dynamic logic with dual cognitive-language models. The paper briefly discusses relationships between the proposed mathematical technique, working of the mind and applications to understanding-based search engines.

Neural Networks for Improved Tracking

In this letter, we have developed a neural network (NN) based upon modeling fields for improved object tracking. Models for ground moving target indicator (GMTI) tracks have been developed as well as neural architecture incorporating these models. The neural tracker overcomes combinatorial complexity of tracking in highly cluttered scenarios and results in about 20-dB (two orders of magnitude) improvement in signal-to-clutter ratio.