Lokendra Shastri

From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony

Human agents draw a variety of inferences effortlessly, spontaneously, and with remarkable efficiency – as though these inferences were a reflexive response of their cognitive apparatus. Furthermore, these inferences are drawn with reference to a large body of background knowledge. This remarkable human ability seems paradoxical given the complexity of reasoning reported by researchers in artificial intelligence. It also poses a challenge for cognitive science and computational neuroscience: How can a system of simple and slow neuronlike elements represent a large body of systemic knowledge and perform a range of inferences with such speed? We describe a computational model that takes a step toward addressing the cognitive science challenge and resolving the artificial intelligence paradox. We show how a connectionist network can encode millions of facts and rules involving n -ary predicates and variables and perform a class of inferences in a few hundred milliseconds. Efficient reasoning requires the rapid representation and propagation of dynamic bindings. Our model (which we refer to as SHRUTI) achieves this by representing (1) dynamic bindings as the synchronous firing of appropriate nodes, (2) rules as interconnection patterns that direct the propagation of rhythmic activity, and (3) long-term facts as temporal pattern-matching subnetworks. The model is consistent with recent neurophysiological evidence that synchronous activity occurs in the brain and may play a representational role in neural information processing. The model also makes specific psychologically significant predictions about the nature of reflexive reasoning. It identifies constraints on the form of rules that may participate in such reasoning and relates the capacity of the working memory underlying reflexive reasoning to biological parameters such as the lowest frequency at which nodes can sustain synchronous oscillations and the coarseness of synchronization.

A Connectionist Approach to Knowledge Representation and Limited Inference

Although the connectionist approach has lead to elegant solutions to a number of problems in cognitive science and artificial intelligence, its suitability for dealing with problems in knowledge representation and inference has often been questioned. This paper partly answers this criticism by demonstrating that effective solutions to certain problems in knowledge representation and limited inference can be found by adopting a connectionist approach. The paper presents a connectionist realization of semantic networks, that is, it describes how knowledge about concepts, their properties, and the hierarchical relationship between them may be encoded as an interpreter-free massively parallel network of simple processing elements that can solve an interesting class of inheritance and recognition problems extremely fast—in time proportional to the depth of the conceptual hierarchy. The connectionist realization is based on an evidential formulation that leads to principled solutions to the problems of exceptions and conflicting multiple inheritance situations during inheritance, and the best-match or partial-match computation during recognition. The paper also identifies constraints that must be satisfied by the conceptual structure in order to arrive at an efficient parallel realization.

Advances in SHRUTI—A Neurally Motivated Model of RelationalKnowledge Representation and Rapid Inference Using Temporal Synchrony

We are capable of drawing a variety of inferences effortlessly, spontaneously, and with remarkable efficiency—as though these inferences are a reflex response of our cognitive apparatus. This remarkable human ability poses a challenge for cognitive science and computational neuroscience: How can a network of slow neuron-like elements represent a large body of systematic knowledge and perform a wide range of inferences with such speed? The connectionist model SHRUTI attempts to address this challenge by demonstrating how a neurally plausible network can encode a large body of semantic and episodic facts, systematic rules, and knowledge about entities and types, and yet perform a wide range of explanatory and predictive inferences within a few hundred milliseconds. Relational structures (frames, schemas) are represented in SHRUTI by clusters of cells, and inference in SHRUTI corresponds to a transient propagation of rhythmic activity over such cell-clusters wherein dynamic bindings are represented by the synchronous firing of appropriate cells. SHRUTI encodes mappings across relational structures using high-efficacy links that enable the propagation of rhythmic activity, and it encodes items in long-term memory as coincidence and coincidence-error detector circuits that become active in response to the occurrence (or non-occurrence) of appropriate coincidences in the on going flux of rhythmic activity. Finally, “understanding” in SHRUTI corresponds to reverberant and coherent activity along closed loops of neural circuitry. Over the past several years, SHRUTI has undergone several enhancements that have augmented its expressiveness and inferential power. This paper describes some of these extensions that enable SHRUTI to (i) deal with negation and inconsistent beliefs, (ii) encode evidential rules and facts, (iii) perform inferences requiring the dynamic instantiation of entities, and (iv) seek coherent explanations of observations.

Rules and variables in neural nets

A fundamental problem that must be addressed by connectionism is that of creating and representing dynamic structures (Feldman 1982; von der Malsburg 1985). In the context of reasoning with systematic and abstract knowledge, this problem takes the form of the variable binding problem. We describe a biologically plausible solution to this problem and outline how a knowledge representation and reasoning system can use this solution to perform a class of predictive inferences with extreme efficiency. The proposed system solves the variable binding problem by propagating rhythmic patterns of activity wherein dynamic bindings are represented as the synchronous firing of appropriate nodes.

Default reasoning in semantic networks: a formalization of recognition and inheritance

Abstract Many knowledge-based systems express domain knowledge in terms of a hierarchy of concepts (or frames)—where each concept is a collection of property value (or slot filler) pairs. One can associate two interesting classes of inference with such information structures: inheritance and recognition. Attempts at formalizing inheritance and recognition, however, have been confounded by the presence of conflicting property values among related concepts. Such conflicting information gives rise to the problems of exceptions and multiple inheritance during inheritance, and partial matching during recognition. This paper presents a formalization of inheritance and recognition based on the principle of maximum entropy. The proposed formalization offers several advantages: it admits necessary as well as default property values, it deals with conflicting information in a principled manner, and it solves the problems of exceptions, multiple inheritance, as well as partial matching. It can also be shown that the proposed formalization may be realized as a massively parallel network of simple processing elements that can solve an interesting class of inheritance and recognition problems extremely fast—in time proportional to the depth of the conceptual hierarchy.

CONNECTIONISM AND THE COMPUTATIONAL EFFECTIVENESS OF REASONING

It is generally acknowledged that tremendous computational activity underlies some of the most commonplace cognitive behavior. If we view these computations as systematic rule governed operations over symbolic structures (i.e., inferences) we are confronted with the following challenge: Any generalized notion of inference is intractable, yet our ability to perform cognitive tasks such as language understanding in real-time suggests that we are capable of performing a wide range of inferences with extreme efficiency almost as a matter of reflex. One response to the above challenge is that the traditional formulation is simply inappropriate and it is erroneous to view computations underlying cognition as inferences. An alternate response and the one pursued in this paper is that the traditional account is basically sound: The notion of symbolic representation is fundamental to a computational model of cognition and so is the view that computations in a cognitive system correspond to systematic rule governed operations. However, there is much more to a computational account of cognition than what is captured by these assertions. What is missing is an appreciation of the intimate and symbiotic relationship between the nature of representation, the effectiveness of inference, and the computational architecture in which the computations are situated. We argue that the structured connectionist approach offers the appropriate framework for explicating this symbiotic relationship and meeting the challenge of computational effectiveness.

Incremental class learning approach and its application to handwritten digit recognition

Incremental Class Learning (ICL) provides a feasible framework for the development of scalable learning systems. Instead of learning a complex problem at once, ICL focuses on learning subproblems incrementally, one at a time — using the results of prior learning for subsequent learning — and then combining the solutions in an appropriate manner. With respect to multi-class classification problems, the ICL approach presented in this paper can be summarized as follows. Initially the system focuses on one category. After it learns this category, it tries to identify a compact subset of features (nodes) in the hidden layers, that are crucial for the recognition of this category. The system then freezes these crucial nodes (features) by fixing their incoming weights. As a result, these features cannot be obliterated in subsequent learning. These frozen features are available during subsequent learning and can serve as parts of weight structures built to recognize other categories. As more categories are learned, the set of features gradually stabilizes and learning a new category requires less effort. Eventually, learning a new category may only involve combining existing features in an appropriate manner. The approach promotes the sharing of learned features among a number of categories and also alleviates the wellknown catastrophic interference problem. We present promising results of applying the ICL approach to the unconstrained Handwritten Digit Recognition problem, based on a spatio-temporal representation of patterns.

A computational model of episodic memory formation in the hippocampal system

Abstract The memorization of events and situations (episodic memory) requires the rapid formation of a memory trace consisting of several functional components. A computational model is described that demonstrates how a transient pattern of activity representing an episode can lead to the rapid recruitment of appropriate circuits as a result of long-term potentiation within structures whose architecture and circuitry match those of the hippocampal formation, a neural structure known to play a critical role in the formation of such memories.

Reflexive Reasoning with Multiple Instantiation in a Connectionist Reasoning System with a Type Hierarchy

Abstract We describe a hybrid knowledge representation and reasoning system that integrates a rule-based reasoner with a type hierarchy and can accommodate multiple dynamic instantiations of predicates. The system—which is an extension of the reasoner described in Shastri and Ajjanagadde (1990)maintains and propagates variable bindings using temporally synchronous (i.e. in-phase)firing of appropriate nodes, and can perform a broad class of reasoning with extreme efficiency. The type hierarchy allows the system to encode generic facts such as ‘cats prey on bird’ and rules such as ‘if x preys on y then y is scared of ’ and use them to infer that Tweety the canary is scared of Sylvester the cat. The system can also encode qualified rules such as ‘if an animate agent collides with a solid object then the agent gets hur’. The ability to accommodate multiple dynamic instantiations of any predicate allows the system to handle a much broader class of inferences, including those involving transitivity and bounded ...

WHY SEMANTIC NETWORKS

Abstract It is often asserted that semantic networks, are mere notational variants of other well-defined and “standard” representation languages. Yet semantic networks seem to have a substantial following and a special appeal of their own. What makes semantic networks special, and why has so much research effort been devoted to developing network-based knowledge representation languages? In this chapter we attempt to answer this question from the perspective of computational effectiveness. We argue that a knowledge representation language must not only be characterized in terms of its representational adequacy but also in terms of its computational effectiveness. Furthermore, a computationally effective knowledge representation framework must explicate the relationship between the nature of representation, the effectiveness of certain inferences and the computational architecture in which the computations/inferences are performed. Semantic networks, or graph-based representation formulations are examples of such knowledge representation frameworks. In particular, semantic networks—realized as massively parallel networks—may provide the appropriate framework for modeling reflexive reasoning—reasoning that we can perform rapidly, effortlessly, and without conscious effort.