Bon K. Sy

Probability model selection using information-theoretic optimization criterion

Probability models with discrete random varibales are often used for probabilistic inference and decision support. A fundamental issue lies in the choice and the validity of the probability model. An information theoretic-based approach for probability model selection is discussed. It will be shown that the problem of probability model selection is discussed. It will be shown that the problem of probability model selection can be formulated as an optimization problem of probability model selection can be formulated as on optimization problem with linear (in)equality constraints and a non-linear objective function. An algorithm for model discovery/selection based on a primal–dual formulation similar to that of the interior point method is presented. The implementation of the algorithm for solving an algebraic system of linear constraints is based on singular value decomposition and the numerical method proposed by Kuenzi, Tzschach and Zehnder. Preliminary comparative evaluation is also discussed.

Information Theory & Statistics

Data mining is about discovering useful information from data. This chapter is devoted to introducing the basic knowledge in information theory and statistics. Readers should be aware that information theory and statistics are useful mathematical tools for data mining. But there are also other approaches for data mining. For example, multivariate data visualization, and understanding the information behind the data from the structure of computational geometry, are two active research areas in data mining [Wong 1999][Inselberg 1994][Beckamn 1995]. Nevertheless, the data mining techniques to be introduced in the subsequent chapters are grounded on information theory, probability and statistics. Thus it is important to have a brief, but sufficient, tutorial in information theory and statistics.

A recurrence local computation approach towards ordering composite beliefs in Bayesian belief networks

Abstract Finding the l Most Probable Explanations (MPE) of a given evidence, S e , in a Bayesian belief network can be formulated as identifying and ordering a set of composite hypotheses, H i s, of which the posterior probabilities are the l largest; ie, Pr(H 1 ¦S e ) ≥ … ≥ Pr(H 1 ¦S e ). When an order includes all the composite hypotheses in the network in order to find all the probable explanations, it becomes a total order and the derivation of such an order has an exponential complexity. The focus of this paper is on the derivation of a partial order, with length l, for finding the l most probable composite hypotheses; where l typically is much smaller than the total number of composite hypotheses in a network. Previously, only the partial order of length two (ie, l = 2) in a singly connected Bayesian network could be efficiently derived without further restriction on network topologies and the increase in spatial complexity. This paper discusses an efficient algorithm for the derivation of the partial ordering of the composite hypotheses in a singly connected network with arbitrary order length. This algorithm is based on the propagation of quantitative vector streams in a feed-forward manner to a designated “root” node in a network. The time complexity of the algorithm is in the order of O(lkn); where l is the length of a partial order, k the length of the longest path in a network, and n the maximum number of node states—defined as the product of the size of the conditional probability table of a node and the number of incoming messages towards the node.

An AI-based communication system for motor and speech disabled persons: design methodology and prototype testing

An intelligent communication device is developed to assist nonverbal, motor-disabled persons in the generation of written and spoken messages. The device is centered on a knowledge base of the grammatical rules and message elements. A belief reasoning scheme based on both the information from external sources and the embedded knowledge is used to optimize the process of message search. The search for the message elements is conceptualized as a path search in the language graph, and a special frame architecture is used to construct and to partition the graph. Bayesian belief reasoning from the Dempster-Shafer theory of evidence is augmented to cope with time-varying evidence. An information fusion strategy is also introduced to integrate various forms of external information. Experimental testing of the prototype system is discussed.<<ETX>>

Signature-Based approach for intrusion detection

This research presents a data mining technique for discovering masquerader intrusion. User/system access data are used as a basis for deriving statistically significant event patterns. These patterns could be considered as a user/system access signature. Signature-based approach employs a model discovery technique to derive a reference ground model accounting for the user/system access data. A unique characteristic of this reference ground model is that it captures the statistical characteristics of the access signature, thus providing a basis for reasoning the existence of a security intrusion based on comparing real time access signature with that embedded in the reference ground model. The effectiveness of this approach will be evaluated based on comparative performance using a publicly available data set that contains user masquerade.

Secure Computation for Biometric Data Security—Application to Speaker Verification

The goal of this research is to develop provable secure computation techniques for two biometric security tasks in complex distributed systems involving multiple parties; namely biometric data retrieval and authentication. We first present models for privacy and security that delineate the conditions under which biometric data disclosure are allowed. We then discuss the secure computation techniques for retrieval and authentication that satisfy the conditions for privacy and security. For proof-of-concept, we show a practical implementation of a privacy preserving speaker verification system and discuss the performance tradeoff.

Information-statistical pattern based approach for data mining

This paper presents information theory and statistical analysis as two fundamental conceptual tools for data mining. A data mining technique based on these two conceptual tools consists of three steps. The first step is a statistical approach for discovering data patterns. The second step is an information-theoretic approach for identifying models that encapsulate the statistical behavior of the data patterns. The last step is a probabilistic approach for pattern-based inference that uncovers unknown significant event patterns.

Discovering association patterns based on mutual information

Identifying and expressing data patterns in form of association rules is a commonly used technique in data mining. Typically, association rules discovery is based on two criteria: support and confidence. In this paper we will briefly discuss the insufficiency on these two criteria, and argue the importance of including interestingness/dependency as a criterion for (association) pattern discovery. From the practical computational perspective, we will show how the proposed criterion grounded on interestingness could be used to improve the efficiency of pattern discovery mechanism. Furthermore, we will show a probabilistic inference mechanism that provides an alternative to pattern discovery. Example illustration and preliminary study for evaluating the proposed approach will be presented.

Secure Computation for Privacy Preserving Biometric Data Retrieval and Authentication

The goal of this research is to develop provable secure computation techniques for two biometric security tasks; namely biometric data retrieval and authentication. We first present models for privacy and security that delineate the conditions under which biometric data disclosure are allowed, the conditions under which the protocol for data exchange should be provable secure, and the conditions under which the computation should be provable private. We then present a novel technique based on singular value decomposition and homomorphic encryption to achieve secure computation for biometric data retrieval. Finally we show a proof-of-concept implementation of the proposed techniques to realize a privacy preserving speaker verification system.

Secure Information Processing with Privacy Assurance - standard based design and development for biometric applications

This paper presents the design and development of a technique referred to as SIPPA - Secure Information Processing with Privacy Assurance - for biometric data reconstruction. SIPPA enables a client/server model with the following two properties: (1) the client party can compare the similarity between his/her sample data with the source data on the server side — without each party revealing his/her data to another, nor to a third party. If the sample data is “sufficiently similar” to the source data, the client can reconstruct the source data by using only the sample data and some helper data with negligible overhead provided by the server. The main contributions of this paper are: (1) algorithmic steps of SIPPA and its relationship to privacy homomorphism, (2) a parallel SIPPA architecture, and (3) the realization of parallel SIPPA as a service component for BioAPI 2.0 framework using Java RMI technology. To demonstrate its potential application, we apply SIPPA to the reconstruction of biometric data, and more specifically, biometric face images represented in terms of linearized vectors.