Panos P. Markopoulos

Optimal Algorithms for L 1 -subspace Signal Processing

We describe ways to define and calculate L1-norm signal subspaces that are less sensitive to outlying data than L2-calculated subspaces. We start with the computation of the L1 maximum-projection principal component of a data matrix containing N signal samples of dimension D. We show that while the general problem is formally NP-hard in asymptotically large N, D, the case of engineering interest of fixed dimension D and asymptotically large sample size N is not. In particular, for the case where the sample size is less than the fixed dimension , we present in explicit form an optimal algorithm of computational cost 2N. For the case N ≥ D, we present an optimal algorithm of complexity O(ND). We generalize to multiple L1-max-projection components and present an explicit optimal L1 subspace calculation algorithm of complexity O(NDK-K+1) where K is the desired number of L1 principal components (subspace rank). We conclude with illustrations of L1-subspace signal processing in the fields of data dimensionality reduction, direction-of-arrival estimation, and image conditioning/restoration.

Fast computation of the L 1 -principal component of real-valued data

Recently, Markopoulos et al. [1], [2] presented an optimal algorithm that computes the L1 maximum-projection principal component of any set of N real-valued data vectors of dimension D with complexity polynomial in N, O(ND). Still, moderate to high values of the data dimension D and/or data record size N may render the optimal algorithm unsuitable for practical implementation due to its exponential in D complexity. In this paper, we present for the first time in the literature a fast greedy single-bit-flipping conditionally optimal iterative algorithm for the computation of the L1 principal component with complexity O(N3). Detailed numerical studies are carried out demonstrating the effectiveness of the developed algorithm with applications to the general field of data dimensionality reduction and direction-of-arrival estimation.

L 1 -fusion: Robust linear-time image recovery from few severely corrupted copies

We address the problem of recovering an unknown image of interest, when only few, severely corrupted copies are available. We employ, for the first time in the literature, corruption-resistant L1-Principal-Components (L1-PCs) of the image data-set at hand. Specifically, the calculated L1-PCs are used for reliability-based patch-by-patch fusion of the corrupted image copies into a single high-quality representation of the original image (L1-fusion). Our experimental studies illustrate that the proposed method offers remarkable recovery results for several common corruption types, even under high corruption rate, small number of copies, and varying corruption type among copies. An additional theoretical contribution of this work is that the L1-PC of a data matrix of non-negative entries (e.g., image data) is for the first time shown to be optimally calculable with complexity linear to the matrix dimensions - as of now, the fastest-known optimal algorithm is of polynomial complexity. In the light of this result, L1-fusion is carried out with linear cost comparable to that of the simple copy-averaging alternative. The linear-low cost of L1-fusion allows for the recovered image to be, optionally, further refined by means of sophisticated single-image restoration techniques.

Reduced-rank filtering on L1-norm subspaces

Recent studies in signal processing have unveiled the remarkable outlier-resistance properties of L1-norm subspaces, calculated by means of L1-norm principal component analysis (L1-PCA). In this work, we present for the first time reduced-rank interference-suppressive filtering on L1-norm subspaces of the received signal vectors. Our simulation studies illustrate that the proposed filtering framework allows for successful suppression of coherent interference while, at the same time, it offers sturdy protection against outliers that appear among the training samples.

Direction finding with L1-norm subspaces

Conventional subspace-based signal direction-of-arrival estimation methods rely on the familiar L2-norm-derived principal components (singular vectors) of the observed sensor-array data matrix. In this paper, for the first time in the literature, we find the L1-norm maximum projection components of the observed data and search in their subspace for signal presence. We demonstrate that L1-subspace direction-of-arrival estimation exhibits (i) similar performance to L2 (usual singular-value/eigen-vector decomposition) direction-of-arrival estimation under normal nominal-data system operation and (ii) significant resistance to sporadic/occasional directional jamming and/or faulty measurements.

L1-Norm Principal-Component Analysis via Bit Flipping

The K L1-norm Principal Components (L1-PCs) of a data matrix X Ε RD × N can be found optimally with cost O(2NK), in the general case, and O(Nrank(X)K - K + 1), when rankX is a constant with respect to N [1],[2]. Certainly, in real-world applications where N is large, even the latter polynomial cost is prohibitive. In this work, we present L1-BF: a novel, near-optimal algorithm that calculates the K L1-PCs of X with cost O (NDmin{N, D} + N2(K4 + DK2) + DNK3), comparable to that of standard (L2-norm) Principal-Component Analysis. Our numerical studies illustrate that the proposed algorithm attains optimality with very high frequency while, at the same time, it outperforms on the L1-PCA metric any counterpart of comparable computational cost. The outlier-resistance of the L1-PCs calculated by L1-BF is documented with experiments on dimensionality reduction and genomic data classification for disease diagnosis.

On the L1-Norm Approximation of a Matrix by Another of Lower Rank

In the past decade, there has been a growing documented effort to approximate a matrix by another of lower rank minimizing the L1-norm of the residual matrix. In this paper, we first show that the problem is NP-hard. Then, we introduce a theorem on the sparsity of the residual matrix. The theorem sets the foundation for a novel algorithm that outperforms all existing counterparts in the L1-norm error minimization metric and exhibits high outlier resistance in comparison to usual L2-norm error minimization in machine learning applications.

Novel full-rate noncoherent alamouti encoding that allows polynomial-complexity optimal decoding

We consider Alamouti encoding that draws symbols from M-ary phase-shift keying (M-PSK) and develop a new differential modulation scheme that attains full rate for any constellation order. In contrast to past work, the proposed scheme guarantees that the encoded matrix maintains the characteristics of the initial codebook and, at the same time, attains full rate so that all possible sequences of space-time matrices become valid. The latter property is exploited to develop a polynomial-complexity maximum-likelihood noncoherent sequence decoder whose order is solely determined by the number of receive antennas. We show that the proposed scheme is superior to contemporary alternatives in terms of encoding rate, decoding complexity, and performance.

Short-data-record filtering of PN-masked data

Pseudo-noise (PN) masking is regarded as an effective means to combat data eavesdropping (for example in military-grade communications or positioning systems). At the same time, PN-masked data transmissions are considered vulnerable to interference/jamming due to lack of practical interference suppression solutions. In this work, (i) we derive an efficient minimum-mean-square-error (MMSE) optimal linear receiver of PN-masked data and (ii) develop an auxiliary-vector (AV) MMSE adaptive filter estimator with state-of-the-art small-sample-support estimation performance. Simulation studies included in this paper illustrate the effectiveness of the theoretical developments.

Small-Sample-Support Suppression of Interference to PN-Masked Data

In the context of secure wireless communications, pseudo-noise (PN) masking of transferred data has proven to be an effective technique against eavesdropping (notable examples are military-grade communication and global-positioning systems). At the same time, PN-masked transmissions are thought to be vulnerable to interference/jamming due to lack of a minimum-mean-square-error (MMSE) disturbance suppressing solution. In this paper, for the first time we establish the MMSE operation for masked data in the form of a time (mask) varying linear filter, suggest an implementation that avoids repeated input autocorrelation matrix inversion, and develop an auxiliary-vector (AV) MMSE filter estimator with state-of-the-art short-data-record estimation performance. Simulation examples included herein illustrate the theoretical developments.