Panagiotis Patrinos

Variable selection in nonlinear modeling based on RBF networks and evolutionary computation.

In this paper a novel variable selection method based on Radial Basis Function (RBF) neural networks and genetic algorithms is presented. The fuzzy means algorithm is utilized as the training method for the RBF networks, due to its inherent speed, the deterministic approach of selecting the hidden node centers and the fact that it involves only a single tuning parameter. The trade-off between the accuracy and parsimony of the produced model is handled by using Final Prediction Error criterion, based on the RBF training and validation errors, as a fitness function of the proposed genetic algorithm. The tuning parameter required by the fuzzy means algorithm is treated as a free variable by the genetic algorithm. The proposed method was tested in benchmark data sets stemming from the scientific communities of time-series prediction and medicinal chemistry and produced promising results.

Forward---backward quasi-Newton methods for nonsmooth optimization problems

The forward–backward splitting method (FBS) for minimizing a nonsmooth composite function can be interpreted as a (variable-metric) gradient method over a continuously differentiable function which we call forward–backward envelope (FBE). This allows to extend algorithms for smooth unconstrained optimization and apply them to nonsmooth (possibly constrained) problems. Since the FBE can be computed by simply evaluating forward–backward steps, the resulting methods rely on a similar black-box oracle as FBS. We propose an algorithmic scheme that enjoys the same global convergence properties of FBS when the problem is convex, or when the objective function possesses the Kurdyka–Łojasiewicz property at its critical points. Moreover, when using quasi-Newton directions the proposed method achieves superlinear convergence provided that usual second-order sufficiency conditions on the FBE hold at the limit point of the generated sequence. Such conditions translate into milder requirements on the original function involving generalized second-order differentiability. We show that BFGS fits our framework and that the limited-memory variant L-BFGS is well suited for large-scale problems, greatly outperforming FBS or its accelerated version in practice, as well as ADMM and other problem-specific solvers. The analysis of superlinear convergence is based on an extension of the Dennis and Moré theorem for the proposed algorithmic scheme.

Asymmetric Forward-Backward-Adjoint Splitting for Solving Monotone Inclusions Involving Three Operators

In this work we propose a new splitting technique, namely Asymmetric Forward–Backward–Adjoint splitting, for solving monotone inclusions involving three terms, a maximally monotone, a cocoercive and a bounded linear operator. Our scheme can not be recovered from existing operator splitting methods, while classical methods like Douglas–Rachford and Forward–Backward splitting are special cases of the new algorithm. Asymmetric preconditioning is the main feature of Asymmetric Forward–Backward–Adjoint splitting, that allows us to unify, extend and shed light on the connections between many seemingly unrelated primal-dual algorithms for solving structured convex optimization problems proposed in recent years. One important special case leads to a Douglas–Rachford type scheme that includes a third cocoercive operator.

GPU-Accelerated Stochastic Predictive Control of Drinking Water Networks

Despite the proven advantages of scenario-based stochastic model predictive control for the operational control of water networks, its applicability is limited by its considerable computational footprint. In this paper, we fully exploit the structure of these problems and solve them using a proximal gradient algorithm parallelizing the involved operations. The proposed methodology is applied and validated on a case study: the water network of the city of Barcelona.

Distributed solution of stochastic optimal control problems on GPUs

Stochastic optimal control problems arise in many applications and are, in principle, large-scale involving up to millions of decision variables. Their applicability in control applications is often limited by the availability of algorithms that can solve them efficiently and within the sampling time of the controlled system. In this paper we propose a dual accelerated proximal gradient algorithm which is amenable to parallelization and demonstrate that its GPU implementation affords high speed-up values (with respect to a CPU implementation) and greatly outperforms well-established commercial optimizers such as Gurobi.

Distributed computing over encrypted data

We present a new theme for performing computations directly on encrypted data: the concept of homomorphic encryption, i.e., cryptosystems that allow a user to manipulate encrypted information even without knowing the secret key. In an attempt to alleviate the gap between cryptography which naturally operates on rings, groups, and fields, and signal processing which typically operates on real(complex)-valued data, we set the stage for distributed operations on encrypted data. We leverage advances in homomorphic encryption (such as the RSA, Paillier and Gentry cryptosystems), and in quantized signal processing and consensus to devise encryption-friendly distributed computing primitives. In specific, we show how to perform encrypted average consensus with finite-time convergence, using modular multiplication and exponentiation on encrypted information. We present the architecture for secure cloud computing and discuss its advantages and applicability to a wide range of data mining, signal processing and control operations over the cloud.

A dual gradient-projection algorithm for model predictive control in fixed-point arithmetic

Although linear Model Predictive Control has gained increasing popularity for controlling dynamical systems subject to constraints, the main barrier that prevents its widespread use in embedded applications is the need to solve a Quadratic Program (QP) in real-time. This paper proposes a dual gradient projection (DGP) algorithm specifically tailored for implementation on fixed-point hardware. A detailed convergence rate analysis is presented in the presence of round-off errors due to fixed-point arithmetic. Based on these results, concrete guidelines are provided for selecting the minimum number of fractional and integer bits that guarantee convergence to a suboptimal solution within a pre-specified tolerance, therefore reducing the cost and power consumption of the hardware device.

Fixed-Point Implementation of a Proximal Newton Method for Embedded Model Predictive Control

Abstract Extending the success of model predictive control (MPC) technologies in embedded applications heavily depends on the capability of improving quadratic programming (QP) solvers. Improvements can be done in two directions: better algorithms that reduce the number of arithmetic operations required to compute a solution, and more efficient architectures in terms of speed, power consumption, memory occupancy and cost. This paper proposes an implementation with fixed-point arithmetic of a proximal Newton method to solve optimization problems arising in input-constrained MPC. The main advantages of the algorithm are its fast asymptotic convergence rate and its relatively low computational cost per iteration since it the solution of a small linear system is required. A detailed analysis on the effects of quantization errors is presented, showing the robustness of the algorithm with respect to finite-precision computations. A hardware implementation with specific optimizations to minimize computation times and memory footprint is also described, demonstrating the viability of low-cost, low-power controllers for high-bandwidth MPC applications. The algorithm is shown to be very effective for embedded MPC applications through a number of simulation experiments.

Uncertainty-aware demand management of water distribution networks in deregulated energy markets

We present an open-source solution for the operational control of drinking water distribution networks which accounts for the inherent uncertainty in water demand and electricity prices in the day-ahead market of a volatile deregulated economy. As increasingly more energy markets adopt this trading scheme, the operation of drinking water networks requires uncertainty-aware control approaches that mitigate the effect of volatility and result in an economic and safe operation of the network that meets the consumers’ need for uninterrupted water supply. We propose the use of scenario-based stochastic model predictive control: an advanced control methodology which comes at a considerable computation cost which is overcome by harnessing the parallelization capabilities of graphics processing units (GPUs) and using a massively parallelizable algorithm based on the accelerated proximal gradient method.

Stochastic gradient methods for stochastic model predictive control

We introduce a new stochastic gradient algorithm, SAAGA, and investigate its employment for solving Stochastic MPC problems and multi-stage stochastic optimization programs in general. The method is particularly attractive for scenario-based formulations that involve a large number of scenarios, for which “batch” formulations may become inefficient due to high computational costs. Benefits of the method include cheap computations per iteration and fast convergence due to the sparsity of the proposed problem decomposition.