Mathematics

This paper develops a novel control-theoretic framework to analyze the non-asymptotic convergence of Q-learning. We show that the dynamics of asynchronous Q-learning with a constant step-size can be naturally formulated as a discrete-time stochastic affine switching system. Moreover, the evolution of the Q-learning estimation error is over- and underestimated by trajectories of two simpler dynamical systems. Based on these two systems, we derive a new finite-time error bound of asynchronous Q-learning when a constant stepsize is used. Our analysis also sheds light on the overestimation phenomenon of Q-learning. We further illustrate and validate the analysis through numerical simulations.

In this paper, we propose an efficient and flexible algorithm to solve dynamic mean-field planning problems based on an accelerated proximal gradient method. Besides an easy-to-implement gradient descent step in this algorithm, a crucial projection step becomes solving an elliptic equation whose solution can be obtained by conventional methods efficiently. By induction on iterations used in the algorithm, we theoretically show that the proposed discrete solution converges to the underlying continuous solution as the grid size increases. Furthermore, we generalize our algorithm to mean-field game problems and accelerate it using multilevel and multigrid strategies. We conduct comprehensive numerical experiments to confirm the convergence analysis of the proposed algorithm, to show its efficiency and mass preservation property by comparing it with state-of-the-art methods, and to illustrates its flexibility for handling various mean-field variational problems.

This paper is devoted to proposing a general weighted low-rank recovery model and designs a fast SVD-free computational scheme to solve it. First, our generic weighted low-rank recovery model unifies several existing approaches in the literature.~Moreover, our model readily extends to the non-convex setting. Algorithm-wise, most first-order proximal algorithms in the literature for low-rank recoveries require computing singular value decomposition (SVD). As SVD does not scale properly with the dimension of the matrices, these algorithms becomes slower when the problem size becomes larger. By incorporating the variational formulation of the nuclear norm into the sub-problem of proximal gradient descent, we avoid to compute SVD which results in significant speed-up. Moreover, our algorithm preserves the {\em rank identification property} of nuclear norm [33] which further allows us to design a rank continuation scheme that asymptotically achieves the minimal iteration complexity. Numerical experiments on both toy example and real-world problems including structure from motion (SfM) and photometric stereo, background estimation and matrix completion, demonstrate the superiority of our proposed algorithm.

In this paper, we propose an algorithmic framework dubbed inertial alternating direction methods of multipliers (iADMM), for solving a class of nonconvex nonsmooth multiblock composite optimization problems with linear constraints. Our framework employs the general minimization-majorization (MM) principle to update each block of variables so as to not only unify the convergence analysis of previous ADMM that use specific surrogate functions in the MM step, but also lead to new efficient ADMM schemes. To the best of our knowledge, in the \emph{nonconvex nonsmooth} setting, ADMM used in combination with the MM principle to update each block of variables, and ADMM combined with inertial terms for the primal variables have not been studied in the literature. Under standard assumptions, we prove the subsequential convergence and global convergence for the generated sequence of iterates. We illustrate the effectiveness of iADMM on a class of nonconvex low-rank representation problems.

Benchmarks in the utility function have various interpretations, including performance guarantees and risk constraints in fund contracts and reference levels in cumulative prospect theory. In most literature, benchmarks are a deterministic constant or a fraction of the underlying wealth; as such, the utility is still a univariate function of the wealth. In this paper, we propose a framework of multivariate utility optimization with general benchmark variables, which include stochastic reference levels as typical examples. The utility is state-dependent and the objective is no longer distribution-invariant. We provide the optimal solution(s) and fully investigate the issues of well-posedness, feasibility, finiteness and attainability. The discussion does not require many classic conditions and assumptions, e.g., the Lagrange multiplier always exists. Moreover, several surprising phenomena and technical difficulties may appear: (i) non-uniqueness of the optimal solutions, (ii) various reasons for non-existence of the Lagrangian multiplier and corresponding results on the optimal solution, (iii) measurability issues of the concavification of a multivariate utility and the selection of the optimal solutions, and (iv) existence of an optimal solution not decreasing with respect to the pricing kernel. These issues are thoroughly addressed, rigorously proved, completely summarized and insightfully visualized. As an application, the framework is adopted to model and solve a constraint utility optimization problem with state-dependent performance and risk benchmarks.

We revisit the S-procedure for general functions with "geometrical glasses". We thus delineate a necessary condition, and almost a sufficient condition, to have the S-procedure valid. Everything is expressed in terms of convexity of augmented sets (convex hulls, conical hulls) of images built from the data functions.

We present a model of continuous-time opinion dynamics for an arbitrary number of agents that communicate over a network and form real-valued opinions about an arbitrary number of options. The model generalizes linear and nonlinear models in the literature. Drawing from biology, physics, and social psychology, we introduce an attention parameter to modulate social influence and a saturation function to bound inter-agent and intra-agent opinion exchanges. This yields simply parameterized dynamics that exhibit the range of opinion formation behaviors predicted by model-independent bifurcation theory but not exhibited by linear models or existing nonlinear models. Behaviors include rapid and reliable formation of multistable consensus and dissensus states, even in homogeneous networks, as well as ultra-sensitivity to inputs, robustness to uncertainty, flexible transitions between consensus and dissensus, and opinion cascades. Augmenting the opinion dynamics with feedback dynamics for the attention parameter results in tunable thresholds that govern sensitivity and robustness. The model provides new means for systematic study of dynamics on natural and engineered networks, from information spread and political polarization to collective decision making and dynamic task allocation.

This paper considers decentralized stochastic optimization over a network of n nodes, where each node possesses a smooth non-convex local cost function and the goal of the networked nodes is to find an ϵ -accurate first-order stationary point of the sum of the local costs. We focus on an online setting, where each node accesses its local cost only by means of a stochastic first-order oracle that returns a noisy version of the exact gradient. In this context, we propose a novel single-loop decentralized hybrid variance-reduced stochastic gradient method, called GT-HSGD, that outperforms the existing approaches in terms of both the oracle complexity and practical implementation. The GT-HSGD algorithm implements specialized local hybrid stochastic gradient estimators that are fused over the network to track the global gradient. Remarkably, GT-HSGD achieves a network topology-independent oracle complexity of O( n ?? ϵ ?? ) when the required error tolerance ϵ is small enough, leading to a linear speedup with respect to the centralized optimal online variance-reduced approaches that operate on a single node. Numerical experiments are provided to illustrate our main technical results.

Algorithm NCL is designed for general smooth optimization problems where first and second derivatives are available, including problems whose constraints may not be linearly independent at a solution (i.e., do not satisfy the LICQ). It is equivalent to the LANCELOT augmented Lagrangian method, reformulated as a short sequence of nonlinearly constrained subproblems that can be solved efficiently by IPOPT and KNITRO, with warm starts on each subproblem. We give numerical results from a Julia implementation of Algorithm NCL on tax policy models that do not satisfy the LICQ, and on nonlinear least-squares problems and general problems from the CUTEst test set.

Motivated by their success in the single-objective domain, we propose a very simple linear programming-based matheuristic for tri-objective binary integer programming. To tackle the problem, we obtain lower bound sets by means of the vector linear programming solver Bensolve. Then, simple heuristic approaches, such as rounding and path relinking, are applied to this lower bound set to obtain high-quality approximations of the optimal set of trade-off solutions. The proposed algorithm is compared to a recently suggested algorithm which is, to the best of our knowledge, the only existing matheuristic method for tri-objective integer programming. Computational experiments show that our method produces a better approximation of the true Pareto front using significantly less time than the benchmark method on standard benchmark instances for the three-objective knapsack problem.