Antonio Agudo

Good Vibrations: A Modal Analysis Approach for Sequential Non-rigid Structure from Motion

We propose an online solution to non-rigid structure from motion that performs camera pose and 3D shape estimation of highly deformable surfaces on a frame-by-frame basis. Our method models non-rigid deformations as a linear combination of some mode shapes obtained using modal analysis from continuum mechanics. The shape is first discretized into linear elastic triangles, modelled by means of finite elements, which are used to pose the force balance equations for an undamped free vibrations model. The shape basis computation comes down to solving an eigenvalue problem, without the requirement of a learning step. The camera pose and time varying weights that define the shape at each frame are then estimated on the fly, in an online fashion, using bundle adjustment over a sliding window of image frames. The result is a low computational cost method that can run sequentially in real-time. We show experimental results on synthetic sequences with ground truth 3D data and real videos for different scenarios ranging from sparse to dense scenes. Our system exhibits a good trade-off between accuracy and computational budget, it can handle missing data and performs favourably compared to competing methods.

Surface properties of molybdenum-impregnated ZSM-5 catalysts

Molybdena-containing H-ZSM5 zeolites have been prepared and used in the hydrodesulfurization of thiophene and dibenzothiophene model compounds. The catalysts containing 3 and 6% MoO3 were prepared by aqueous impregnation with ammonium heptamolybdate, followed by calcination at 383, 623 and 773 K. They have been characterized by X-ray diffraction (XRD), infrared spectroscopy (FTIR), temperature-programmed reduction (TPR), and scanning electron microscopy (SEM). The set of data indicated that the zeolite crystallinity is mostly preserved for the high Mo-containing catalysts, and that most of the molybdena appears as MoO3 crystals on the external zeolite surface. The appearance of MoS2 structures produced by sulfidation has already been observed by X-ray photoelectron spectroscopy (XPS) and FTIR spectroscopy of an NO probe molecule. These also showed a better dispersion for the low Mo-containing catalysts. All these data and activity data are jointly discussed.

Stability and catalytic properties of Mo-containing Y-type zeolite catalysts calcined by an isobaric thermal decomposition procedure

Abstract A study has been made of the effect of calcination temperature, under isobaric thermal conditions, on the structural stability and catalytic activity for thiophene conversion of Mo-impregnated NaY and NH4NaY zeolite catalysts. The IR spectra, X-ray diffraction, BET surface area and sorption capacity results indicated that catalyst crystallinity was largely preserved after treatment at 110°C, and decreased substantially after calcination at over 350°C. All Mo-containing zeolite catalysts remained stable in air.

Sequential Non-Rigid Structure from Motion Using Physical Priors

We propose a new approach to simultaneously recover camera pose and 3D shape of non-rigid and potentially extensible surfaces from a monocular image sequence. For this purpose, we make use of the Extended Kalman Filter based Simultaneous Localization And Mapping (EKF-SLAM) formulation, a Bayesian optimization framework traditionally used in mobile robotics for estimating camera pose and reconstructing rigid scenarios. In order to extend the problem to a deformable domain we represent the objects surface mechanics by means of Naviers equations, which are solved using a Finite Element Method (FEM). With these main ingredients, we can further model the materials stretching, allowing us to go a step further than most of current techniques, typically constrained to surfaces undergoing isometric deformations. We extensively validate our approach in both real and synthetic experiments, and demonstrate its advantages with respect to competing methods. More specifically, we show that besides simultaneously retrieving camera pose and non-rigid shape, our approach is adequate for both isometric and extensible surfaces, does not require neither batch processing all the frames nor tracking points over the whole sequence and runs at several frames per second.

Finite Element based sequential Bayesian Non-Rigid Structure from Motion

Naviers equations modelling linear elastic solid deformations are embedded within an Extended Kalman Filter (EKF) to compute a sequential Bayesian estimate for the Non-Rigid Structure from Motion problem. The algorithm processes every single frame of a sequence gathered with a full perspective camera. No prior data association is assumed because matches are computed within the EKF prediction-match-update cycle. Scene is coded as a Finite Element Method (FEM) elastic thin-plate solid, where the discretization nodes are the sparse set of scene points salient in the image. It is assumed a set of Gaussian forces acting on solid nodes to cause scene deformation. The EKF combines in a feedback loop an approximate FEM model and the frame rate measurements from the camera, resulting in an efficient method to embed Naviers equations without resorting to expensive non-linear FEM models. Classical FEM modelling has implied an interactive identification of boundary points to constrain the scene rigid motion, in this work this dissatisfying prior knowledge is no longer needed. The scene and camer rigid motion are combined in a unique pose vector and the estimation is coded relative to the camera. Additionally, the deforming effect of the Gaussian forces on the thin-plate is computed by means of the Moore-Penrose pseudoinverse of the FEM stiffness matrix. The proposed algorithm is validated with three real sequences gathered with hand-held camera observing isometric and non-isometric deformations. It is also shown the consistency of the EKF estimation with respect to ground truth computed from stereo.

Online dense non-rigid 3D shape and camera motion recovery

© 2014. The copyright of this document resides with its authors. This paper describes a sequential solution to dense non-rigid structure from motion that recovers the camera motion and 3D shape of non-rigid objects by processing a monocular image sequence as the data arrives. We propose to model the time-varying shape with a probabilistic linear subspace of mode shapes obtained from continuum mechanics. To efficiently encode the deformations of dense 3D shapes that contain a large number of mesh vertexes, we propose to compute the deformation modes on a down sampled rest shape using finite element modal analysis at a low computational cost. This sparse shape basis is then grown back to dense exploiting the shape functions within a finite element. With this probabilistic low-rank constraint, we estimate camera pose and non-rigid shape in each frame using expectation maximization over a sliding window of frames. Since the time-varying weights are marginalized out, our approach only estimates a small number of parameters per frame, and hence can potentially run in real time. We evaluate our algorithm on both synthetic and real sequences with 3D ground truth data for different objects ranging from inextensible to extensible deformations and from sparse to dense shapes. We show the advantages of our approach with respect to competing sequential methods.

Simultaneous pose and non-rigid shape with particle dynamics

In this paper, we propose a sequential solution to simultaneously estimate camera pose and non-rigid 3D shape from a monocular video. In contrast to most existing approaches that rely on global representations of the shape, we model the object at a local level, as an ensemble of particles, each ruled by the linear equation of the Newtons second law of motion. This dynamic model is incorporated into a bundle adjustment framework, in combination with simple regularization components that ensure temporal and spatial consistency of the estimated shape and camera poses. The resulting approach is both efficient and robust to several artifacts such as noisy and missing data or sudden camera motions, while it does not require any training data at all. Validation is done in a variety of real video sequences, including articulated and non-rigid motion, both for continuous and discontinuous shapes. Our system is shown to perform comparable to competing batch, computationally expensive, methods and shows remarkable improvement with respect to the sequential ones.

Real-time 3D reconstruction of non-rigid shapes with a single moving camera

Abstract This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small maps.

Learning Shape, Motion and Elastic Models in Force Space

In this paper, we address the problem of simultaneously recovering the 3D shape and pose of a deformable and potentially elastic object from 2D motion. This is a highly ambiguous problem typically tackled by using low-rank shape and trajectory constraints. We show that formulating the problem in terms of a low-rank force space that induces the deformation, allows for a better physical interpretation of the resulting priors and a more accurate representation of the actual objects behavior. However, this comes at the price of, besides force and pose, having to estimate the elastic model of the object. For this, we use an Expectation Maximization strategy, where each of these parameters are successively learned within partial M-steps, while robustly dealing with missing observations. We thoroughly validate the approach on both mocap and real sequences, showing more accurate 3D reconstructions than state-of-the-art, and additionally providing an estimate of the full elastic model with no a priori information.

3D reconstruction of non-rigid surfaces in real-time using wedge elements

We present a new FEM (Finite Element Method) model for the 3D reconstruction of a deforming scene using as sole input a calibrated video sequence. Our approach extends the recently proposed 2D thin-plate FEM+EKF (Extended Kalman Filter) combination. Thin-plate FEM is an approximation that models a deforming 3D thin solid as a surface, and then discretizes the surface as a mesh of planar triangles. In contrast, we propose a full-fledged 3D FEM formulation where the deforming 3D solid is discretized as a mesh of 3D wedge elements. The new 3D FEM formulation provides better conditioning for the rank analysis stage necessary to remove the rigid boundary points from the formulation. We show how the proposed formulation accurately estimates deformable scenes from real imagery even for strong deformations. Crucially we also show, for the first time to the best of our knowledge, NRSfM (Non-Rigid Structure from Motion) at 30Hz real-time over real imagery. Real-time can be achieved for our 3D FEM formulation combined with an EKF resulting in accurate estimates even for small size maps.