Network


Latest external collaboration on country level. Dive into details by clicking on the dots.

Hotspot


Dive into the research topics where Youssef Mroueh is active.

Publication


Featured researches published by Youssef Mroueh.


computer vision and pattern recognition | 2017

Self-Critical Sequence Training for Image Captioning

Steven J. Rennie; Etienne Marcheret; Youssef Mroueh; Jarret Ross; Vaibhava Goel

Recently it has been shown that policy-gradient methods for reinforcement learning can be utilized to train deep end-to-end systems directly on non-differentiable metrics for the task at hand. In this paper we consider the problem of optimizing image captioning systems using reinforcement learning, and show that by carefully optimizing our systems using the test metrics of the MSCOCO task, significant gains in performance can be realized. Our systems are built using a new optimization approach that we call self-critical sequence training (SCST). SCST is a form of the popular REINFORCE algorithm that, rather than estimating a baseline to normalize the rewards and reduce variance, utilizes the output of its own test-time inference algorithm to normalize the rewards it experiences. Using this approach, estimating the reward signal (as actor-critic methods must do) and estimating normalization (as REINFORCE algorithms typically do) is avoided, while at the same time harmonizing the model with respect to its test-time inference procedure. Empirically we find that directly optimizing the CIDEr metric with SCST and greedy decoding at test-time is highly effective. Our results on the MSCOCO evaluation sever establish a new state-of-the-art on the task, improving the best result in terms of CIDEr from 104.9 to 114.7.


international conference on acoustics, speech, and signal processing | 2015

Deep multimodal learning for Audio-Visual Speech Recognition

Youssef Mroueh; Etienne Marcheret; Vaibhava Goel

In this paper, we present methods in deep multimodal learning for fusing speech and visual modalities for Audio-Visual Automatic Speech Recognition (AV-ASR). First, we study an approach where uni-modal deep networks are trained separately and their final hidden layers fused to obtain a joint feature space in which another deep network is built. While the audio network alone achieves a phone error rate (PER) of 41% under clean condition on the IBM large vocabulary audio-visual studio dataset, this fusion model achieves a PER of 35.83% demonstrating the tremendous value of the visual channel in phone classification even in audio with high signal to noise ratio. Second, we present a new deep network architecture that uses a bilinear softmax layer to account for class specific correlations between modalities. We show that combining the posteriors from the bilinear networks with those from the fused model mentioned above results in a further significant phone error rate reduction, yielding a final PER of 34.03%.


international symposium on information theory | 2014

On efficiency and low sample complexity in phase retrieval

Youssef Mroueh; Lorenzo Rosasco

In this paper we show that the problem of phase retrieval can be efficiently and provably solved via an alternating minimization algorithm suitably initialized. Our initialization is based on One Bit Phase Retrieval that we introduced in [1], where we showed that O(n log(n)) Gaussian phase-less measurements ensure robust recovery of the phase. In this paper we improve the sample complexity bound to O(n) measurements for sufficiently large n, using a variant of Matrix Bernstein concentration inequality that exploits the intrinsic dimension, together with properties of one bit phase retrieval.


neural information processing systems | 2012

Multiclass Learning with Simplex Coding

Youssef Mroueh; Tomaso Poggio; Lorenzo Rosasco; Jean-Jacques E. Slotine


international conference on machine learning | 2015

Convex Learning of Multiple Tasks and their Structure

Carlo Ciliberto; Youssef Mroueh; Tomaso Poggio; Lorenzo Rosasco


international conference on machine learning | 2017

McGan: Mean and Covariance Feature Matching GAN.

Youssef Mroueh; Tom Sercu; Vaibhava Goel


arXiv: Computer Vision and Pattern Recognition | 2014

Can a biologically-plausible hierarchy e ectively replace face detection, alignment, and recognition pipelines?

Qianli Liao; Joel Z. Leibo; Youssef Mroueh; Tomaso Poggio


neural information processing systems | 2017

Fisher GAN

Youssef Mroueh; Tom Sercu


international conference on artificial intelligence and statistics | 2016

Local Group Invariant Representations via Orbit Embeddings

Anant Raj; Abhishek Kumar; Youssef Mroueh; P. Thomas Fletcher; Bernhard Schölkopf


The Handbook of Multimodal-Multisensor Interfaces | 2017

Audio and visual modality combination in speech processing applications

Gerasimos Potamianos; Etienne Marcheret; Youssef Mroueh; Vaibhava Goel; Alexandros Koumbaroulis; Argyrios Vartholomaios; Spyridon Thermos

Collaboration


Dive into the Youssef Mroueh's collaboration.

Top Co-Authors

Avatar

Lorenzo Rosasco

Massachusetts Institute of Technology

View shared research outputs
Top Co-Authors

Avatar

Tomaso Poggio

Massachusetts Institute of Technology

View shared research outputs
Top Co-Authors

Avatar

Jean-Jacques E. Slotine

Massachusetts Institute of Technology

View shared research outputs
Top Co-Authors

Avatar
Top Co-Authors

Avatar
Top Co-Authors

Avatar

Joel Z. Leibo

Massachusetts Institute of Technology

View shared research outputs
Researchain Logo
Decentralizing Knowledge