Learning with Sparse Latent Structure

Priberam

Abstract:
Structured representations are a powerful tool in machine learning, and in particular in natural language processing: The discrete, compositional nature of words and sentences leads to natural combinatorial representations such as trees, sequences, segments, or alignments, among others. At the same time, deep, hierarchical neural networks with latent representations are increasingly widely and successfully applied to language tasks. Deep networks conventionally perform smooth, soft computations resulting in dense hidden representations.
We study deep models with structured and sparse latent representations, without sacrificing differentiability. This allows for fully deterministic models which can be trained with familiar end-to-end gradient-based methods. We demonstrate sparse and structured attention mechanisms, as well as latent computation graph structure learning, with successful empirical results on large scale problems including sentiment analysis, natural language inference, and neural machine translation.
Joint work with Claire Cardie, Mathieu Blondel, and André Martins.
Relevant publications:
<UL>
<LI>Vlad Niculae, André F. T. Martins, Mathieu Blondel, Claire Cardie. SparseMAP: Differentiable sparse structured inference. In: Proc. of ICML 2018. [ <A HREF="https://arxiv.org/abs/1802.04223" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">arXiv</A> ] [ <A HREF="https://github.com/vene/sparsemap" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">code</A> ] [ <A HREF="http://vene.ro/talks/sparsemap-icml18-talk.pdf" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">slides</A> ]</LI>
<LI>Vlad Niculae, André F. T. Martins, Claire Cardie. Towards dynamic computation graphs via sparse latent structure. In: Proc. of EMNLP 2018. [ <A HREF="https://arxiv.org/abs/1809.00653" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">arXiv</A> ] [ <A HREF="https://github.com/vene/sparsemap/tree/master/cpp" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">code</A> ]</LI>
<LI>Vlad Niculae and Mathieu Blondel. A regularized framework for sparse and structured neural attention. In: Proc. of NeurIPS 2017. [ <A HREF="https://arxiv.org/abs/1705.07704" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">arXiv</A> ] [ <A HREF="https://github.com/vene/sparse-structured-attention" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">code</A> ]</LI>
</UL>
Short Bio:
Vlad [1] is a postdoc in the DeepSPIN project at the Instituto de Telecomunicações in Lisbon, Portugal. His research aims to bring structure and sparsity to neural network hidden layers and latent variables, using ideas from convex optimization, and motivations from natural language processing. He earned a PhD in Computer Science from Cornell University in 2018. He is co-organizing the NAACL 2019 Workshop on Structured Prediction for NLP [2].
[1] <A HREF="https://vene.ro" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">https://vene.ro</A>
[2] <A HREF="http://structuredprediction.github.io/SPNLP19" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">http://structuredprediction.github.io/SPNLP19</A>
More info: <A HREF="http://labs.priberam.com/Academia-Partnerships/Seminars.aspx" TARGET="_blank" DATA-MSYS-CLICKTRACK="0" REL="nofollow noopener noreferrer">http://labs.priberam.com/Academia-Partnerships/Seminars.aspx</A>

Instituto Superior Técnico

Our {communityGuidelinesLink} describe the sort of content we prohibit on Eventbrite. If you suspect an event may be in violation, you can report it to us so we can investigate.

To report other categories of prohibited or illegal content, submit {link}

Learning with Sparse Latent Structure

More events from Priberam

Discover more events from Priberam, from Science & Tech to other experiences you might love.

Still looking for the right event?

Explore all events in Lisboa and filter by date, category, and more to find the perfect fit.

Learning with Sparse Latent Structure

More events from Priberam

Discover more events from Priberam, from Science & Tech to other experiences you might love.

More Science & Tech events

Browse more Science & Tech events with different dates, prices, and formats to find your next great experience.

Still looking for the right event?

Explore all events in Lisboa and filter by date, category, and more to find the perfect fit.