André Martins, a Portuguese researcher from Instituto de Telecomunicações (IT) and CMU Portugal alum, has won a European Research Council Consolidator grant (ERC CoG) of 2 M€ to study artificial neural networks applied to natural language processing (NLP).
The ERC Consolidator funding will be applied to the DECOLLAGE project (DEep COgnition Learning for LAnguage GEneration), focused on finding solutions to some of the fundamental problems of NLP, using an innovative interdisciplinary methodology that brings together tools from artificial intelligence, sparse modeling, neuroscience, and cognitive sciences.
The existing deep learning models for natural language processing, while sometimes impressive, are often unreliable and even misleading: they do not generalize well to new domains, they do not exploit contextual information, they are poorly calibrated, and their memory is not traceable. These limitations stem from their monolithic architectures, suitable for perception but unsuitable for tasks that require higher-level cognition.
“This project is a step further in overcoming the limitations of current NLP technologies, making it possible for humans and machines to communicate effectively in natural language and to work collaboratively to solve increasingly harder problems”, André Martins
With the funding available under this new Grant, the research will focus on three main steps. First, using uncertainty and quality estimates as a guiding principle for controlled generation, combining this control mechanism with efficiently encoding contextual information and integrating multiple modalities. Second, developing sparse and structured models of long-term memory, with attention to descriptive representations, and third, devising new mathematical models for sparse communication (bridging the gap between discrete and continuous representations), supporting end-to-end differentiability and enabling a shared workspace where multiple modules and agents can communicate. These innovations will be applied to highly challenging generation tasks, including machine translation, dialogue, and open-ended generation.
This ERC award follows an early starting grant won by the researcher in 2017, worth 1.4 million euros, applied to the DeepSPIN project that allowed him to develop his research in deep learning and structured prediction to solve challenging tasks in natural language processing, including machine translation, quality estimation, and syntactic parsing.
André Martins was the first CMU Portugal Dual Degree Ph.D. Alumni in Language Technologies, having concluded his doctoral degree in 2012 granted by Técnico and Carnegie Mellon University (CMU). Since then, he has been highly involved with the Program initiatives, supervising two Ph.D. candidates and leading two CMU Portugal Large Scale projects (MAIA and GoLocal) in strict collaboration with teams at CMU. He is currently Vice President of AI Research at Unbabel and an Associate Professor at Instituto Superior Técnico (Técnico). He co-founded and co-organizes the Lisbon Machine Learning School (LxMLS), already in its 12th Edition, which counts with the support of the CMU Portugal Program, and is a Fellow of the ELLIS society, the pan-European AI network. With over 100 papers published, +6,000 citations, and an h-index of 37 in top-tier conferences and journals, André stands among the top 2% of the most-cited scientists in his scientific field by Elsevier.
FCT news article
In the Media: Público, RTP online, 24 Sapo Online
