On July 3rd at 11am, Louis-Philippe Morency, Associate Professor at Carnegie Mellon University, will host the talk “What is Multimodal?” (Hybrid), providing a look into the world of multimodal research.
GAIPS Lab promotes the talk from INESC ID/Técnico and will take place in room 1.38, Instituto Superior Técnico, Taguspark and via Zoom.
Louis-Philippe Morency was PI under the CMU Portugal Exploratory Research project AGENTS, led in Portugal by Ana Paiva (GAIPS Lab I INESC D and Técnico) focused on how humor can be used in the interaction between social robots and humans.
Abstract: Our experience of the world is multimodal – we see objects, hear sounds, feel texture, smell odors, and taste flavors. In recent years, a broad and impactful body of research emerged in artificial intelligence under the umbrella of multimodal, characterized by multiple modalities. As we formalize a long-term research vision for multimodal research, it is important to reflect on its foundational principles and core technical challenges. What is multimodal? Answering this question is complicated by the multidisciplinary nature of the problem, spread across many domains and research fields. Two key principles have driven many multimodal innovations: heterogeneity and interconnections from multiple modalities. Historical and recent progress will be synthesized in a research-oriented taxonomy, centered around 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification. The talk will conclude with open questions and unsolved challenges essential for a long-term research vision in multimodal research.
Bio: Louis-Philippe Morency is Associate Professor at the Language Technology Institute at Carnegie Mellon University, where he leads the Multimodal Communication and Machine Learning Laboratory (MultiComp Lab). He was formerly research faculty in the Computer Sciences Department at the University of Southern California and received his Ph.D. degree from MIT Computer Science and Artificial Intelligence Laboratory. His research focuses on building the computational foundations to enable computers with the abilities to analyze, recognize and predict subtle human communicative behaviors during social interactions. He received diverse awards, including AI’s 10 to Watch by IEEE Intelligent Systems, NetExplo Award in partnership with UNESCO, and 10 best paper awards at IEEE and ACM conferences. His research was covered by media outlets such as Wall Street Journal, The Economist, and NPR.
