A Portuguese team won the second prize at the 2022 Alexa Prize TaskBot Challenge . Promoted by Amazon USA, the ten university teams in competition had to develop bots to assist customers in completing cooking or do-it-yourself home improvement tasks requiring multiple steps and decisions. The awarded team was led by CMU Portugal Faculty member, João Magalhães, professor at the Department of Informatics at NOVA School of Science and Technology (FCT NOVA) and researcher at NOVA LINCS, and counted, among others, with the participation of 3 CMU Portugal Ph.D. students: Diogo Tavares, Diogo Silva and Gustavo Gonçalves.
After nearly a year of research, the three best conversational assistants were announced and the FCT NOVA Portuguese team has proudly won second place. Their bot TWIZ, a conversational Task Wizard with multimodal curiosity-exploration, stood out by introducing curiosity to the challenge and mentioning curious facts related to the tasks being performed, which increased customer engagement. João Magalhães and David Semedo led the Portuguese team composed by Diogo Tavares and Diogo Silva (CMU Portugal Affiliated Ph.D Students), Gustavo Gonçalves (CMU Portugal Dual Degree Ph.D.) and 6 other students (Rafael Ferreira, Hélder Rodrigues, Mariana Bonito, Frederico Vicente, Rui Margarido, Paula Figueiredo).
The opportunity to participate in the 2021 edition of the Alexa TaskBot Challenge was possible thanks to the research work that was already being developed on conversational assistants under the CMU Portugal large-scale collaborative project IFetch, which is being led at FCT NOVA by João Magalhães. Under this project, the FCT NOVA team is supporting the development of a multimodal conversational agent for the online fashion marketplace with the company Farfetch.
Regarding CMU Portugal contribution to this achievement João Magalhães explains that “the Program played a key role as an enabler, both through the iFetch program and through the CMU Portugal funded PhD students. The iFetch team at NOVA University had already a running solid research program in multimodal conversational agents, so this was a natural step. Now, this outstanding achievement was mainly due to a combination of several groundbreaking AI advances invented by our students: the robust dialog understanding Transformer algorithm, and the language generation algorithms capable of talking about curious facts and algorithms to answer questions about the recipe and its video.“
Being part of this competition had, according to the team leader two very rewarding aspects “first, the access to real-world data and real-world users. This offered us a great opportunity to push the frontiers of the state of the art. Second, working with an amazing team towards this goal was extremely fulfilling.”.
About the Alexa Taskbot Challenge, sponsored by Amazon, João Magalhães states that it “was a great test-bed to validate multimodal conversational assistants that guide users in executing complex tasks like fixing a broken chair or cooking a chocolate cake. There were 10 universities participating world-wide and our solution was among the top three systems. “
The first place went to GRILLBot, a “multimodal task-oriented digital assistant to guide users through complex real-world tasks” developed by a team of graduate students at the University of Glasgow. A team from Ohio State University (OSU) won third place with its TacoBot.
The prize money for the top three was $500,000 for first place, $100,000 for second, and $50,000 for third.
Amazon News article here.
