José David Lopes Benefited from the CMU Portugal Program Support
 |
José David Lopes, a researcher at INESC-ID, has recently concluded his Ph.D. as part of the doctoral program in Electrical and Computer Engineering at the Instituto Superior Técnico of the Universidade de Lisboa (IST-UL). During his doctoral studies, José David Lopes had the opportunity to spend time in the Language Technologies Institute at Carnegie Mellon University (CMU), in the United States, where he used lexical entrainment in Spoken Dialog Systems to improve the interaction between machines and humans. |
José David Lopes was co-advised by Isabel Trancoso, a researcher at INESC-ID and professor at IST-UL, and Maxine Eskenazi, professor at CMU. Although he did not participate directly in the CMU Portugal Program, he benefited from its support, through funding for his visits to CMU, the opportunity to stay with dual degree doctoral students from the Program while in Pittsburgh, as well as the integration in CMU’s research team.
“When we started planning his Ph.D.,” stated Isabel Trancoso, “we were looking for a co-advisor for José David since the plan was to have him spend some time abroad and the topic of his thesis matched Maxine’s research areas perfectly.” When they asked her if she wanted to co-advise him, “she accepted immediately.” The research topic, according to Isabel Trancoso, “was new and challenging” and “basically he tried to adapt the way the Spoken Dialog System talks to the user as the conversation is happening.” For that, he used lexical entrainment to improve the performance of the system. A Spoken Dialog System (SDS) is a computer system that aims to establish spoken dialog with humans, while lexical entrainment is a phenomenon in conversational linguistics of the process where the subject adopts the reference terms (for instance, a word) of their interlocutor to ensure maximum clarity in the communication between the parties. This process is necessary to overcome ambiguity.
José David Lopes defended his thesis, titled Lexical Entrainment in Spoken Dialog Systems , in the beginning of December 2013. According to Maxine Eskenazi, “he did a great job. I know that when I get students from INESC-ID they have a really strong background, they have taken excellent courses and they’re capable of doing great things. They are not daring enough, I would say, but we make them daring,” she explained. “I pushed him to do a lot of things and he ended up doing an excellent two-way lexicon entrainment system that we are hoping others will want to copy,” the professor added. According to José David Lopes, “the ultimate purpose was to allow for a better interaction between SDSs and humans.”
Teaching Machines How To Speak With Humans
While at CMU, José David Lopes started by trying to understand if people pick up the words provided by an automatic dialog system. It is known that when humans talk to each other they adapt the speech if they want the dialog to succeed. “From previous work we knew that users would pick things up and if they do not we try to understand how the system can throw out a new word that is easier for the speech recognizer,” Maxine Eskenazi explained. Therefore, first the idea was trying to see if the system could understand the words provided by the human and if it could throw out words that are easier for the system to recognize. Then, “he developed a decision mechanism to help the system find out the best word to use so that it can perform better. He put it on our live Let’s Go! system, which people use to find bus schedules in Pittsburg. So he had real users testing it,” Maxine Eskenazi clarified. After the work in Pittsburg, José David Lopes went back to Portugal, where he created the Noctívago , a query system for the night buses in Lisbon. “José made very significant contributions, and he never gave up, even when facing the many difficulties of building a multimodal dialogue system for Portuguese with non-real users,” Isabel Trancoso explained. “There were some contacts with the bus company in Lisbon at an early development stage of the system, but unfortunately so far the system has not yet reached real users,” José David Lopes added.
As a result of his research, José David Lopes found that lexical entrainment is a relevant behavior in dialogs between humans and machines. “We were able to prove that the systems that used lexical entrainment performed better than the systems that did not.” And according to the researcher, this happens for two reasons: the system was not only able to detect whenever a lexical component was hindering the dialog flow, and propose a different synonym that could be better recognized by the user, but also to detect when the user preferred a specific lexical item, adopting it to enhance the engagement between the user and the system. One of the most positive things about being at CMU was “that having Maxine Eskenazi as a co-advisor allowed me to test this on their Let’s Go! system, which provides bus schedule information to real users in Pittsburgh. It was a great challenge, and the results were very encouraging,” the researcher confessed, adding that the experience of being co-advised by two professors, one in Portugal and one at CMU, “was very easy. They [Isabel Trancoso and Maxine Eskenazi] are very important references in the area and it was a real pleasure working with them.”
System On The Way To The Spanish City of Bilbao
| José David Lopes’s research efforts in this area are now beginning to bear fruit. After developing a system that provided schedule information for night buses in Lisbon based on CMU’s system, a research group from the University of the Basque Country proposed to the Bilbao city council a system similar to the Let’s Go! . “I was in Pittsburgh at the time and after developing a similar system for the Portuguese language and working with Let’s Go! , they invited me to come to Bilbao to help them build it,” José David Lopes explained, adding that “hopefully the first demo will soon be available for testing.” |  |
When questioned about the future, José David Lopes stated that he does “not have long-term plans. I’m finishing the task in Bilbao, and soon I will start a post-doc at KTH in Sweden.” The experience he acquired during the Ph.D. made he think that “I want to keep doing research for the next couple of years. I learned a lot both at INESC-ID’s Spoken Language Laboratory and at CMU’s Language Technologies Institute, and I want to use that knowledge in my research, always bearing in mind that what I do should be used to make people’s lives better,” he concluded adding that he “would like to thank the CMU Portugal Program for the help and support provided throughout my Ph.D.. Although I was not a Ph.D. student from the program, I felt I was treated like one of their own.”
February 2014
