Real-world task assistance with GRILLBot, the winning Amazon Alexa Prize TaskBot.

Add to your list(s) Download to your calendar using vCal

• Carlos Gemmell, University of Glasgow
• Thursday 16 March 2023, 11:00-12:00
• English Faculty Building, second floor, SR24.

If you have a question about this talk, please contact Panagiotis Fytas.

Effective virtual assistants have to guide users through long and complex tasks, be engaging, and help solve unpredictable problems along the way. This talk presents GRILL Bot, a multi-modal task-oriented assistant, as the winning system in the 2022 Amazon Alexa Prize TaskBot challenge. GRILL Bot personalizes assistance in cooking, DIY , and crafts by leveraging TaskGraphs, a novel task representation. TaskGrahps unify steps, requirements, and curated domain knowledge into dynamic graphs enabling detailed contextual explanations and adaptable task execution. Further augmentations with segmented videos and linked step images allow full use of the multi-modal experience. We will explore the critical roles language models play in GRILL Bot both in offline task augmentations like contextual jokes and in online settings with knowledge-grounded question answering and a semantic parsing approach to dialogue management. We will finally discuss the lessons learned from a full year of real-world users and showcase GRILL Bot in action.

This talk is part of the pf376's list series.

This talk is included in these lists:

• English Faculty Building, second floor, SR24

Note that ex-directory lists are not shown.