University of Cambridge > > Language Technology Lab Seminars > Learning language by observing the world and learning about the world from language

Learning language by observing the world and learning about the world from language

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Marinela Parovic.

Children learn about the visual world from implicit supervision that language provides. Most children learn their language, at least to some extent, by observing the world. Recently released datasets of instructional videos are interesting as they can be considered a rough approximation of a child’s visual and linguistic experience—in these videos, the narrator performs a high-level task (e.g., cooking pasta) while describing the steps involved in that task (e.g., boiling water). Moreover, these datasets pose challenges similar to those children need to address; for example, identifying relevant activities to the task (e.g., boiling water) and ignoring the rest (e.g., shaking head). I will present two projects where we study the interaction of visual and linguistic signals in these videos: (1) We show that using language and the structure of tasks is important in discovering action boundaries. (2) I will discuss how visual signal improves the quality of unsupervised word translation, especially for dissimilar languages, and where we do not have access to large corpora.

This talk is part of the Language Technology Lab Seminars series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.


© 2006-2023, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity