University of Cambridge > Talks.cam > Microsoft Research Cambridge, public talks > Title to be confirmedMaking sense of language: It's okay to count

Title to be confirmedMaking sense of language: It's okay to count

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Microsoft Research Cambridge Talks Admins.

Vector-based representations have long been a popular choice for researchers interested in unsupervised learning of word meaning. With the 2013 release of word2vec, the use of shallow neural language models (sometimes called word embeddings or prediction-based models) for constructing such vectors has become extremely popular. In the past year, however, several researchers have demonstrated that traditional distributional models (sometimes called count-based models) are capable of similar levels of performance when properly parameterized. Furthermore, count-based models give rise to more easily interpretable lexical representations, making them preferable to neural models in certain use cases. I give examples of several areas in which simple co-occurrence based models demonstrate surprisingly high levels of utility or performance: predicting human judgments of semantic relatedness and similarity, estimation of geographic locations, extraction of semantic topics, and a toy question-answering task and dataset recently proposed by researchers at Facebook AI Research as a possible step towards human-level natural language understanding. Applying a very simple, interpretable model to this dataset highlights benefits and shortcomings of the proposed task, and points the way to improved training and testing environments for natural language understanding systems.Abstract not available

This talk is part of the Microsoft Research Cambridge, public talks series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity