Learning Hierarchical Word and Sentence Representations

Add to your list(s) Download to your calendar using vCal

• Dani Yogatama, DeepMind
• Friday 03 March 2017, 12:00-13:00
• FW26, Computer Laboratory.

If you have a question about this talk, please contact Kris Cao.

Languages encode meaning in terms of hierarchical, nested structures. For example, we often found coarse-to-fine organization of words’ meanings in the field of lexical semantics (e.g., WordNet); and relationships among words in a sentence are largely organized in terms of latent nested structures (Chomsky, 1957). In this talk, I will first discuss how to incorporate hierarchical prior knowledge into a word representation model. I will show how to use regularizers to encourage hierarchical organization of the latent dimensions of vector-space word embeddings.

I will then talk about a reinforcement learning method to learn tree-structured neural networks for computing representations of natural language sentences. In contrast to sequential RNNs which ignore tree structure, our model generates a latent tree for each sentence using a reward from a semantic interpretation task to syntactically structure the composition. I will show that learning how words compose to form sentence meanings leads to better performance on various downstream tasks.

This talk is part of the NLIP Seminar Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• FW26, Computer Laboratory
• Graduate-Seminars
• Guy Emerson's list
• Interested Talks
• Language Sciences for Graduate Students
• NLIP Seminar Series
• PMRFPS's
• Simon Baker's List
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.