Paraphrases do not explain word analogies

Louis Fournier, Ewan Dunbar

Interpretability and Analysis of Models for NLP Short paper Paper

Gather-3C: Apr 23, Gather-3C: Apr 23 (13:00-15:00 UTC) [Join Gather Meeting]

You can open the pre-recorded video in separate windows.

Abstract: Many types of distributional word embeddings (weakly) encode linguistic regularities as directions (the difference between jump and jumped will be in a similar direction to that of walk and walked, and so on). Several attempts have been made to explain this fact. We respond to Allen and Hospedales’ recent (ICML, 2019) theoretical explanation, which claims that word2vec and GloVe will encode linguistic regularities whenever a specific relation of paraphrase holds between the four words involved in the regularity. We demonstrate that the explanation does not go through: the paraphrase relations needed under this explanation do not hold empirically
NOTE: Video may display a random order of authors. Correct author list is at the top of this page.

Connected Papers in EACL2021

Similar Papers

Ellipsis Resolution as Question Answering: An Evaluation
Rahul Aralikatte, Matthew Lamm, Daniel Hardt, Anders Søgaard,
Probing for idiomaticity in vector space models
Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, Aline Villavicencio,
Disambiguatory Signals are Stronger in Word-initial Positions
Tiago Pimentel, Ryan Cotterell, Brian Roark,