Deep nets trained on large amounts of unannotated text develop impressive linguistic skills. For years now, linguistically-inclined computational linguists have systematically studied the behaviour of these models through a variety of grammatical tasks, in search for new insights on the nature of language. However, this line of work has had virtually no impact on theoretical linguistics. In my talk, after reviewing some of the most exciting work in the area, I would like to provide some conjectures about why theoretical linguists do not care, and suggest a few possible avenues for a more fruitful convergence between the fields.
Speakers must decide how to convert unordered thoughts and ideas into a structured sequence of linguistic forms that communicates their intended message; that is, they must make a series of linearization decisions. One approach to this decision-making challenge is for speakers to begin with information that is easy to access and encode, allowing them to retrieve more difficult material during articulation and minimizing the need for pauses and other disfluencies. On this view, which is sometimes referred to as the Easy-First strategy, ordering decisions emerge as a byproduct of speakers’ attempts to accommodate the early placement of a linguistic expression. This incremental strategy is also thought to characterize multi-utterance production, which implies that the initial utterance of a discourse will reflect easily accessed or primed content. Using scene description tasks, we have developed a competing theory which assumes that speakers instead build a detailed macro-plan for an upcoming sequence of utterances that reflects the semantics of the scene. Our research shows that the order in which objects in a scene are described correlates with a specific aspect of object meaning, namely what we term “interactability”: the extent to which a human would be likely to interact with the object. We conclude that linearization decisions in language production are primarily driven not by an Easy-First strategy but instead emerge from a hierarchical plan that is based on a semantic representation of object affordances.
Since its beginning in the 1950s, the field of artificial intelligence has cycled several times between periods of optimistic predictions and massive investment (“AI Spring”) and periods of disappointment, loss of confidence, and reduced funding (“AI Winter”). Even with today’s seemingly fast pace of AI breakthroughs, the development of long-promised technologies such as self-driving cars, housekeeping robots, and conversational companions has turned out to be much harder than many people expected. One reason for these repeating cycles is our limited understanding of the nature and complexity of intelligence itself. In this talk I will discuss some fallacies in common assumptions made by AI researchers, which can lead to overconfident predictions about the field. I will also speculate on what is needed for the grand challenge of making AI systems more robust, general, and adaptable—in short, more intelligent.