Teleosemantics for neural word embeddings

Teleosemantics is one of the most promising naturalistic theories of what makes something a representation. It explains the category of representation in terms of function; where a function is an effect which has been selected by an evolutionary process. The core idea is that certain states have evolved to represent how things stand in the world because cognitive or computational mechanisms have evolved to reliably produce and consume these states.

While teleosemantics was not developed to explain the contents of artificial neural networks, it has had some success in this domain (Shea, 2018, Buckner, 2021, Piccinini, 2022). This talk begins the project of applying teleosemantics to neural language models at the base, Tomáš Mikolov’s Word2Vec algorithm (Mikolov, 2013). Word2Vec was one the first widely-applied method for the production of dense word embeddings. These embeddings appear to track semantic features of expressions but are not directly interpretable.

While the main focus of the talk will be on this particular, relatively straight-forward framework of neural language modelling (i.e., no attention heads, no fine-tuning), a secondary aim is to indicate how teleosemantics can be applied to neural networks more generally and so we will engage with some debates about the role that intentions play in determining the function of artefacts and how we should individuate vehicles of representation in neural networks.