Neural compositionality

Description

The idea of compositionality is that mental representations are built out of parts and possess a meaning that is derived from the meanings of the parts and how the parts are combined (Fodor & Pylyshyn, 1988; Pinker, 1997). A symbolic expression exhibits what is called concatenative compositionality, which means that the expression incorporates its constituents without changing them. Compositionality makes it possible for symbolic propositions to express the hierarchical, tree-like structure of sentences in a natural language. Because neural networks supposedly cannot express compositionality, it is claimed that they cannot simulate comprehension and production of language, nor other forms of thought, at least some of which are considered to be language like. In response, it has been argued that current neural networks exhibit a unique functional form of compositionality that may be able to model the compositional character of cognition even if the constituents are altered when composed into a complex expression (van Gelder, 1990). Here the original constituents are retrievable from the complex expression by natural connectionist means. For example, an encoder network can learn to encode simple syntactic trees on distributed hidden unit representations and then decode them back into the same syntactic trees at the outputs (Pollack, 1990).

A new algorithm called knowledge-based cascade-correlation (KBCC) (Shultz & Rivest, 2001) recruits previously learned source networks as well as single hidden units, perhaps implementing a new kind of neural compositionality in which recruited components are preserved intact as they are combined into an overall solution of the target task, as in classical concatenative compositionality. This project involves testing KBCC as a model of human performance on a number of compositional tasks, contrasting it with other connectionist models such as Pollack’s (1990).

Qualifications

Sufficient facility in Java to use a neural-network simulator. Ability to read cognitive-science sources on compositionality and to use statistical packages such as SPSS for ANOVA and PCA.

References

Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 3-71.

Pinker, S. (1997). How the mind works. New York: Norton.

Pollack, J. (1990). Recursive distributed representations. Artificial Intelligence, 46, 77-105.

Shultz, T. R., & Rivest, F. (2001). Knowledge-based cascade-correlation: Using knowledge to speed learning. Connection Science, 13, 1-30.

van Gelder, T. (1990). Compositionality: A connectionist variation on a classical theme. Cognitive Science, 14, 355-364.