The “Plato's problem”-- how do people know as much as they do with as little information as they get?-- also known as “the poverty of the stimulus”, 搉egative evidence? or 搕he logical problem of language acquisition?, has aroused the interest of many philosophers, psychologists, linguists, and computational scientists. Nativism is the answer provided by Chomsky, but psychologists like MacWhinney and computational linguists like Sampson offer different explanation. Quine calls the problem “the scandal of induction”, whereas Shepard maintains that a general theory of generalization and similarity is as necessary to psychology as Newton's laws are to physics. However, the acceptance of the hereditary nature of language propensity does not mean the solution of the general theory of generalization and similarity--the problem of categorization. Many models have been suggested to find a mechanism by which a set of stimuli, words, or concepts come to be treated as similar. They attempt to postulate some constraints that can narrow the solution space of the problem that is to be solved by induction. Latent semantic analysis (LSA) put forth by Landauer et al is“a high-dimensional linear associative model that embodies no human knowledge beyond its general learning mechanism, to analyze a large corpus of natural text and generate a representation that captures the similarity of words and text passages.”The model employs a statistical technique of linear algebra known as singular value decomposition (SVD).
The input to LSA is a matrix {A} consisting of rows representing unitary event types by columns representing contexts in which instances of the event types appear. SVD then decomposes the matrix into three matrices: {A}={U}{w}{V}T, and reduction of dimensionality is carried out in the reconstruction of the original matrix. To illustrate the power of reduction of dimensionality, two examples are given. In the example given by Landauer, the text input is titles of nine technical articles, five about human-computer interaction, four about mathematical graph theory. LSA shows how in the two-dimensionally reconstructed matrices two words that were totally uncorrelated in the original are quite strongly correlated (r =.9) in the reconstructed approximation. The other example is the use of SVD in a preliminary study of the relationship among the errors by Chinese learners of English. Reduction of dimensionality offers a better explanation of trends of development of spelling errors, misuse of words, and syntactic construction among five different types of learners. LSA have a wide area of application in connection with text processing.