Deduction as stochastic simulation

    We present a system for building mental models of quantified assertions. This system implements constraints on the size of model, the sorts of individual it represents, and on the likelihood of a search for alternative models. It yields quantitative predictions at a fine-grained level, and they fit data from experiments on quantificational reasoning better than previous accounts.

    Artwork credit: Droplet from Neil Banas’s Rain and Flow (2012) exhibition.
    Many theorists argue that deduction is based on the construction of mental models or simulations of descriptions.  Individuals tend to reason intuitively from a single mental model, but on occasion they make a deliberate search for alternative models. Previous computer implementations of the theory were deterministic, but evidence from empirical studies suggested that a stochastic algorithm would have greater predictive power. We developed such a system for inferences from assertions with single quantifiers, such as, “All the agents are lawyers.

    Building mental models

    The mental model theory of higher cognition postulates that reasoning relies on the construction and manipulation of mental models, i.e., iconic simulations of possible situations (Johnson-Laird, 2006). As the theory posits, reasoners who are told that some of the agents are lawyers simulate the situation, that is, they build a model of a small arbitrary number of tokens that denote individual agents, and then add the property of being a lawyer to some of the agents. A model of such a situation is akin to the following diagram, in which each row denotes the properties of an individual:
    [sourcecode language=”text”]
    agent lawyer
    agent lawyer
    agent
    agent
    [/sourcecode]
    Reasoners can immediately make deductions by inspecting the model. For example, the model shows that some of the agents are not lawyers.

     

    The model-building system embodies three stochastic parameters. The first parameter constrains the size of a model, that is, the number of different tokens representing entities that it contains. The second parameter constrains the contents of a model, and in particular the different sorts of entities that it represents. The third parameter constrains the likelihood that the system searches for an alternative to the initial model of premises. The motivation for each of these parameters is provided in Khemlani, Trafton, and Johnson-Laird (2013).
    The system is prototyped in R, and its source code is available at the bottom of the page. The code provides several functions for building models. The build function takes a single abbreviated quantified assertion as input. For example, “Iab” is often used to denote the quantified expression, “Some of the As are Bs”. Every time the build function is called with “Iab” as input, it outputs a model in which some of the As are in fact Bs:
    [sourcecode language=”R”]
    build(“Iab”)
    # [1] ” A B” ” A -B” ” A -B” “-A -B” “-A -B”
    build(“Iab”)
    # [1] ” A B” ” A B” ” A B” ” A -B” ” A -B” “-A B” “-A -B”
    build(“Iab”)
    # [1] ” A B” ” A B” ” A -B” “-A -B”
    [/sourcecode]
    Of course, simply building models is the first and most critical step. The second step is for the system to analyze the model to draw deductive conclusions. It is designed to make immediate inferences. The function immediateInference outputs an array of information about the system’s prediction of a particular inference. For example, this function call:
    [sourcecode language=”R”]
    immediateInference(“0a”, “Iab”, “Aab”)
    [/sourcecode]
    describes an immediate inference akin to the following reasoning problem:

    Suppose some of the As are Bs.
    Is it necessary that all of the As are Bs?

    because “Iab” is an abbreviation of the first premise and “Aab” is an abbreviation of the second premise. The task (“Is it necessary that…”) is governed by the particular experimental data being modeled, in this case Experiment 0a, which corresponds to Newstead and Griggs (1983, Experiment 1). When the immediate inference is executed, it yields this output:
    [sourcecode language=”R”]
    # [1] “Iab” “Aab” ” A B | A B | A B | A B | A -B |”
    # [4] “Simulation” “FALSE” “1”
    # [7] “One” “0.4” “0.3”
    # [10] “4”
    [/sourcecode]
    This output can be used for further analyses of the system’s performance. It provides, in order:

    1. The first premise
    2. The second premise
    3. The particular model that the stochastic system constructed
    4. Whether the system is in “Simulation” or “Data” mode
    5. The correct answer (in this case, the correct answer is “FALSE”, i.e., it’s not necessary that all of the As are Bs)
    6. The number of models that were needed to derive the solution (in this case, the initial model sufficed)
    7. Whether the system is a zero-, one-, or multiple-model problem (see Khemlani, Lotstein, Trafton, and Johnson-Laird, 2012)
    8. The sigma parameter value (see the paper above)
    9. The epsilon parameter value (ditto)
    10. The lambda parameter value (ditto)

    The code also includes a runExperiment function that takes an experimental design description (see above), the number of synthetic datasets to be output, and the system parameters. Here is what calling the runExperiment function looks like:
    [sourcecode language=”R”]data0a <- runExperiment(“0a”, N=100, sigma=.4, epsilon=.3, lambda=4)
    # Running experiment 0a with parameters σ = 0.4 ϵ = 0.3 λ = 4……………………………………………………………………………………….done.[/sourcecode]
    The data is then stored in data0a and can be analyzed statistically as one sees fit. Mechanisms for grid-based parameter searching are also provided in the source code.

     

    The system provides an excellent account of immediate inference data across five experiments. For instance, Figure 1 shows the proportion of correct responses in the observations (histograms with error bars) and predictions (circles) in the two studies as a function of the type of inference:
    [lightbox link=”http://mentalmodels.princeton.edu/skhemlani/files/2013/06/Fig2.png”][/lightbox]
    Figure 2 shows the system’s fit for each of the inferences:
    [lightbox link=”http://mentalmodels.princeton.edu/skhemlani/files/2013/06/Fig3.png”][/lightbox]
    Both figures reveal that the system closely matches the experimental data.

     

    The code for the prototype stochastic model building and immediate inference system can be downloaded here:
    http://mentalmodels.princeton.edu/programs/Stochastic-Immediate-Inferences.R

     
    • Khemlani, S., & Johnson-Laird, P. N. (2013). The processes of inference. Argument & Computation, 4, 1-20.
    • Khemlani, S., Lotstein, M., Trafton, J., & Johnson-Laird, P. N. (under review). Immediate inferences from quantified assertions.
    • Khemlani, S., Trafton, J. G., & Johnson-Laird, P. N. (2013). Deduction as stochastic simulation. In R. West & T. Stewart (Eds.), Proceedings of the 12th International Conference on Cognitive Modeling.
    • Newstead, S.E., & Griggs, R.A. (1983). Drawing inferences from quantified statements: A study of the square of opposition. Journal of Verbal Learning and Verbal Behavior, 22, 535–546.