Applying Cognitive Principles to Model-Finding Output: The Positive Value of Negative Information
Tristan Dyer, Tim Nelson, Kathi Fisler, Shriram Krishnamurthi
ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages & Applications, 2022
Abstract
Model-finders, such as SAT/SMT-solvers and Alloy, are used widely both directly and embedded in domain- specific tools. They support both conventional verification and, unlike other verification tools, property-free exploration. To do this effectively, they must produce output that helps users with these tasks. Unfortunately, the output of model-finders has seen relatively little rigorous human-factors study.Conventionally, these tools tend to show one satisfying instance at a time. Drawing inspiration from the cognitive science literature, we investigate two aspects of model-finder output: how many instances to show at once, and whether all instances must actually satisfy the input constraints. Using both controlled studies and open-ended talk-alouds, we show that there is benefit to showing negative instances in certain settings; the impact of multiple instances is less clear. Our work is a first step in a theoretically grounded approach to understanding how users engage cognitively with model-finder output, and how those tools might better support users in doing so.
Paper
These papers may differ in formatting from the versions that appear in print. They are made available only to support the rapid dissemination of results; the printed versions, not these, should be considered definitive. The copyrights belong to their respective owners.