Cognitive Science Program

Extracting the Wisdom of Crowds When Information is Shared (2019)
Asa B Palley and Jack B Soll
Management Science, 65 (5), 2291-2309

Abstract

Using the wisdom of crowds—combining many individual judgments to obtain an aggregate estimate—can be an effective technique for improving judgment accuracy. In practice, however, accuracy is limited by the presence of correlated judgment errors, which often emerge because information is shared. To address this problem, we propose an elicitation procedure in which respondents are asked to provide both their own best judgment and an estimate of the average judgment that will be given by all other respondents. We develop an aggregation method, called pivoting, which separates individual judgments into shared and private information and then recombines these results in the optimal manner. In several studies, we investigate the method and examine the accuracy of the aggregate estimates. Overall, the empirical data suggest that the pivoting method provides an effective judgment aggregation …

Lossed in translation: An off-the-shelf method to recover probabilistic beliefs from loss-averse agents (2016)
Theo Offerman and Asa B Palley
Experimental Economics, 19 (1), 30-Jan

Abstract

Strictly proper scoring rules are designed to truthfully elicit subjective probabilistic beliefs from risk neutral agents. Previous experimental studies have identified two problems with this method: (i) risk aversion causes agents to bias their reports toward the probability of <svg aria-label=" 1 / 2 " class="gs_fsvg" height="14px" style="vertical-align:-4px;" width="23px"><g transform="matrix(0.01400, 0.00000, 0.00000, 0.01400, 0.00000, 10.50000)"><g><g><path d="M 190 0 V 72 Q 446 72 446 137 V 1212 Q 340 1161 178 1161 V 1233 Q 429 1233 557 1364 H 586 Q 593 1364 599 1358 T 606 1346 V 137 Q 606 72 862 72 V 0 H 190 Z " transform="scale(0.48828, -0.48828)"></path><g transform="translate(500.00000, 0.00000)"><path d="M 115 -471 Q 115 -465 117 -463 L 829 1511 Q 833 1523 843 1529 T 866 1536 Q 884 1536 895 1525 T 907 1495 V 1487 L 195 -487 Q 183 -512 156 -512 Q 139 -512 127 -500 T 115 -471 Z " transform="matrix(0.48828, 0.00000, 0.00000, -0.48828, 55.55556, 0.00000)"></path></g><path d="M 102 0 V 55 Q 102 60 106 66 L 424 418 Q 496 496 541 549 T 630 671 T 699 811 T 725 963 Q 725 1047 694 1123 T 601 1246 T 453 1292 Q 364 1292 293 1238 T 193 1100 Q 201 1102 215 1102 Q 261 1102 293 1071 T 326 991 Q 326 944 293 911 T 215 879 Q 167 879 134 912 T 102 991 Q 102 1068 131 1135 T 214 1255 T 337 1336 T 483 1364 Q 600 1364 701 1314 T 861 1174 T 920 963 Q 920 874 881 794 T 781 648 T 625 500 T 500 389 L 268 166 H 465 Q 610 166 707 168 T 811 176 Q 835 202 860 365 H 920 L 862 0 H 102 Z " transform="matrix(0.48828, 0.00000, 0.00000, -0.48828, 1111.11108, 0.00000)"></path></g></g></g></svg>, and (ii) for moderate beliefs agents simply report <svg aria-label=" 1 / 2 " class="gs_fsvg" height="14px" style="vertical-align:-4px;" width="23px"><g transform="matrix(0.01400, 0.00000, 0.00000, 0.01400, 0.00000, 10.50000)"><g><g><path d="M 190 0 V 72 Q 446 72 446 137 V 1212 Q 340 1161 178 1161 V 1233 Q 429 1233 557 1364 H 586 Q 593 1364 599 1358 T 606 1346 V 137 Q 606 72 862 72 V 0 H 190 Z " transform="scale(0.48828, -0.48828)"></path><g transform="translate(500.00000, 0.00000)"><path d="M 115 -471 Q 115 -465 117 -463 L 829 1511 Q 833 1523 843 1529 T 866 1536 Q 884 1536 895 1525 T 907 1495 V 1487 L 195 -487 Q 183 -512 156 -512 Q 139 -512 127 -500 T 115 -471 Z " transform="matrix(0.48828, 0.00000, 0.00000, -0.48828, 55.55556, 0.00000)"></path></g><path d="M 102 0 V 55 Q 102 60 106 66 L 424 418 Q 496 496 541 549 T 630 671 T 699 811 T 725 963 Q 725 1047 694 1123 T 601 1246 T 453 1292 Q 364 1292 293 1238 T 193 1100 Q 201 1102 215 1102 Q 261 1102 293 1071 T 326 991 Q 326 944 293 911 T 215 879 Q 167 879 134 912 T 102 991 Q 102 1068 131 1135 T 214 1255 T 337 1336 T 483 1364 Q 600 1364 701 1314 T 861 1174 T 920 963 Q 920 874 881 794 T 781 648 T 625 500 T 500 389 L 268 166 H 465 Q 610 166 707 168 T 811 176 Q 835 202 860 365 H 920 L 862 0 H 102 Z " transform="matrix(0.48828, 0.00000, 0.00000, -0.48828, 1111.11108, 0.00000)"></path></g></g></g></svg>. Applying a prospect theory model of risk preferences, we show that loss aversion can explain both of these behavioral phenomena. Using the insights of this model, we develop a simple off-the-shelf probability assessment mechanism that encourages loss-averse agents to report true beliefs. In an experiment, we demonstrate the effectiveness of this modification in both eliminating uninformative reports and eliciting true probabilistic beliefs.

Sequential search and learning from rank feedback: theory and experimental evidence (2014)
Asa B Palley and Mirko Kremer
Management Science, 60 (10), 2525-2542

Abstract

This paper studies the effect of limited information in a sequential search setting where a single selection is to be made from a set of random potential options. We consider both a full-information problem, where the decision maker observes the exact value of each option as she searches, and a partial-information problem, in which the decision maker only learns the rank of the current option relative to the options that have already been observed. We develop a model that allows for a sharp contrast between search behavior in the two information settings, both theoretically and empirically. We present the results of an experiment that tests, and supports, the key prediction of our model analysis—limited information induces longer search. Our data further suggest systematic deviations from the theoretical benchmarks in both informational settings. Importantly, subjects in our partial-information conditions are prone to …

Decision strategies to reduce teenage and young adult deaths in the United States (2013)
Ralph L Keeney and Asa B Palley
Risk Analysis, 33 (9), 1661-1676

Abstract

This article uses decision analysis concepts and techniques to address an extremely important problem to any family with children, namely, how to avoid the tragic death of a child during the high‐risk ages of 15–24. Descriptively, our analysis indicates that of the 35,000 annual deaths among this age group in the United States, approximately 20,000 could be avoided if individuals chose readily available alternatives for decisions relating to these deaths. Prescriptively, we develop a decision framework for parents and a child to both identify and proactively pursue decisions that can lower that child's exposure to life‐threatening risks and positively alter decisions when facing such risks. Applying this framework for parents and the youth themselves, we illustrate the logic and process of generating proactive alternatives with numerous examples that each could pursue to lower these life‐threatening risks and possibly …