Schedule - Parallel Session 5 - Information and Experience
Engineering F1.10 (note the change of room from F105-106 to F1.10) - 14:00 - 15:30Cases in Memory, Decisions from Experience, and Black Swans
Ilke Aydogan; Yu Gao
Abstract
This paper investigates the putative underweighting of rare and extreme events – so called “black swans”- to understand the gap between decisions from experience (DFE) and decisions from description (DFD). We first resolve the problem of lack of control over experienced probabilities by adjusting the sampling paradigm of Hertwig et al. (2004). Accordingly, our subjects were required to draw complete samples without replacement from finite outcome distributions. By doing so, they acquired complete knowledge of probabilities and outcomes. Our experimental design also controlled for utility, which enabled us to observe the true weightings of probabilities. Although our results confirmed the well-known gap, they did not provide evidence for underweighting of small probabilities. Overall, our findings suggested a clear de-biasing effect of sampling experience: while it attenuates “rather than reverses “cognitive deviations from Expected Utility (reduced likelihood insensitivity), it has no impact on motivational deviations (persistent pessimism).
Bringing New Medicines to Market Sooner? Bayes Decision Theoretic Modelling of Sequential Trials
Martin Forster; Stephen Chick; Paolo Pertile
Abstract
Increasing pressure on health care budgets has led to extensive innovation in clinical trial design and health technology assessment: adaptive designs seek to alter the balance of allocation of patients to treatments as trial evidence accumulates, sequential designs permit early stopping when evidence becomes sufficiently persuasive. Despite such progress, much work either follows the frequentist tradition, defining stopping rules via error spending functions, or implements Bayesian policies based on posterior probabilities of selecting the best alternative. We present a Bayesian decision-theoretic model [1-3] of a sequential clinical trial which fully accounts for the costs of carrying out research, benefits accruing to patients, the cost of switching technologies and the size of the population to benefit. The model yields an optimal stopping policy based on the solution of a free boundary problem in which the investigator learns about comparative cost-effectiveness as trial evidence accumulates. It provides a unified policy defining optimal `do not experiment’/`fixed sample size experiment’/`sequential experiment’ decisions as a function of prior information. The important matter of delay in the arrival of primary endpoints is dealt with using predictive distributions for the posterior reward once observations on the trial’s `pipeline’ patients have arrived. We use a published trial investigating the clinical- and cost-effectiveness of drug-eluting stents versus bare metal stents to illustrate the potential gains from the approach, in terms of the maximisation of net benefits of the trial and adoption decision. Of particular interest is that the model does not seek to minimise expected sample size, a traditional statistical objective; rather it commits additional patients (in expectation) when the benefit of learning by sampling is highest. Hence the design does not always imply earlier stopping, in expectation. We illustrate how longer delays in observing the primary endpoint reduce the benefits of learning over time, and how smaller populations imply earlier stopping, so that adoption decisions will be made earlier for rare diseases.
Information Communication and Quality of Risky Decisions: Reinterpreting the `D-E Gap'
Orestis Kopsacheilis; Robin Cubitt; Chris Starmer
Abstract
`A person in need of serious surgery must consent to undergo general anaesthesia. Prior to the planned operation the scared patient searches on-line for the associated risks and discovers that the number of deaths as a result of general anaesthesia has reached the stunning figure of 5.4 deceased per 100,000 patients. Panicking, the patient calls the doctor who points out that 0.0054% is an extremely small chance and that she would no doubt take it if she was in need of such an intervention. Nevertheless, the patient seriously considers foregoing the operation altogether.’ Why is the opinion of the doctor at odds with that of the patient? A predominant stipulation as to why this conflict arises is that people’s risky decisions depend on whether they are made based on described outcomes and probabilities (patient) or whether they learn about them through experience (doctor). In particular, when people make decisions from description, they tend to overweight rare events. This is in accord with Prospect Theory’s inverse S-shaped probability weighting function. Conversely, when making decisions from experience people act as if they underweight rare events. This disparity is referred to as the `Description-Experience gap’. This study extends the set of questions by addressing the following: Which type of decision is more beneficial to the patient’s well-being? We conduct between-subjects laboratory experiments and elicit – at the individual level – Prospect Theory’s components in two contexts: Decisions from Description and Decisions from Experience. In `Description’ subjects learn about the properties of lotteries via objectively defined numerical representations. In `Experience’ subjects infer those properties by pressing on unlabelled buttons and observing their associated outcomes. In Study 1, deviating from the standard sampling paradigm, we include a history table that records sampled outcomes. We use Expected Utility as a benchmark for the quality of decisions and employ two measures to characterize them. First, we use an index measuring the rate of violations of the Independence Axiom. Second, we use certainty equivalents to elicit individual weighting functions and examine their proximity to linearity across the two contexts. Firstly, we find that by including a history table in `Experience’, subjects’ memory constraints are alleviated, which leads to an unprecedented increase of the sampling amount. Secondly, we observe only partial support for the underweighting hypothesis. Thirdly and perhaps more intriguingly, we find evidence supporting the claim that subjects in `Experience’ behave more like Expected Utility maximisers than those in `Description’. In Study 2, we disseminate between the most prominent culprits of the `Description-Experience gap’ (sampling bias, ambiguity aversion, memory constraints and presentation format) by examining their partial contribution to the shape of individuals’ weighting functions.
Wait, Wait, Don't Tell Me: Repeated Choices with Clustered Feedback
David Hagmann; Cleotilde Gonzalez
Abstract
When decision makers repeatedly choose between safe risky assets, their decision can depend on how frequently they receive feedback [1]. Someone who invests in stocks, for example, can expect to see a larger gain after a year than someone who invests in bonds. However, on any given day, she is more likely to experience a loss, as stock returns exhibit greater volatility. A loss-averse decision maker may prefer the safer asset if she receives feedback every day, but the risky asset if she receives feedback annually. Decision makers choose differently when they have to learn about outcomes by receiving feedback from experience than when outcomes and probabilities are given explicitly [2]. Feedback is instrumentally valuable and learning about a good option early on allows a decision maker to exploit the option, rather than continue exploring an inferior alternative. However, experiencing an unfavorable outcome early on may lead her to give up too quickly and miss options that are more rewarding. The present study explores whether giving participants less frequent feedback can improve ex post outcomes and whether this can close the decision-experience gap. Participants (n=1,200) in a 2x2x3 design make 110 static, binary decisions and earn the sum of all realized outcomes. On the first dimension, we vary whether decision makers receive ex ante a description of the two options. On the second dimension, we vary how frequently participants receive feedback about the realization of the selected option: either immediately after each choice or in blocks after every 10 choices. Immediate feedback has been the standard form of feedback in decisions from experience paradigms [3]. In the novel clustered feedback condition, participants make 10 decisions without seeing any outcomes, then observe the individual realizations of the past 10 trials at once. We vary on the third dimension the structure of the lottery-pairs, each consisting of a safe and a risky option. The safe option in all lotteries provides a guaranteed payoff of 4. The risky option has a fixed expected value of 5 and includes two outcomes: a low payoff of 0 and a high payoff that is different in every lottery-pair. The high payoff is either 6.25 with 80% probability (low-variance condition), 10 with 50% probability (medium-variance), or 25 with 20% probability (high-variance). In decisions from experience, the proportion of risky choices is substantially higher with clustered feedback compared to immediate feedback in the medium and high variance lotteries (both p < 0.001), but not in the low-variance lottery. The effect size is substantial, shifting decisions by about 20 percentage points toward the risky option. However, we find no significant difference across the feedback conditions with description. The net effect is that clustering feedback closes the description-experience gap and improves ex post outcomes.