Episode Banner Image

Episode 35 Kentucky Derby: Leveraging Data to Build False Confidence

In this episode, we explore how more data doesn't necessarily mean better results and what alternatives marketers have to successful metric-based outcomes.

Episode Highlights

Slovic Study

Psychologist Paul Slovic gathered a group of seasoned horse racing bettors to see how well they could predict the outcome of 40 horse races conducted over four rounds. The participants received any five pieces of information that they wanted on each horse, including things like age, fastest speed, and the jockey’s experience or weight. Each handicapper was then asked to both predict the winner of the race and their confidence in that outcome. However, the increased availability of data did not impact their accuracy. At the same time, their confidence nearly doubled, to 34%.

Gary Klein and Premortems

Unlike a typical critiquing session, in which project team members are asked what might go wrong with a project, a premortem operates on the assumption that something did go wrong. The team members’ task is to generate plausible reasons for the project’s failure. Psychologist Gary Klein ran a study using this technique and found that overconfidence in outcomes dropped by 25% when doing a premortem.

Robert Knox Study

Psychologist Robert Knox ran a study where he surveyed race bettors asking about their confidence levels before and after they placed their bets. His study found that bettors who were asked how confident they were about their bets after they placed them were 38% more confident than those that were asked before they placed them. He hypothesized that rather than attitude influencing behavior, behavior would impact attitude.