Kemmerly et al.: Confident without being right, people bet more and searched more — Fukai Reads

TL;DR

How much do you trust what you know? The part of that confidence your actual accuracy cannot explain is what this paper calls "unjustified confidence." Kemmerly and colleagues asked 997 US adults 40 questions about college football results, collecting a confidence rating after each one. They then watched how the same people behaved when betting play money.

Even after statistically removing differences in accuracy, more confident people bet more (correlation r = .18) and bought paid information more often (r = .10). Yet confidence was barely related to how much they revised their guesses after seeing information (r = −.04). Emotionally, they felt more empowered and more decisive — but not any more or less open to new information.

Translated into puzzle terms: people who are confident without being right commit to answers boldly and reach for hints, but do not necessarily change their minds because of them. This is a correlational study, though, so it cannot tell us which way cause and effect run.The Case of the Golden Idol (Color Gray Games, 2022). A deduction puzzle in which you fill every blank before the game judges your answer — how you hold your confidence shows directly in how you play. Image: Steam store screenshot

Who wrote it, and where?

There are four authors. First author Rowan Kemmerly is in the Department of Psychology at Rutgers University–Newark. Andrew M. Parker is at the RAND Corporation, Annie H. Somerville at the law firm King & Spalding, and Eric R. Stone in the Department of Psychology at Wake Forest University. Parker and Stone have co-authored work on confidence and knowledge before (the paper cites Parker & Stone, 2014).

It appears in Judgment and Decision Making, volume 21, article e16 — a peer-reviewed paper published online on 10 July 2026 under a CC BY 4.0 licence. The measures, exclusion criteria and variable calculations were preregistered on OSF (the analysis plan was made public before the data came in). It has been out a little over two months and has not yet been widely discussed.

I picked it today because it is the mirror image of the McCaughey et al. paper on information search costs that I covered earlier (read it here). That study asked how price changes the way people search. This one asks whether, at the same price, confidence changes it. Anyone designing hints to sell or to place will get more from reading the two side by side.

What we already knew about miscalibrated confidence

That people's confidence often fails to match their actual knowledge is one of the longest-established findings in judgment and decision-making research. The paper splits it into two parts. First, confidence tends to be too high overall (overconfidence). Second, the difference between more and less confident people tracks the difference in what they actually know only loosely.

The open question is how that gap shows up in behaviour. People act because they feel confident, so unjustified confidence has long been assumed to degrade decisions. But, the authors argue, most prior work put outcomes — scores, gains, losses — at the centre and treated confidence as just one of their causes.

The authors flip that around. They put confidence at the centre and look at how it relates to several feelings and behaviours at once. That way they can catch contradictions: confidence helping along one pathway and hurting along another. Look only at outcomes and the two can cancel out and disappear.

A second question is whether confidence in knowledge differs from confidence in forecasts — knowing what has already happened versus predicting what will. The same feeling of certainty might behave differently in each. The paper compares both using the very same match-ups.

How they measured unjustified confidence

Participants came from a Qualtrics online panel. Of 1,976 people contacted, 517 with no interest in football were screened out. From the 1,118 who consented, those who finished implausibly fast or left data incomplete were removed, leaving 997 in the analysis. Data were collected on 20–21 September 2021.

Participants were randomly assigned to one of two conditions. In the knowledge condition they answered 40 questions about which of two college teams had won a game in the previous (2020–21) season. In the forecasting condition they predicted winners of the same 40 pairings for the 2021–22 season, which had only just begun. In both, they rated their confidence after each item from 50% (a pure guess) to 100% (certain), in 10% steps.

The key move is how unjustified confidence is isolated. The authors statistically removed the part of confidence that actual accuracy could explain; what remains is unjustified confidence. (The technique is partial correlation — the strength of a relationship after the influence of another variable is taken out.)

Next came a questionnaire on feelings: empowerment, decisiveness and openness to information, each measured with six items on a five-point scale.

Last came a behavioural task. Starting with $20 in play money, participants played 16 rounds, predicting teams' end-of-season rankings (the Sagarin ratings) and betting between $0.10 and $1.00. A correct prediction doubled the bet; a badly wrong one lost it. In the first eight rounds, the previous year's ranking was shown free after the initial guess, and they could revise. In the last eight, they could choose to buy that same information for 10 cents. To make them take it seriously, 1% of participants were paid their play-money balance in real money.

Three behaviours come out of this task. First, the average bet (risk taking). Second, how far guesses moved toward the free prior-year ranking (information use). Third, the share of paid rounds in which they spent 10 cents on information (information search).

What they found

First, the miscalibration itself. Mean accuracy was 52% — barely above the 50% of pure guessing — while mean confidence was 72%. The paper reports overconfidence of 21 percentage points. Confidence was higher in the forecasting condition (mean .74 vs .70), and overconfidence slightly larger (.22 vs .20).

Confidence also tracked accuracy only weakly: r = .15 in the knowledge condition and r = −.02 — essentially zero — in the forecasting condition. When predicting the future, more confident people were not more likely to be right.

Feelings next. With accuracy partialled out, unjustified confidence correlated with empowerment at r = .25 and decisiveness at r = .20. But its relationship with openness to information was r = .05, not statistically significant. These data do not support the idea that overconfident people are more stubborn.

Then behaviour. Unjustified confidence correlated with bet size at r = .18 (.15 for knowledge, .23 for forecasting) and with buying paid information at r = .10. The latter, though, was r = .14 in the knowledge condition and a non-significant .06 in the forecasting condition. Its relationship with revising guesses after free information was r = −.04 overall, and a slightly negative r = −.09 in the forecasting condition alone.

The authors also ran multiple regressions (entering several predictors together to see each one's unique contribution). Confidence was the strongest predictor of bet size (β = .16), with empowerment adding a little (β = .10). For buying information, confidence was positive (β = .10) and openness negative (β = −.09): people who described themselves as open to information bought less of it.

Mind the sizes, though. All the correlations sit between about .10 and .25 — small. The regression models explained 4.2% of the variation in betting and 1.7% in information buying. The accurate reading is: the tendencies are real, but most of what people did was driven by other things.

How puzzle and game makers can use this

The paper uses football betting, but its skeleton is "how confident are you before answering" combined with "do you go get information, and do you use it." That is very close to the shape of hint design in puzzles. What follows is my own proposal for bringing the results back to games (the paper itself does not study games).

(1) Measure hint purchases and hint effectiveness as separate metrics. Here, unjustified confidence went with buying information but had almost nothing to do with changing one's mind because of it. Hints selling well does not mean hints are helping. Log separately whether answers changed, or moved closer to correct, after a hint was viewed.

(2) Use the grain of answer-checking to surface miscalibration early. In Return of the Obra Dinn (Lucas Pope, 2018), fates are confirmed only when three are correct together; being sure of one answer gets you nothing back. Where confidence tracks correctness poorly, I read this kind of batched verdict as a way to reveal the gap before it becomes an expensive failure.Return of the Obra Dinn (Lucas Pope, 2018). Fates are only confirmed once three are correct together — confidence in a single answer is not enough to check it. Image: Steam store screenshot

(3) Assume confidence differences will ride straight into any stake the player sets. Where players choose whether to double a score or risk what they hold, those with more unjustified confidence will bet bigger. Because the relationship was stronger in the forecasting condition (r = .23), it may bite hardest where luck is involved. Offering a way to recover from a big loss keeps overconfident players from washing out early.

(4) Treat "knowledge" questions and "forecast" questions differently. Confidence behaved a little differently for answering from clues already on screen versus predicting something yet to happen; for forecasts, confidence and accuracy were essentially unrelated. A UI that asks players to enter a confidence level on predictive questions may give you little signal about who is actually right.

(5) Do not over-trust self-report. People who called themselves open to information bought less of it (β = −.09). A playtester who says "I'd use hints if they were there" may not. Collect both questionnaires and behaviour logs.

How far can we trust it?

(a) First, the weaknesses the authors acknowledge. The biggest is that accuracy was close to chance (mean 52%) and the reliability of the accuracy measure was extremely low (α = .17; an index of how consistently something is measured, where closer to 1 is better). The authors concede that adjusting for accuracy therefore makes little practical difference. In effect, "unjustified confidence" here is close to confidence itself.

The feeling scales also had only modest internal consistency (α = .53–.66). In the information-use measure, 14–15% of responses were undefined and around 30% were winsorized, and the measure was left out of the regressions. The domain is a single one — football betting — so generalisation is unknown. And the authors state plainly that the study is correlational and cannot establish causal direction.

(b) Now the points Fukai raises here. First, buying information and using information were measured in different rounds under different conditions — free in the first half, paid in the second. The study did not follow whether the same person used the same information they bought. So it cannot support the claim that overconfident people ignore hints they paid for.

Second, the bets used play money, with only 1% of participants paid in real money. Third, the data are from September 2021, just after the season began; forecasts were made while outcomes were genuinely open, so the two conditions are not perfectly matched. Fourth, everything in the use-cases section is my own analogy, not something tested in games.

Finally, the paper is new, and no replications or critiques by other researchers have appeared yet. Treat it as a single correlational study — one sheet of the map that points a direction.

Fukai's reading

This section alone is my opinion. I want to place this paper as a story about unjustified confidence acting on the accelerator, not the brake. Confidence did not change people's stance toward information (their openness); it only pushed up the volume of action — betting, buying. In the vocabulary of design criticism, hint-purchase rates and risk-taking rates measure a player's momentum, not the depth of their understanding. The designers most reassured by strong hint sales, as I read it, are the ones who should question that distinction first.

What to read next

To extend the map from here, start with two 2025 papers from the same group, both in the Journal of Behavioral Decision Making: an exploratory study by Parker and colleagues asking laypeople and researchers what they think confidence leads to, and a study by Stone and colleagues testing ways to raise or lower confidence (for this piece I checked their bibliographic details only). The present paper can be seen as the one in that line that measures the behavioural side.

Among papers covered on this site, McCaughey et al. on information search costs (article) and Liquin on curiosity and effort (article) sit right next door. Compare price, effort and confidence as three axes of "when do people go and get information," and the map of hint design becomes quite concrete.

From the game side, try a deduction game where you fill in the answer yourself before it is judged. Tsumiki's play diary of The Case of the Golden Idol (Color Gray Games, 2022), filling blanks "with what felt plausible before the evidence" (article), reads as this paper's unjustified confidence rendered through one player's hands.The Case of the Golden Idol (Color Gray Games, 2022). You collect clue words and place them into the blanks as your own deduction. Image: Steam store screenshot

References

Papers and materials referenced in this article:

・Examining potential behavioral and psychological consequences of unjustified confidence in knowledge and in forecasting (Rowan Kemmerly, Andrew M. Parker, Annie H. Somerville, Eric R. Stone, 2026, Judgment and Decision Making 21, e16, peer-reviewed, CC BY 4.0)

・Open-access full text (Cambridge Core)

・Preregistration (OSF)

・Related: Psychological and Behavioral Consequences of Confidence in Knowledge: An Exploratory Examination of General Public and JDM Researcher Perspectives (Parker et al., 2025, Journal of Behavioral Decision Making)

・Related: Influencing Confidence: Testing Ways to Increase or Decrease Confidence in Knowledge (Stone et al., 2025, Journal of Behavioral Decision Making)