Transparency is often treated as a cure-all in AI design. When trust breaks down, the instinct is to explain more: longer rationales, more metrics, more detail. The assumption makes sense: if people can see how the system works, they’ll make better decisions.
Research on AI-assisted decision-making shows that additional explanation often increases user confidence faster than it improves decision accuracy (Buçinca et al., 2021).
In other words, people feel informed, but their judgment doesn’t improve.
More explanation is not always better
Human attention is limited and when explanations exceed what people can process, they stop evaluating and start accepting.
Studies of AI-supported decisions consistently find that longer explanations raise confidence even when decision quality stays flat or declines; a pattern linked to overreliance (Buçinca et al., 2021). This is the illusion of understanding: exposure to information is mistaken for comprehension.
The right explanation depends on context
There is no universally good explanation. What works depends on the situation and the user. Three factors matter most:
- Decision risk: High-stakes decisions need deeper justification and verifiable evidence. Low-stakes decisions benefit from short, clear summaries.
- User expertise: Experts want detail. Novices need clarity. Showing both the same explanation undermines judgment for at least one group.
- Confidence: When AI sounds certain, users scrutinize less. When uncertainty is visible, users slow down and engage more carefully.
Most systems ignore these differences and show the same explanation to everyone, every time.
Adaptive explanations work better
Research increasingly supports adaptive explanation over static transparency.
What this can look like:
- Start with a brief, decision-relevant summary
- Let users pull in more detail only if needed
- Keep raw data available without forcing it upfront
This preserves accountability while respecting cognitive limits and individual user needs.
A practical way to do this is the explanation ladder.
The explanation ladder
Level 1: Summary
- What the system noticed and why it matters.
- “This recommendation is based on a recent deviation from normal spending patterns.”
Level 2: Evidence
- The key data supporting that summary.
- “Spending increased 42% over the six-month baseline, concentrated in three categories.”
Level 3: Raw data
- The underlying records for verification.
- “Here are the transaction logs and time series.”
Most users never need Level 3. But knowing it’s there, and being able to access it, improves trust and judgment.
When explanations hurt decisions
Over-explaining changes how users behave. When explanations are too detailed by default, teams show predictable patterns:
- Less independent reasoning
- Fewer challenges or corrections
- Greater reliance on surface plausibility
These effects are strongest when explanations look rigorous. Dense text and long metric lists create a false sense of certainty, especially in ambiguous situations where human judgment matters most.
Good explanations don’t answer every question
The goal of explanation is judgment support, not persuasion.
When explanations work, users ask better questions, request evidence selectively, and override the AI more appropriately. When explanations fail, scrutiny drops while confidence rises.
Design takeaways for AI builders
If the goal is better human decisions, not just more transparency, three design rules matter:
- Default to summaries, not details: Start with what matters and why, making depth optional.
- Match explanation depth to risk and expertise: High stakes and expert users justify more detail.
- Design explanations to invite challenge: Show uncertainty, expose assumptions, and make follow-up easy.
The key takeaway is simple: If explanation increases confidence without improving accuracy, it isn’t transparency.
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 187. https://doi.org/10.1145/3449283
Koriat, A. (1997). Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General, 126(4), 349–370. https://doi.org/10.1037/0096-3445.126.4.349
