AI-UX — does the human trust it?
Brief 44

Trust Calibration Audit

Confidence theater vs honest hedging — uncertainty channels, citation integrity, and what the product does after being wrong.

TRUST.md4 phases7 lenses~2.8k chars

Free & open · no signup · read-only — it ends by asking · nothing leaves your machine

What it does

You are working inside this repo. Mission: audit whether this product's expressed confidence matches its actual reliability — and fix the places where certainty is theater.

Read-only pass. Your only write is the report file.

What it looks for

7 lenses it looks through

Phase 2 sweeps the codebase through every one of these, citing file and line for each finding.

1

Confidence theater

uniformly authoritative tone regardless of task difficulty; prompts or post-processing that strip the model's own hedges

2

Uncertainty channels

any mechanism to express "not sure" — phrasing, scores, abstention — and whether it correlates with actual error or is cosmetic

3

Citation integrity

sources displayed: do they exist, and do they support the sentence they decorate? Spot-check the mechanism, not just the rendering

4

Abstention design

can the agent decline, ask, or escalate instead of guessing — and does the UX punish declining so hard the model never does

5

Failure honesty

after a wrong answer: correction visible, silent edit, or never acknowledged at all

6

Capability framing

what onboarding and empty states promise vs what evals show it delivers; the gap is churn in incubation

7

Trust repair

the flow after a user catches an error: apology-and-fix path, or nothing (see 45)

The deliverable

What lands in your repo

One structured report at the repo root — or in reports/, if you keep one — the same shape every time, ready for a teammate — or the optional Studio — to act on.

TRUST.md
1
Surface inventory
surface · current signal · reality it should reflect
2
Theater findings
with verbatim examples
3
Calibration fixes
where uncertainty enters, how it renders, what triggers abstention
4
Citation fix plan
from decoration to verification
5
Copy rewrites
worst 5, before/after
One file. Evidence-backed. It ends by asking before touching anything.
How it works

What it makes the agent do

Every Goal Prompt follows the same four steps, so results are consistent and repeatable — no matter which one you run.

1
Phase 1

Collect the confidence surfaces

Everywhere the product signals capability or certainty.

2
Phase 2

Audit through 7 lenses

Confidence theater — uniformly authoritative tone regardless of task difficulty.

3
Phase 3

Curate

Rank by damage.

4
Phase 4

Report

Create TRUST.md at repo root.

Get started

Three ways to run this Goal Prompt

01 · COPY

Paste it in

Copy it and paste it into your agent inside the repo you want checked.

02 · INSTALL

As a slash command

Install the goal plugin once — two commands — then just type /goal:trust-calibration-audit.

1/plugin marketplace add GhostlyGawd/goal-prompts
2/plugin install goal@goal-prompts

Or install only this Goal Prompt as /goal-trust-calibration-audit:

curl -fsSL https://goal-prompts.vercel.app/install | BRIEF=44 sh
03 · AGENT

From an agent (MCP)

Let an agent fetch it mid-conversation, or pull the raw Goal Prompt by URL.

https://goal-prompts.vercel.app/raw/44.md
Transparency

The exact prompt

Nothing hidden — this is the whole Goal Prompt, verbatim. Read it in a minute, edit it, or copy it as-is.

Read the full brief (2,776 characters)
# Goal: Trust Calibration Audit

You are working inside this repo. Mission: audit whether this product's expressed confidence matches its actual reliability — and fix the places where certainty is theater.

Read-only pass. Your only write is the report file.

## Phase 1 — Collect the confidence surfaces
- Everywhere the product signals capability or certainty: output tone, disclaimers, onboarding promises, empty states, error copy, marketing strings in the UI.
- What does the system actually know about its own reliability — eval scores (34), failure rates (37) — and does any surface reflect it?

## Phase 2 — Audit through 7 lenses
1. **Confidence theater** — uniformly authoritative tone regardless of task difficulty; prompts or post-processing that strip the model's own hedges
2. **Uncertainty channels** — any mechanism to express "not sure" — phrasing, scores, abstention — and whether it correlates with actual error or is cosmetic
3. **Citation integrity** — sources displayed: do they exist, and do they support the sentence they decorate? Spot-check the mechanism, not just the rendering
4. **Abstention design** — can the agent decline, ask, or escalate instead of guessing — and does the UX punish declining so hard the model never does
5. **Failure honesty** — after a wrong answer: correction visible, silent edit, or never acknowledged at all
6. **Capability framing** — what onboarding and empty states promise vs what evals show it delivers; the gap is churn in incubation
7. **Trust repair** — the flow after a user catches an error: apology-and-fix path, or nothing (see 45)

## Phase 3 — Curate
- Rank by damage: overconfidence on high-stakes outputs first
- Every fix is concrete copy, a rendering change, or a mechanism — not "be more honest"

## Phase 4 — Report
Create `TRUST.md` at repo root:
1. **Surface inventory** — surface · current signal · reality it should reflect
2. **Theater findings** — with verbatim examples
3. **Calibration fixes** — where uncertainty enters, how it renders, what triggers abstention
4. **Citation fix plan** — from decoration to verification
5. **Copy rewrites** — worst 5, before/after

Start the report with today's date. If `TRUST.md` already exists from a previous run, read it first and lead with what changed since.

## Rules
- Calibrated trust beats maximal trust: users who know when to check stay longer
- Never let styling erase the model's honest doubt
- No AI output shown to users in this repo? Say so in a one-paragraph null report and stop — a null result is a valid finding.
- If a `reports/` directory exists at the repo root, write the report there instead of the root.
- Before asking, present the top findings as a ranked list in plain words
- Report only — end by asking which fixes to make

House rules for this Goal Prompt

  • Calibrated trust beats maximal trust: users who know when to check stay longer
  • Never let styling erase the model's honest doubt
  • No AI output shown to users in this repo? Say so in a one-paragraph null report and stop — a null result is a valid finding.
  • If a reports/ directory exists at the repo root, write the report there instead of the root.
  • Before asking, present the top findings as a ranked list in plain words
  • Report only — end by asking which fixes to make
Keep exploring

More AI-UX briefs