Diff what the UI says about the AI against what the code does — disclosure strings quoted, dropped confidence traced, retention claims checked against real payloads.
Free & open · no signup · read-only — it ends by asking · nothing leaves your machine
You are working inside this repo. Mission: diff the product's story about its AI against its code. Every claim the UI makes — "AI-generated", "we never store your data", an answer delivered with total confidence — is checkable here: find the disclosure strings that ship, the model calls they describe, and the gaps between them.
Read-only pass. Your only write is the report file.
Phase 2 sweeps the codebase through every one of these, citing file and line for each finding.
model output reaching the user with no cue in the rendering component; cite the component and the missing string
the API returns scores, citations, or caveats the UI discards; file:line where the signal dies
data-use claims ("not stored", "never trained on") diffed against the request the code sends and any retention or vendor training flag you can find
UI copy promising what the prompt or model config can't back: "understands", "always accurate", against temperature, truncation, and absent retrieval
model ids pinned in config while the UI implies one stable "AI"; behavior can change under users with no cue
the "talk to a human" or feedback affordance: trace where its handler actually sends things, or whether the button is decorative
One structured report at the repo root — or in reports/, if you keep one — the same shape every time, ready for a teammate — or the optional Studio — to act on.
Every Goal Prompt follows the same four steps, so results are consistent and repeatable — no matter which one you run.
Grep the shipped copy.
Label every finding measured (both sides cited from code) or suspected (one side inferred); an unlabeled finding doesn…
Rank by miscalibrated trust.
Create TRANSPARENCY.md at repo root.
Copy it and paste it into your agent inside the repo you want checked.
Install the goal plugin once — two commands — then just type /goal:model-transparency.
/plugin marketplace add GhostlyGawd/goal-prompts/plugin install goal@goal-promptsOr install only this Goal Prompt as /goal-model-transparency:
curl -fsSL https://goal-prompts.vercel.app/install | BRIEF=119 shLet an agent fetch it mid-conversation, or pull the raw Goal Prompt by URL.
https://goal-prompts.vercel.app/raw/119.mdNothing hidden — this is the whole Goal Prompt, verbatim. Read it in a minute, edit it, or copy it as-is.
# Goal: Model Transparency Audit
You are working inside this repo. Mission: diff the product's story about its AI against its code. Every claim the UI makes — "AI-generated", "we never store your data", an answer delivered with total confidence — is checkable here: find the disclosure strings that ship, the model calls they describe, and the gaps between them.
Read-only pass. Your only write is the report file.
## Phase 1 — Collect both sides of the diff
- Grep the shipped copy: disclosure strings, AI badges, tooltips, empty-state text, claims in templates ("AI", "generated", "assistant", "never trained on").
- Find the model calls: which code paths invoke a model, what the request payload actually contains, what comes back (confidence, citations, model ids), and what of it the UI keeps or drops.
- Map each AI touchpoint to the disclosure a user actually sees there — or to none.
## Phase 2 — Audit through 6 lenses
Label every finding **measured** (both sides cited from code) or **suspected** (one side inferred); an unlabeled finding doesn't ship.
1. **Undisclosed AI** — model output reaching the user with no cue in the rendering component; cite the component and the missing string
2. **Confidence dropped on the floor** — the API returns scores, citations, or caveats the UI discards; file:line where the signal dies
3. **Copy vs payload** — data-use claims ("not stored", "never trained on") diffed against the request the code sends and any retention or vendor training flag you can find
4. **Capability overstatement** — UI copy promising what the prompt or model config can't back: "understands", "always accurate", against temperature, truncation, and absent retrieval
5. **Version silence** — model ids pinned in config while the UI implies one stable "AI"; behavior can change under users with no cue
6. **Recourse reality** — the "talk to a human" or feedback affordance: trace where its handler actually sends things, or whether the button is decorative
## Phase 3 — Curate
- Rank by miscalibrated trust: an undisclosed AI decision outranks a missing model name.
- Write each fix as the artifact itself: the string to add, the field to surface, the claim to correct.
- Keep measured and suspected findings in separate ranks; every suspected one names the check that would confirm it.
## Phase 4 — Report
Create `TRANSPARENCY.md` at repo root:
1. **The honesty diff** — touchpoint · what the user is told (quoted, file:line) · what the code does (file:line) · gap
2. **Findings** — each: measured/suspected · lens · trust risk · the fix, written out
3. **Dropped signals** — the confidence, citations, and ids the backend has and the UI hides
4. **Priority** — the three corrections that most recalibrate user trust
Start the report with today's date. If `TRANSPARENCY.md` already exists from a previous run, read it first and lead with what changed since.
## Rules
- Quote the string and cite the call — a claim about the product's honesty must itself show its evidence
- Under-disclosure and over-warning are both findings; noise teaches users to ignore the truth
- No model output reaching a user in this repo? Say so in a one-paragraph null report and stop — a null result is a valid finding.
- If a `reports/` directory exists at the repo root, write the report there instead of the root.
- Before asking, present the top findings as a ranked list in plain words
- Report only — end by asking which transparency gaps to close firstreports/ directory exists at the repo root, write the report there instead of the root.Find the code that ranks, scores, or filters people, list the exact fields it consumes, and check whether any eval ever sliced results by group — suspicions labeled.
Whether the product's AI outputs are grounded in real sources or confidently invented — and whether it cites, hedges, and refuses when it should.
Attack the product's own AI like an adversary — make it ignore instructions, leak secrets, or misuse tools — and find where untrusted input can hijack it.
Whether the data used to train, fine-tune, or retrieve is licensed, clean, and free of the PII and contamination that create legal and quality risk.
Runs inside these playbooks — curated sequences you can launch with one paste: