Home/Catalog/AI-Ethics
AI-Ethics — is the AI responsible?
Brief 119

Model Transparency Audit

Diff what the UI says about the AI against what the code does — disclosure strings quoted, dropped confidence traced, retention claims checked against real payloads.

TRANSPARENCY.md4 phases6 lenses~3.5k chars

Free & open · no signup · read-only — it ends by asking · nothing leaves your machine

What it does

You are working inside this repo. Mission: diff the product's story about its AI against its code. Every claim the UI makes — "AI-generated", "we never store your data", an answer delivered with total confidence — is checkable here: find the disclosure strings that ship, the model calls they describe, and the gaps between them.

Read-only pass. Your only write is the report file.

What it looks for

6 lenses it looks through

Phase 2 sweeps the codebase through every one of these, citing file and line for each finding.

1

Undisclosed AI

model output reaching the user with no cue in the rendering component; cite the component and the missing string

2

Confidence dropped on the floor

the API returns scores, citations, or caveats the UI discards; file:line where the signal dies

3

Copy vs payload

data-use claims ("not stored", "never trained on") diffed against the request the code sends and any retention or vendor training flag you can find

4

Capability overstatement

UI copy promising what the prompt or model config can't back: "understands", "always accurate", against temperature, truncation, and absent retrieval

5

Version silence

model ids pinned in config while the UI implies one stable "AI"; behavior can change under users with no cue

6

Recourse reality

the "talk to a human" or feedback affordance: trace where its handler actually sends things, or whether the button is decorative

The deliverable

What lands in your repo

One structured report at the repo root — or in reports/, if you keep one — the same shape every time, ready for a teammate — or the optional Studio — to act on.

TRANSPARENCY.md
1
The honesty diff
touchpoint · what the user is told (quoted, file:line) · what the code does (file:line) · gap
2
Findings
each: measured/suspected · lens · trust risk · the fix, written out
3
Dropped signals
the confidence, citations, and ids the backend has and the UI hides
4
Priority
the three corrections that most recalibrate user trust
One file. Evidence-backed. It ends by asking before touching anything.
How it works

What it makes the agent do

Every Goal Prompt follows the same four steps, so results are consistent and repeatable — no matter which one you run.

1
Phase 1

Collect both sides of the diff

Grep the shipped copy.

2
Phase 2

Audit through 6 lenses

Label every finding measured (both sides cited from code) or suspected (one side inferred); an unlabeled finding doesn…

3
Phase 3

Curate

Rank by miscalibrated trust.

4
Phase 4

Report

Create TRANSPARENCY.md at repo root.

Get started

Three ways to run this Goal Prompt

01 · COPY

Paste it in

Copy it and paste it into your agent inside the repo you want checked.

02 · INSTALL

As a slash command

Install the goal plugin once — two commands — then just type /goal:model-transparency.

1/plugin marketplace add GhostlyGawd/goal-prompts
2/plugin install goal@goal-prompts

Or install only this Goal Prompt as /goal-model-transparency:

curl -fsSL https://goal-prompts.vercel.app/install | BRIEF=119 sh
03 · AGENT

From an agent (MCP)

Let an agent fetch it mid-conversation, or pull the raw Goal Prompt by URL.

https://goal-prompts.vercel.app/raw/119.md
Transparency

The exact prompt

Nothing hidden — this is the whole Goal Prompt, verbatim. Read it in a minute, edit it, or copy it as-is.

Read the full brief (3,471 characters)
# Goal: Model Transparency Audit

You are working inside this repo. Mission: diff the product's story about its AI against its code. Every claim the UI makes — "AI-generated", "we never store your data", an answer delivered with total confidence — is checkable here: find the disclosure strings that ship, the model calls they describe, and the gaps between them.

Read-only pass. Your only write is the report file.

## Phase 1 — Collect both sides of the diff
- Grep the shipped copy: disclosure strings, AI badges, tooltips, empty-state text, claims in templates ("AI", "generated", "assistant", "never trained on").
- Find the model calls: which code paths invoke a model, what the request payload actually contains, what comes back (confidence, citations, model ids), and what of it the UI keeps or drops.
- Map each AI touchpoint to the disclosure a user actually sees there — or to none.

## Phase 2 — Audit through 6 lenses
Label every finding **measured** (both sides cited from code) or **suspected** (one side inferred); an unlabeled finding doesn't ship.
1. **Undisclosed AI** — model output reaching the user with no cue in the rendering component; cite the component and the missing string
2. **Confidence dropped on the floor** — the API returns scores, citations, or caveats the UI discards; file:line where the signal dies
3. **Copy vs payload** — data-use claims ("not stored", "never trained on") diffed against the request the code sends and any retention or vendor training flag you can find
4. **Capability overstatement** — UI copy promising what the prompt or model config can't back: "understands", "always accurate", against temperature, truncation, and absent retrieval
5. **Version silence** — model ids pinned in config while the UI implies one stable "AI"; behavior can change under users with no cue
6. **Recourse reality** — the "talk to a human" or feedback affordance: trace where its handler actually sends things, or whether the button is decorative

## Phase 3 — Curate
- Rank by miscalibrated trust: an undisclosed AI decision outranks a missing model name.
- Write each fix as the artifact itself: the string to add, the field to surface, the claim to correct.
- Keep measured and suspected findings in separate ranks; every suspected one names the check that would confirm it.

## Phase 4 — Report
Create `TRANSPARENCY.md` at repo root:
1. **The honesty diff** — touchpoint · what the user is told (quoted, file:line) · what the code does (file:line) · gap
2. **Findings** — each: measured/suspected · lens · trust risk · the fix, written out
3. **Dropped signals** — the confidence, citations, and ids the backend has and the UI hides
4. **Priority** — the three corrections that most recalibrate user trust

Start the report with today's date. If `TRANSPARENCY.md` already exists from a previous run, read it first and lead with what changed since.

## Rules
- Quote the string and cite the call — a claim about the product's honesty must itself show its evidence
- Under-disclosure and over-warning are both findings; noise teaches users to ignore the truth
- No model output reaching a user in this repo? Say so in a one-paragraph null report and stop — a null result is a valid finding.
- If a `reports/` directory exists at the repo root, write the report there instead of the root.
- Before asking, present the top findings as a ranked list in plain words
- Report only — end by asking which transparency gaps to close first

House rules for this Goal Prompt

  • Quote the string and cite the call — a claim about the product's honesty must itself show its evidence
  • Under-disclosure and over-warning are both findings; noise teaches users to ignore the truth
  • No model output reaching a user in this repo? Say so in a one-paragraph null report and stop — a null result is a valid finding.
  • If a reports/ directory exists at the repo root, write the report there instead of the root.
  • Before asking, present the top findings as a ranked list in plain words
  • Report only — end by asking which transparency gaps to close first
Keep exploring

More AI-Ethics briefs

Runs inside these playbooks — curated sequences you can launch with one paste: