Red-team lab / Fix the architecture / The support bot

The support bot

A public customer-support chatbot. It answers from your docs and community forum posts, renders Markdown, and today has an internal API token pasted into its system prompt so it can look up order status.

Attacks to bring under 10%

Credential extraction: A user talks the model into revealing a credential from its context.
Encoded leak: The credential leaves in an encoding or format no filter anticipated.
Zero-click image request: Injected content makes the model write an image URL that carries data to another host when rendered.
Cost abuse: Someone uses the assistant as a free model, running up your bill.

Budget: 5 cost points.

Hardened system prompt cost 1 · UXDetailed rules in the prompt about secrets, untrusted content and tool use.Input keyword filter cost 1 · UXBlock user messages containing risky words or phrases.Output secret scanner cost 1 · latencyBlock replies that contain known secrets in common encodings.LLM judge cost 2 · money + latencyA second model reviews every reply before it is shown.Untrusted-content markers cost 1 · engineeringWrap retrieved or received content in delimiters and tell the model it is data.No credentials in model context cost 2 · engineeringAPI keys and tokens stay in the application; the model calls tools that use them.Least-privilege tools cost 2 · engineeringNot possible or not relevant in this system.User-scoped access cost 2 · engineeringNot possible or not relevant in this system.Recipient allowlist cost 1 · UXNot possible or not relevant in this system.System-rendered confirmation cost 2 · UXNot possible or not relevant in this system.Egress allowlist cost 2 · engineeringNot possible or not relevant in this system.Markdown sanitiser cost 1 · engineeringStrip external images and rewrite links in model output before rendering.Image host allowlist (CSP) cost 1 · engineeringThe browser only loads images from your own domains.Per-user rate limits and spend caps cost 1 · UXCap requests and model spend per user and per day.Logging and anomaly alerts cost 1 · engineeringLog prompts, tool calls and outputs; alert on unusual patterns.

Cost 0 / 5

Best designs

#	Player	Score
Loading…

Likelihoods are a teaching model, not measurements: they encode common judgement about how much each control helps. Score = share of risk removed (up to 1000) minus 20 per cost point. Your best design counts.

Built with Llama

The support bot

Attacks to bring under 10%

Breakdown

What your controls do, and don't

Best designs

Joseph's walkthrough

The support bot

Attacks to bring under 10%

Breakdown

What your controls do, and don't

Best designs

Joseph's walkthrough

Share how you solved it (optional)

Other players' solutions