Before your first paying customer
Use this stage before a customer puts their own data into the product and expects you to look after it. Trace where that data goes, what the model can retrieve, and what you will need to investigate a failure. Test those paths with separate users rather than a shared development account.
Apply tenant boundaries in retrieval code. Treat stored memory as untrusted input with a source, and make deletion reach the places you say it reaches. Keep enough trace information to investigate tool decisions while redacting personal data and secrets before you write it.
Add a repeatable injection test pack with benign controls. Assert on calls and arguments, not just a reassuring reply. Check any judge that grades those replies, and make sure the assistant stays within the job you gave it. If customers bring provider keys, keep those keys on the server and out of traces.
Done means you have tests that fail when these boundaries break. Keep the canary data and the expected outcomes so another developer can reproduce the checks after a change.
Know the limit
A passing test pack covers the cases you put in it. It does not prove that every tenant, memory path or future model behavior is safe.
Controls in this stage
Handle your users' API keys safely
User provider keys are encrypted at rest, never logged or returned after saving, deletable and used only for the owning user's requests.
Make delete mean the model can no longer recall it
Deletion reaches every documented copy of user content, or the product explicitly identifies retained copies and their limits.
Run injection tests on every change
A fixed pack of attacks and benign controls checks tool calls and arguments in CI when prompts, models or tools change.
Do not let untrusted text steer your LLM judge
Judges use strict structured output, an explicit cannot-decide outcome and a tested confidence threshold, with separate controls for high-impact actions.
Log enough to investigate, without leaking
Traces record request IDs, user IDs, tool calls and decisions, redact secrets and personal data before writing, and have restricted access and retention.
Treat agent memory as untrusted input
Memory writes use explicit user actions or validated paths, carry their source, exclude secrets and remain visible and deletable to users.
Keep the assistant on its job
The assistant's job and prohibited actions are explicit, tested across turns and backed by code-level checks for high-impact tools.
Isolate tenants in retrieval
Retrieval uses the session's tenant ID in the datastore query, and cross-tenant canaries never appear in retrieved chunks.