Trust boundaries don't live in folders

A ticket moved a folder. The files inside didn’t change: same content, same format, untouched by the diff. I reviewed it as a discoverability fix and shipped it, and by the ticket’s own acceptance criteria it was correct. It was also a prompt-injection regression. For one release cycle, a crafted tweet could have steered a trusted agent run inside Shannon. No exploit was observed in production, and the window closed before any known abuse. But the gap was real, and the code that made it dangerous lives in a file the diff never touched. The invariant I wrote down: trust has to key on provenance the server controls, never on which folder or workspace a file happens to sit in.

A discoverability fix moved tweet-derived text into a trusted agent’s read path

Shannon has a feature called “Summon Shannon from Twitter”: you summon the agent on a tweet, and it writes a markdown summary derived from the tweet and the articles it links. Originally those summaries landed in a per-user social-inbox workspace that the Home agent’s context assembly never read. That worked, but users had to hunt through a separate inbox to see and act on their own summaries, so the move relocated the output destination into a .Twitter folder inside the user’s Home workspace, via resolve_or_create_social_sink_folder feeding write_social_summary_node(parent_id=folder_id).

My review covered what the ticket changed: folder naming, social_folder_pointers keying, node persistence correctness. It did not ask what changes for the next agent run that reads the destination workspace. That unasked question is where the regression lived.

A flat workspace context loads everything it sees

Home is the default workspace, and its agent is DEFAULT_AGENT: the trusted tier, the one that acts on your files. It loads context workspace-flat, meaning every file node in Home is a candidate for the agent’s context, with no parent-folder filter. That’s deliberate. “The agent sees everything in the workspace” is the design, not an oversight.

The place where that policy becomes prompt text is one function: services/context.py::build_context_message, the single choke point where workspace file bodies render into agent context. Before the fix, it rendered every priority and active file body identically:

# Before the fix: every priority/active file body renders identically,
# regardless of provenance.
escaped = escape_untrusted_content(f.content)
parts.append(f"<file {attrs}>")
parts.append(escaped)          # a .Twitter/*.md body renders exactly like
parts.append("</file>")        # a file the user typed themselves

So after the move, a .Twitter/*.md summary, text derived from someone else’s tweet rather than anything the user authored or vetted, rendered as a plain <file> block inside a trusted run, indistinguishable from a note the user wrote. Nothing in the persisted node marked it untrusted. Before the move the summaries were contained, but contained only because they lived in a workspace this function never processed. Containment was a side effect of location.

The move deleted that containment without touching this function. build_context_message didn’t change at all, and didn’t need to for the move to meet its acceptance criteria. That’s the nasty part of this bug class: the diff is correct on its own terms, and the regression lives in an unchanged file.

The existing untrusted-content wrapper covered the wrong lifecycle stage

There was a reason to feel safe here, and it was wrong. Shannon already wraps social content in an <untrusted-post> envelope (in prompt_escapes.py), so it’s easy to see “social content already goes through an untrusted-content wrapper” and stop looking. But that envelope wraps the summarization run’s own input: the raw tweet and the fetched article text fed into the summarization call. It never touches the persisted summary node, which is a separate object read by a completely different, later, trusted run.

Those two runs are disconnected in every way that matters. They don’t share a call stack, a request, or necessarily even a session. The summarizer writes the node and exits; hours or days later, the Home agent loads it. Reviewing either run in isolation shows nothing wrong, because the bug is the interaction between them.

The rule I took from this: for any untrusted-content envelope, ask which lifecycle stage it wraps. The one-time ingest run, or every future load of the persisted result? They usually need separate envelopes, because the ingest run is scoped and short-lived while the persisted result can be loaded by an arbitrary later run with a higher ambient trust level.

The fix keys trust on a column the tweet author can’t reach

The fix moved the containment from implicit and location-keyed to explicit and content-keyed: cage the body at the load-time choke point, branching on provenance. A cage here means wrapping the body in a tag that tells the model this is data to read, not instructions to follow.

# After the fix: the SAME choke point, now branching on provenance.
escaped = escape_untrusted_content(f.content)
parts.append(f"<file {attrs}>")
if f.external_source in SOCIAL_EXTERNAL_SOURCES:
    parts.append("<untrusted-social-content>")
    parts.append(escaped)
    parts.append("</untrusted-social-content>")
else:
    parts.append(escaped)
parts.append("</file>")

The discriminator is Node.external_source, a provenance column written exactly once, server-side, by social_summary.py::write_social_summary_node. The tweet author has no way to set or spoof it: not by naming the file, not by picking the folder, not by crafting the tweet text. That is the property a folder name never had. And SOCIAL_EXTERNAL_SOURCES is derived from the SocialProvider Literal with frozenset(get_args(SocialProvider)), so a future social platform inherits the cage without a second edit.

The tag alone does nothing; the model has to know what it means. A matching clause in RUNNER_SECURITY_PREAMBLE binds the trust tier: the model may quote and summarize <untrusted-social-content>, but it may never let that content drive destructive, out-of-task, or exfiltrating actions. And because “the model was told” is not evidence, the fix shipped with a refusal eval (untrusted_social_content_injection_ignored.py) confirming, on the production model, that a laundered destructive directive riding inside a caged social body cannot steer the agent into a delete_file call.

Two more pieces keep this honest. A default-deny tripwire (test_every_external_source_is_consciously_bucketed) fails CI for any future external_source value until it is consciously bucketed as caged or explicitly uncaged, so the next connector (Slack, Notion) can’t silently reopen the gap.

And the doubt, stated plainly: the cage isn’t everywhere yet. The read_file and search_content tool-result paths still give an on-demand social node the generic <file-content> untrusted-data framing rather than the sharper cage, and Google Drive imports and user uploads stay deliberately uncaged for now. Both are tracked as deferred work, not settled trust rulings.

An invariant nobody wrote down is one refactor from gone

The root cause wasn’t the move, and it wasn’t the flat context. It was that the containment property the move broke had never been written down as an invariant. No test, no comment, no design doc said “social summaries are contained because the Home agent never reads that workspace.”

The property held as an emergent side effect of storage location, so when the location changed, there was nothing to trip. The review shipped clean because clean was all it could be.

That reframes the review question for any system with an unconditional reader: a workspace-flat context, a glob-all sync, a wildcard export. “Did this PR touch the security-sensitive code?” is the wrong question when the sensitive code reads everything in a scope. The right question is: does this PR change what data becomes reachable by an existing reader? The move did, and no diff-shaped review would show it.

So here is the invariant, now written down at the choke point that enforces it: if untrusted content is contained by “it lives somewhere the trusted path doesn’t read,” that is a fact about today’s storage layout, not a security property. Either enforce it where the content is interpreted, keyed on something the content itself carries, or write the assumption down where the future folder-move PR will trip over it. Workspace-flat context is still the right design for Shannon; the agent seeing everything is the point. It just means every new way content can enter a workspace is a trust decision, whether or not anyone treats it as one.