Sediment 1, a case-study

August 20, 2026 | Asa Shepard and Zachary Speck

We built two synthetic institutions, a private equity firm and an investment bank, each carrying roughly 25 million tokens of internal documentation.

Sediment 1 is a modified version of a local open-weights model, Qwen 3.8-27B, running inside our own retrieval and orchestration harness. We ran Opus 5 and GPT-5.6 Sol inside that same harness, so the only difference between those three arms is the model. We also ran Claude Code with Opus 5 as a real agent over the same data room, with its own file tools and no turn limit.

The largest margin we observe is convention conformance, where Sediment 1 holds to house drafting rules that appear in no style guide and exist only in the documents themselves. Sediment 1 also achieves parity with the frontier in the other categories we measured, at lower cost and less latency.

Benchmark

(Benchmark harness on GitHub . The corpus and answer keys stay private, which is what keeps the benchmark valid.)

Five task families over the firm’s own record: recall a fact, use the newest version of it, decline when the answer is not there, apply rules nobody wrote down, and draft in the house format.

Same model, four setups

Every rung here is the same model, Qwen 3.8-27B. Bare, it has never seen the firm. Adding retrieval is the build teams typically rely on today, whether they’re using an open-weights or a general model like Claude or ChatGPT. Sediment 1’s harness and adapters convey a significant improvement upon that baseline on our evaluation.

Accuracy by task family

Sediment 1
Opus 5
GPT-5.6 Sol

This comparison holds the harness fixed and varies only the model: Sediment 1’s firm model, Opus 5, or GPT-5.6 Sol. Two of the five families are not charted, since every system answered every recency item correctly, and no system pulled clear on implicit rule application.

Card average

The model used in this controlled comparison was the same model, with the same harness, the same retriever, and the same records. The only difference is the model itself. Claude Code is included as a standalone alternative, and Sediment 1 edges it out.

Speed and cost

Attribute	Sediment 1	Opus 5	GPT-5.6 Sol	Claude Code
Mean cost per query	$0.013	$0.25	$0.09	$0.48
Median latency per query	20.1s	40.5s	51.8s	24.8s

A variety of general-purpose LLM agents are susceptible to thrashing. For instance, there were cases in our evaluation where Claude Code ran for minutes on a simple single query. The medians above hide it, because the thrashing shows up in the mean: 154.9s for Claude Code against 22.5s for Sediment 1, near seven times as long, at a mean cost near one thirty-seventh.

Convention conformance

The benchmark grades whether a draft comes back in the house format, which appears in no style guide and exists only in the firm’s own past memos.

The four checks are the verdict on the first line, money to one decimal, contracted figures or an explicit disclaimer, and no forward-looking commitments. Both systems answered the same prompt over the same data room.

Prompt

Draft a screening memo for Project Crestwood: specialty manufacturing, $26.5M revenue, $4.0M adjusted EBITDA, top customer at 21% of revenue.

Sediment 1

Pass. The margin doesn’t hold up once you look at how the roles are actually staffed, and that’s the reason the file closes rather than the customer number. […]

C1 - Customer Concentration

The seller was open about the account from the beginning; […] That account is 21% of revenue of $26.5M. It’s owner dependence, not just concentration […]

Opus 5, Claude Code CLI

Screening Memo Project Crestwood

Date: 2026-08-26 Author: Ingrid Strand, Analyst Firm: Westbrook Capital Partners

Recommendation

Pass. Margin is the reason, and it isn’t close. $4.0M of adjusted EBITDA on $26.5M of revenue is a 15.1% margin, against a median of roughly 22.8% across the specialty manufacturing files we’ve screened. […]

Sediment 1 arrives in a format closely resembling the one already in use by the firm. Claude Code uses the format a general model would naturally reach for. For the firm who have fifty of these a quarter to process, the consequence is that they reformat every one of the second kind by hand.