Sediment

Research

  1. arXiv preprint, August 2026

    What It Costs to Compose, Rebuild, and Correct Precomputed Memory

    Asa Shepard (Williams College)

    A model can read a body of material once, save that reading, and answer from it instead of reading it again on every request. When does that still give you the right answer? We test saved key-value caches and trained compressions of them on Llama-3.1-8B-Instruct. Memories built from separately prepared parts degrade, keeping one current costs a large (but maybe not so large) share of building it from scratch, and a correction served beside a memory lands conditional on its phrasing.

    arxiv.org/abs/2608.30647

  2. arXiv preprint, August 2026

    Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training

    Zachary Speck (Arizona State University), Asa Shepard (Williams College)

    What does a language model learn from a single training document that it sees only once during pre-training? We use a counterfactual to find out: we measure what the model still knows after being trained with that example, versus what happens to its knowledge when we re-train it without that example. With this procedure, we measure what the model learns, what it retains, and what it loses as it trains.

    arxiv.org/abs/2608.19168

  3. COLM WAB 2026, June 2026

    Probe-and-Refine Tuning of Repository Guidance for Coding Agents

    Asa Shepard (Williams College), Co-author, Williams College (name on the linked arXiv page)

    Before touching a repository (say, a pull request), you want to get some guidance in your coding agent. How do you brief it? Tune its guidance file using synthetic bug-fix probes that surface failures. Then, patch each failure back into the text and try again. The result on SWE-bench Verified is a resolve rate that goes from 25.5% without guidance to 33.0% with the tuned guidance (mostly by getting the agent to put the right files into its first plan).

    arxiv.org/abs/2606.20512