Every week I try to catch up on the latest talks from the frontier labs. Lately, many of them have focused on loops and higher levels of abstraction. One talk in particular, Lamis Mukta’s “Learning While You Sleep: Beyond Memory to Dreaming”, caught my attention because it made me wonder what was sitting in my own session transcripts, mostly unread once a session ended. So I started reading them.
I wanted to know where my agents and skills were running into trouble. My tooling found one permissions issue the harness had silently dismissed across more than 60 sessions. A different mismatch appeared independently in nine review sessions. Both had been visible inside individual runs, but neither looked like a pattern until I read across them. The scan also surfaced subtler problems in an app I am building, including lessons that never carried from one session to the next despite my earlier work on the Session-to-Session Wall.
I have written before about the Session-to-Session Wall, the point where a coding agent’s context resets and takes the lesson with it. For a private project I have been calling Project Nidus, I built a real answer to that: Core memories that load every session, Topical memories that load on demand, ADRs that record why a decision was made instead of just what it was. It works. My agents stop re-suggesting the thing I already told them not to do.
Except the wall did not actually go away. It moved.
Structured memory preserved the lessons I captured. I still needed a way to discover the lessons no single session recognized.
The gap that structured memory does not close#
Core and Topical memory only capture what I or an agent remember to write down. That sounds obvious, but it took me a while to notice because the harness did not look like it was forgetting. A correction worked, then never became a lesson. In the middle of fixing an issue, neither I nor the agent stopped to update memory.
Most lessons do not announce themselves. The first occurrence looks like a one-off. The second looks like a coincidence. Somewhere around the third or ninth time, I might connect the dots if I am reading across all of my sessions. Most days I am not, because I do not read every transcript from every agent run.
So I built something that does.
Dreaming: a nightly job that mines its own history#
Mukta, a Member of Technical Staff at Anthropic, framed dreaming as a school: students submit work, teachers grade it, and a headteacher reviews everything at once, in batch, looking for the gap no single teacher would notice alone, like an entire class missing the same topic from the curriculum. Applied to agents, dreaming takes a memory store and a stretch of session transcripts, looks for patterns across all of them, and proposes an updated memory store. The person running it still decides what gets accepted. I kept that human decision point in my implementation because it came from the original framing.
I built my own version for Project Nidus. An unattended job runs on a schedule, reads recent session transcripts, and looks for corrections or gotchas that appear more than once but are not already captured in shared memory or the session’s active context. It writes candidates to a pending queue. It never touches shared memory directly. I look at each candidate and accept or reject it, one at a time.
That last part is the whole design. The job proposes. I decide what becomes memory. Here is the shape of it, rewritten as a clean example rather than pulled from anything Nidus-specific:
graph LR
T[Session transcripts] --> R[Redact on-device]
R --> M[Nightly mining job]
M --> C[Candidate findings]
C --> H{Human review}
H -->|accept| K[Shared knowledge base]
H -->|reject| D[Discarded]Making the loop safe enough to run unattended came down to three constraints: process each transcript window once, redact locally before analysis, and reserve the write path for me.
Process each transcript window once#
The mining job tracks the timestamp of the last fully processed run and only looks at transcripts newer than that. The watermark only advances after a run proves it finished everything in the window. A crash mid-run just means the next run picks up the same files again, which is wasteful but never wrong.
#!/usr/bin/env bash
# list-since.sh: print transcripts touched after the last watermark
set -euo pipefail
WATERMARK_FILE="${DREAM_WATERMARK_FILE:-$HOME/.dreaming/watermark}"
PROJECTS_DIR="${DREAM_PROJECTS_DIR:?set this to your session-transcript directory}"
watermark_epoch=0
if [ -f "$WATERMARK_FILE" ]; then
watermark_epoch="$(date -u -d "$(cat "$WATERMARK_FILE")" +%s 2>/dev/null || echo 0)"
fi
find "$PROJECTS_DIR" -name '*.jsonl' -type f -print0 |
while IFS= read -r -d '' f; do
mtime="$(stat -c %Y "$f" 2>/dev/null || stat -f %m "$f")"
[ "$mtime" -gt "$watermark_epoch" ] && printf '%s\t%s\n' "$mtime" "$f"
done | sort -n | cut -f2-# set-watermark.sh: only called after a run proves it fully processed everything up to this point
set -euo pipefail
new_watermark="$1"
tmp="$(mktemp)"
printf '%s' "$new_watermark" > "$tmp"
mv -f "$tmp" "$WATERMARK_FILE" # atomic write. a crash mid-write never corrupts the cursorRedact locally before analysis#
The redaction step has one rule that matters more than the regex list underneath it: if it finds something that looks like a real credential, it does not try to scrub the file and send along a “safer” version. It excludes the file entirely.
#!/usr/bin/env python3
"""redact.py <src> <dest>: scrub identifying patterns. refuse, don't half-fix, on high-severity content."""
import re, sys
PATTERNS = [
(re.compile(r'\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b', re.I), '<uuid>'),
(re.compile(r'\b[\w.+-]+@[\w-]+\.[\w.-]+\b'), '<email>'),
(re.compile(r'/Users/[^/\s]+'), '/Users/<redacted>'),
]
HIGH_SEVERITY = [
re.compile(r'-----BEGIN [A-Z ]*PRIVATE KEY-----'),
re.compile(r'\bAKIA[0-9A-Z]{16}\b'), # cloud access-key shape
]
def redact(text: str) -> str | None:
"""Returns None (caller should exclude the file entirely) if a high-severity
pattern is found. A regex substitution over something that already matched a
credential shape is not a safety guarantee. Only refusal is."""
if any(p.search(text) for p in HIGH_SEVERITY):
return None
for pattern, replacement in PATTERNS:
text = pattern.sub(replacement, text)
return text
if __name__ == '__main__':
src, dest = sys.argv[1], sys.argv[2]
content = open(src, encoding='utf-8', errors='replace').read()
result = redact(content)
if result is None:
print(f'{src}: excluded (high-severity pattern)', file=sys.stderr)
sys.exit(1)
open(dest, 'w', encoding='utf-8').write(result)A regex is a pattern match, not a proof. Treating a match as grounds to exclude, rather than grounds to patch, is what makes the redaction step trustworthy enough to run unattended.
The nightly run, tied together#
#!/usr/bin/env bash
# dreaming-nightly.sh: the shape of the whole pipeline, backend-agnostic
set -euo pipefail
HAD_FAILURE=0
transcripts=($(./list-since.sh))
[ "${#transcripts[@]}" -eq 0 ] && { echo "nothing new"; exit 0; }
batch_dir="$(mktemp -d)"; candidates_dir="$(mktemp -d)"
for f in "${transcripts[@]}"; do
hash="$(sha256sum "$f" | cut -d' ' -f1 | cut -c1-16)" # identify by hash, never real filename
python3 redact.py "$f" "$batch_dir/$hash.txt" || continue # excluded files are just skipped
done
prompt="Read every transcript in $batch_dir. Report a recurring correction/gotcha NOT
already captured in our knowledge base (attached below). For each, output one JSON object:
{session_id, slug, topic, symptom, rule}. Output ONLY a JSON array, no prose."
# 'your-ai-cli' stands in for whichever coding-agent CLI you have. the interesting part is
# the contract (read-only tools, structured JSON out, exit code is the source of truth),
# not the specific vendor.
response="$(your-ai-cli --print "$prompt" --tools read-only --add-dir "$batch_dir" \
--add-dir "$KNOWLEDGE_BASE_DIR" --output-format json)"
echo "$response" > "$candidates_dir/batch.json" || HAD_FAILURE=1
survivors="$(python3 aggregate.py "$candidates_dir")" # keep only patterns seen in 2+ sessions
if [ "$HAD_FAILURE" -eq 0 ] && [ -n "$survivors" ]; then
echo "$survivors" > "./pending/$(date -u +%Y%m%dT%H%M%SZ)-pending.json"
./set-watermark.sh "$(date -u +%Y-%m-%dT%H:%M:%SZ)" # only advance on full success
fiNotice the threshold in aggregate.py: keep a pattern only if it showed up in two or more sessions. I picked two as the floor because one session is just a session. What surprised me was how often the patterns cleared that bar. The most repeated finding exceeded it by a factor of four and a half.
The candidate format the job writes out is deliberately narrow:
[
{
"session_id": "a1b2c3d4e5f6",
"slug": "unscoped-search-hits-timeout",
"topic": "workflow",
"symptom": "A recursive file search across the whole repo root hit the tool's timeout and returned nothing.",
"rule": "Scope searches to a specific subdirectory or file pattern instead of the repo root."
}
]session_id is a hash, never a real path. Nothing in this file can be traced back to a real transcript without a separate, deliberately inconvenient lookup step that lives outside the repo.
Reserve the write path for review#
Only one script is allowed to touch shared memory:
#!/usr/bin/env bash
# accept.sh --topic <t> --symptom <s> --rule <r>: the sanctioned write path. a human runs
# this by hand for each candidate they choose to keep. nothing in the pipeline calls it automatically.
set -euo pipefail
# ...arg parsing omitted...
key="$(derive_dedup_key "$symptom" "$rule")"
if existing="$(grep -il "$key" "$KNOWLEDGE_BASE_DIR"/*.md)"; then
echo "Already known. See: $existing" >&2
exit 3 # dedup, nothing written
fi
printf -- '- [%s] SYMPTOM: %s RULE: %s\n' "$(date +%F)" "$symptom" "$rule" >> "$KNOWLEDGE_BASE_DIR/$topic.md"
echo "captured"The mining job can propose. Only I run accept.sh and commit a candidate to shared memory. That keeps a noisy or wrong candidate from ever quietly becoming official. The job proposes. I decide what becomes memory.
Nine sessions, one lesson#
Once the job had been running long enough to build up a corpus worth reading, the most repeated finding in it turned out to be one gotcha, described nine different ways by nine sessions that each thought they were the first to hit it. The gotcha itself was almost mundane: some of my automated review agents are deliberately restricted to read-only tools, no shell access, but a few of the prompts dispatching them still said things like “verify this by running git diff” or “confirm by running the tests,” an instruction those specific agents structurally cannot follow. Nine review sessions, spread out over months, each independently flagged the same mismatch as a fresh discovery.
None of them was being careless. A session can only reason about what is in its own context window, and a pattern that exists only in aggregate is invisible from inside any single one of them. Noticing it takes something reading across all of them at once, which is the job description of an unattended nightly process and something I would never do reliably by reading transcripts one at a time.
The humbling number#
That finding did not arrive on its own. One batch surfaced around 175 candidate lessons. After checking each one against the real codebase, most turned out to be things already fixed, or already documented somewhere I had not thought to check. Only a genuine handful were new.
I will take that result over the alternative. A naive version of this, one that wrote every candidate straight to shared memory, would have buried the real handful of new lessons under a pile of already-solved problems. The scan found the signal. The review gate kept the much larger pile of stale findings from becoming memory beside it. That is what separates a knowledge base that compounds from one that just accumulates.
Since drafting this, the historical backlog has finished draining. The job read and reconciled several thousand old sessions, then surfaced real candidates the first time it ran unattended overnight. I also added a small append-only ledger that tracks which candidates I have already reviewed, so working back through a backlog does not mean starting from zero every time.
Three lessons that came the hard way#
Building the backend that does the mining went sideways more than once before it went right.
The first was a quota surprise. I assumed the rate limit on the AI backend doing the mining was hourly. The actual limit used a rolling multi-hour window, and running it against a large historical backfill throttled hard, down to roughly one batch per window. The fix was to respect the limit. I split the workload: the heavy historical backfill moved to a different backend entirely, on my personal Claude subscription, while the lightweight nightly job stayed where it was. I no longer assume one provider’s rate limits generalize to another, especially when the workload looks nothing like my normal usage.
The second was a security finding I ended up verifying live. Wiring up that second backend, I assumed a restricted, read-only permission mode meant read-only. Testing proved that assumption wrong. One specific tool, web search, sat outside the restriction, which meant a redacted transcript could, in theory, trigger an outbound network call the mode was supposed to prevent. I reproduced it, then fixed it with an explicit allowlist instead of trusting the restriction to be complete. I now verify exactly which tools a restricted mode covers instead of treating restricted and read-only as interchangeable.
The third was the permissions issue from the opening. After the pipeline had been running unattended, I found that a mode meant to keep the backend read-only had quietly swallowed the model’s finished output across more than 60 sessions. It treated the prompt as an interactive planning task, found no tool to formally exit that mode, and handed back plan-exit confusion instead of the real analysis. I only caught it because a diagnostic I had added earlier preserved the raw output of any batch that failed instead of throwing it away. Once I could see what had actually come back, the fix was simple. The flag was redundant anyway. The tool allowlist was already doing the same job, so I just removed it.
None of those lessons came from reading a changelog. They came from building the dreaming capability and watching where it broke.
It is one thing to stand on the comfortable ground of placid inaction and put forth words of cynical wisdom, and another to plunge into the work itself and through strenuous experience earn the right to express strong conclusions.
John D. Rockefeller
Memory as a process#
The nightly batch job matters less to me than what it points at: an agent’s institutional memory improving without me curating every entry by hand. In my own work, this knowledge was captured inconsistently, depending on whether I happened to be paying attention. A process that reads its own history and only asks me about genuinely new findings has a different cost structure than relying on me to update a wiki page or a codebase’s shared memory.
Watching this run has changed how I think about memory for a coding agent. I now see memory as a process that notices and promotes lessons, rather than simply a file that stores them.
Can I make this work in Kiro or Codex?#
I use Kiro, Claude Code, and Codex, so the next question for me was whether I could reproduce this loop outside Claude Code.
For Kiro, I chose a harness in ExtraLife rather than Kiro Crew. The harness reuses the same redaction, watermark, and review queue while asking Kiro to fill the analysis role. It is still under construction, so I do not have results yet. When I do, I want to compare the quality of its candidates, the permissions model, and how reliably it resumes after a failure.
Codex comes next. I still need to determine what transcript history is available, what memory Codex can reconcile against, and whether I can keep the entire path read-only until review. I have not run that experiment yet, so I am not treating Codex as a supported path.
I did not find this exact consolidation loop documented as a native feature in either tool, which is why I am testing the harness approach.
The first Kiro run will use a small transcript window, read-only analysis, and manual review before I add scheduling. If it produces useful candidates inside those boundaries, I will run the same test with Codex. That will tell me whether dreaming belongs to one coding agent or can become a portable layer around any of them.
Keith