Concepts

The mental model behind agent-memory, in the order you'll meet it.

The record format

A MemoryRecord is a typed, timestamped, provenance-carrying unit. Four types: fact · episode · procedure · rule.

Identity is content-addressed:

id = "mem:" + sha256( canonical_json(birth_content) )[:32 hex]

where birth content is {type, body, slots, provenance, created_at} — and only that. Lifecycle fields (state, links, confidence, flags, tags, signature) are versioned around the stable id:

• A state change never renames the record.
• Two agents extracting identical content at the same instant name it identically — dedup is a byte comparison.

Other format properties:

• Confidence is a string on the wire — a 4-decimal string, because canonical JSON forbids floats. confidence_to_wire / confidence_from_wire convert.
• occurred_at is optional event time (distinct from created_at write time): immutable, signed, and omitted from the wire when unset so pre-existing signatures re-serialize byte-identically.
• extra preserves unknown wire fields — they round-trip and are signed, so a newer writer's fields survive an older reader.

Ownership & signing

An agent's memory is a file it owns. Two signature layers:

• Record signatures — individual records sign with an fg-agent-id KeyPair. The signer's address lives inside the signed payload (signed_by), so a signature can never be re-attributed.
• File signature — a MemoryFile carries a file-level signature by the owning agent.

Both use a domain-separated signing input (fg-agent-memory/v1 with per-context tags), so a memory signature can never be replayed as an identity artifact.

Two deliberate rules:

• evolve() clears the signature. A lifecycle mutation produces a new unsigned version rather than carrying a stale signature; the signed birth version stays verifiable in history.
• Memory.load(verify=True) (the default) requires a valid file signature and verifies every record naming a signer — it never silently downgrades to unverified.

The lifecycle state machine

Enforced only in lifecycle.py as pure functions — never by convention, never by an LLM. States: active · transitional · superseded · archived.

The complete legality table (LEGAL_TRANSITIONS):

Rules enforced in code:

• Supersession must name its successor and a reason — the successor must already reference the old record via supersedes.
• Contradiction produces a Transition pair: both records enter transitional and carry the same transition block (sides + reason), so either record leads to the other. Visible, never a silent overwrite.
• Resolution returns the winner to active and supersedes the loser — the loser stays addressable with superseded_by and a recorded reason.
• Rules require evidence: a rule must carry a non-empty evidence list of record ids, and refresh_rule_flag flags a rule when all its evidence is archived or superseded (any live evidence clears the flag).
• Archival is terminal; records are never deleted.

Recall — the read stage

Retrieval is state-aware and budgeted:

• Transitional records surface both sides atomically — you never see half a dispute presented as settled truth.
• Superseded records ride the supersedes provenance trail only under include_history; archived records only under include_archived.
• The token budget selects whole records — compression is selection, never rewriting. Ties break deterministically by id, and every included record carries a machine-readable inclusion reason.
• Scoring: relevance × state_weight × recency × salience × confidence, with state weights active 1.0 / transitional 0.8 / superseded 0.3 / archived 0.1, a 30-day recency half-life (floor 0.1), and ~4 chars/token estimation.
• Touch reinforcement: each retrieval hit bumps a TouchCounts sidecar, so records that keep proving useful resist decay.

Operators & proposals

Every mutation an operator — LLM or heuristic — wants must arrive as a typed Proposal of one of four kinds: create · supersede · transition · archive. apply_proposal validates it against the lifecycle machine before any write; an invalid proposal raises without writing — rejected, never repaired — and rejections are logged in the consolidation report.

The default operators are deliberately dumb, model-free heuristics (trigram Jaccard near-dup, slot/negation-polarity contradiction, a cautious newer-wins resolver), so the whole engine runs with zero model dependency. Swap in an LLM operator and the validation gate is identical.

The consolidation pass order

consolidate() runs a fixed sequence:

1. Exact dedup — byte-identical content addresses merge.
2. Near-dup merge — trigram similarity above threshold (default 0.6).
3. Contradiction detection — opens transition pairs.
4. Resolution — only if a resolver operator is configured; the porcelain default has no auto-resolver, so contradictions stay visible until Memory.resolve.
5. Episode → rule promotion — a rule is proposed once 3 episodes share a signature (threshold configurable), carrying those episodes as evidence.
6. Rule-flag sweep — flags rules whose evidence has all been invalidated.

Blocked candidate generation (records sharing a stemmed content word or slot key) keeps consolidation near-linear.

Footgun: pipeline-level default_operators() includes an auto-resolver (HeuristicResolutionOperator) — bare consolidate(store) picks winners automatically. The Memory porcelain deliberately runs without an auto-resolver, so contradictions stay visible until you call Memory.resolve. Also note Memory.load into an existing store merges: records whose ids already exist are skipped, never overwritten.

Decay & salience

Decay is mechanical, not judgment: salience combines recency (half-life 30 days), touch reinforcement (bonus 0.25 per touch), and confidence. Records below the archive threshold (default 0.05) are archived — a lifecycle state, not deletion — and protected records (e.g. disputed ones) are reported rather than touched. Touch counts live in an engine-local salience.json sidecar; losing it degrades gracefully.

Redaction vs archival

Two different exits with different guarantees:

• Archival (decay) — the record leaves default recall but its content is kept.
• Redaction (§11 of the spec) — the escape hatch for secrets/PII. redact() replaces every stored version with a single tombstone: id, type, timestamps, and the link graph are preserved; body, slots, tags, and provenance are destroyed; the full birth_digest is recorded. The body becomes "[redacted]".

Redaction is available on all three stores, the porcelain, and the MCP server — but it is deliberately not on the RecordStore port and is unreachable by operators. No proposal kind can destroy content.

Storage model — append-only, three stores

RecordStore.put never overwrites: each put of an existing id appends a new version; get returns the latest; history returns all versions oldest-first. Deletion is not in the contract. Three shipped stores pass one shared port-contract suite:

• InMemoryRecordStore — dict of version-lists; the reference.
• FileRecordStore — one directory per record id (<hex>/000001.json, …), one canonical-JSON file per version. Git-friendly: stable content-addressed paths, append-only diffs, no churning index. Atomic and concurrent-writer-safe (temp file + fsync + os.link; a taken version number retries rather than clobbers).
• SQLiteRecordStore — single-file SQLite: records (latest), record_tags, history (all versions). The payload column stores canonical JSON so rows round-trip to exact signed bytes. WAL mode, BEGIN IMMEDIATE, retry on contention, thread-safe.

Search indexes: SQLiteSearchIndex (FTS5 BM25 over body + tags; query text is reduced to alphanumeric OR-joined tokens so raw text can't inject FTS syntax) and InMemorySearchIndex (cosine over an Embedder, default the deterministic HashEmbedder). Sidecars (salience.json, index.sqlite) are engine-local, non-standard state — losing them degrades gracefully.

Portability & golden vectors

RecordStore / SearchIndex / Embedder / Operator are ports; every invariant is stateable over the record format alone. Export produces one canonical-JSON MemoryFile (signed if an identity is set); import via Memory.load. Byte-level constructions are pinned by golden vectors in spec/vectors.json — where prose and code disagree, the vectors win.