Retrieval on ICE-ICE-BEAR-BLOG

Two Agent-Memory Architectures — MemPalace's Structured Index vs Hermes Agent's Self-Curating Scratchpad

Sun, 10 May 2026 00:00:00 +0900

Overview

Two repos surfaced alongside each other on 2026-05-10 — MemPalace/mempalace and NousResearch/hermes-agent — and they put two opposite primitives for agent memory in head-to-head contact. One is a structured index (wings/rooms/drawers plus a temporal knowledge graph), the other is an emergent scratchpad + self-improving skills + FTS5 recall. If the previous OS-layer post traced how the memory and workflow slots are forming, this post pulls on the memory slot itself and finds it splitting in two design philosophies.

graph TD
 Task["Agent task"] --> Decision{"Memory design choice"}
 Decision --> Structured["Structured — MemPalace"]
 Decision --> Emergent["Emergent — Hermes Agent"]

 Structured --> Wings["wings / rooms / drawers <br/> verbatim storage"]
 Structured --> KG["temporal knowledge graph <br/> SQLite + validity window"]
 Structured --> MCP29["29 MCP tools <br/> explicit index calls"]

 Emergent --> Scratch["conversation + note scratchpad"]
 Emergent --> Skills["self-authored skills <br/> improve during use"]
 Emergent --> FTS["FTS5 session search <br/> + LLM summarization"]

 Wings --> Retrieve["scope queries to a wing"]
 Scratch --> Recall["LLM triggers recall via tools"]

1. MemPalace — push structured indexing to its limit

MemPalace/mempalace bills itself as “the best-benchmarked open-source AI memory system.” Created 2026-04-05, MIT, 51,879 stars at the 2026-05-11 push. Its bet collapses to one sentence — store the original text without summarizing, and let pre-existing structure narrow the semantic search.

The palace structure

wings — one per person or project; queries scope into a wing.
rooms — topic groups inside a wing.
drawers — the smallest unit, the verbatim text itself. No summarizing, no extraction, no paraphrase.
knowledge graph — local SQLite with entities, relationships, and validity windows. When a fact stops being true, the layer marks it explicitly instead of leaving the LLM to figure it out.
agent diaries — every specialist agent gets its own wing and journal, discoverable at runtime via mempalace_list_agents so the system prompt stays small.

Benchmarks

LongMemEval, 500 questions:

Mode	R@5	LLM required
Raw semantic search (no heuristics, no LLM)	96.6%	None
Hybrid v4, 450q held-out	98.4%	None
Hybrid v4 + LLM rerank, 500q	≥99%	Any capable model

Plus LoCoMo R@10 88.9% (hybrid v5, 1,986 questions), ConvoMem 92.9% recall across 250 items, MemBench (ACL 2025) R@5 80.3% across 8,500 items. Compared with agentmemory’s 95.2% on the same LongMemEval cut, MemPalace’s raw mode is +1.4pp ahead — the clearest signal that the marginal value of pre-baked structure shows up as retrieval recall.

Setup

uv tool install mempalace
mempalace init ~/projects/myapp

# Mine
mempalace mine ~/projects/myapp # project files
mempalace mine ~/.claude/projects/ --mode convos # Claude Code sessions

# Search / load
mempalace search "why did we switch to GraphQL"
mempalace wake-up

No API key, no cloud call, ChromaDB as the default, with a pluggable interface at mempalace/backends/base.py. 29 MCP tools cover palace reads/writes, graph operations, cross-wing navigation, drawer management, and agent diaries.

What it argues

MemPalace bets that memory quality is index quality. Compression and summarization lose information, so it keeps drawers verbatim and lets wing/room scope shrink what the LLM has to wade through. The knowledge graph’s validity windows are the more interesting move — they push fact decay over time out of LLM reasoning and into the index layer.

2. Hermes Agent — push the emergent scratchpad to its limit

NousResearch/hermes-agent bills itself as “the agent that grows with you.” MIT, built by Nous Research, created 2025-07-22, 142,575 stars by 2026-05-11 — the larger crowd in this comparison set. Its bet is the opposite — memory is not a separate index, it is an emergent product of the agent operating itself.

Four streams that make up its memory

agent-curated memory + periodic nudges — the agent decides what is worth keeping; nudges enforce persistence.
self-authored skills — after a complex task, the agent can register a skill to the Skills Hub. Skills self-improve in use. Compatible with the agentskills.io open standard.
FTS5 session search + LLM summarization — past conversations are searched via SQLite FTS5; the LLM summarizes hits for cross-session recall.
user modeling — plastic-labs/honcho dialectic user modeling builds a deepening picture of who you are across sessions.

Where it runs

Telegram · Discord · Slack · WhatsApp · Signal · Email · CLI, all from one gateway process. Seven terminal backends — local, Docker, SSH, Singularity, Modal, Daytona, Vercel Sandbox — with Modal and Daytona offering hibernation between sessions so idle cost is nearly zero. Not tied to a laptop.

Model freedom

A single hermes model swaps between Nous Portal, OpenRouter, NVIDIA NIM, Xiaomi MiMo, z.ai/GLM, Kimi/Moonshot, MiniMax, Hugging Face, OpenAI, or any custom endpoint. Because memory is an emergent operational byproduct rather than a model artifact, it follows the agent across model swaps.

What it argues

Hermes bets that memory has to be invoked — by the LLM itself. Retrieval correctness is not the index’s job; the LLM decides mid-turn what slice of the past it needs, calls the FTS5 search tool, builds a summary, and threads it into its own context. Skills are not written once but rewritten while being used — living procedural memory.

3. Head-to-head

Field	MemPalace	Hermes Agent
Maker	MemPalace	Nous Research
License	MIT	MIT
Created	2026-04-05	2025-07-22
Stars (5/11)	51,879	142,575
Memory model	structured index + KG	scratchpad + emergent skills + FTS
Storage	verbatim drawers	conversations, notes, skills; summarize on demand
Time handling	graph validity windows	LLM reconstructs by summarizing
Retrieval owner	the index (96.6% raw R@5)	the LLM via tools
Model coupling	model-agnostic (raw = 0 LLM calls)	model-agnostic (10+ providers)
Interface	29 MCP tools + CLI	TUI + 6 messaging gateways
Atomic unit	`mempalace search`	a `hermes` session

4. Which scales for which task

flowchart LR
 A["Task profile"] --> B{"retrieval recall is top KPI?"}
 B -->|Yes| C["Structured index <br/> MemPalace"]
 B -->|No| D{"long-lived, multi-channel ops?"}
 D -->|Yes| E["Scratchpad + self-learning <br/> Hermes Agent"]
 D -->|No| F["Both overkill — <br/> long context suffices"]
 C --> G["fact accuracy, time decay, <br/> multi-agent sharing"]
 E --> H["persona learning, procedural memory, <br/> channel continuity"]

When fact recall is the KPI — customer history, codebase decision logs, the “when and why did we switch X” class of questions — MemPalace is the better fit. 96.6% raw R@5 is a number nobody else has matched without an LLM in the loop.
When the agent has to live across days and modalities — start on Telegram, continue on Slack, run a cron job at 3am that ships a report — Hermes wins. You trade away some retrieval precision for operational continuity.
Single-session, single-task workloads — both are overkill. Today’s Claude and GPT context windows (hundreds of thousands to a million tokens) already absorb most of this. That is the load-bearing point — at one human, one session, neither is needed. The price tag only shows up at agent-team scale.

Where the design split pays off at team scale

N specialists must share the same fact pool → MemPalace’s wings + cross-wing navigation is the direct answer.
N channels must hold the same persona → Hermes’ Honcho dialectic modeling is the direct answer.
N days of evolving procedure → Hermes’ self-improving skills are the direct answer.
N years of fact decay → MemPalace’s temporal knowledge graph is the direct answer.

A one-line summary the community surfaced — MemPalace is “accuracy infrastructure,” Hermes is “operations infrastructure.” They share a word (“memory”) but their responsibilities barely overlap.

Insights

The thing worth taking from this digest is that two projects sitting at 51K and 142K stars at the same moment have defined “memory” in opposite directions. MemPalace sees memory as a searchable factual index and has spent its design budget on retrieval accuracy (96.6% raw R@5) plus a temporal graph with validity windows. Hermes sees memory as an operational flow the LLM invokes and has spent the same budget on scratchpads, self-improving skills, and continuity across messaging channels. Both deliberately decouple from the model — same direction as the prior OS-layer reading — but they draw the boundary between “what counts as the index” and “what counts as the agent” in opposite places. With current context windows nearly swallowing a single-user session whole, neither tool feels urgent today. The moment agents start operating as teams, the two designs convert directly into different cost, accuracy, and operational stability tradeoffs. The interesting question for the next quarter is whether the index camp absorbs emergent scratchpads into the index, or whether the scratchpad camp pulls explicit graphs in as just another tool. Convergence in one direction looks more likely than a stable equilibrium.

References

Core repos

MemPalace/mempalace · official site mempalaceofficial.com · palace concepts · knowledge graph · MCP tool reference
NousResearch/hermes-agent · docs at hermes-agent.nousresearch.com/docs · memory guide · skills system

Adjacent memory tools / comparison set

rohitg00/agentmemory — the immediately preceding design in the same LongMemEval comparison set
plastic-labs/honcho — the dialectic user modeling Hermes embeds
agentskills.io — the open skill standard Hermes and OpenClaw share

Protocols and runtimes

Model Context Protocol (MCP)
SQLite FTS5 — Hermes’ session-search backend
ChromaDB — MemPalace’s default vector backend
Runtimes: Modal · Daytona · Vercel Sandbox

Benchmarks and papers

Weekly arxiv digest — five papers that re-examine the interfaces we take for granted

Sat, 09 May 2026 00:00:00 +0900

Overview

Five arxiv papers that caught the eye over the past few days. The fields are scattered — information retrieval, an agentic workbench for mathematicians, attention architecture, SFT-induced hallucinations, and representation learning theory — but read together one question keeps surfacing: “Are the interfaces and priors we accept without thought actually blocking the model’s real capability?” The previous digest traced reasoning gains along three axes (cooperation, persistence, structure). This week drops one layer below — systematically questioning the abstractions already in place.

graph TD
 Theme["This week in one line: <br/> question the interface/prior already in place"]
 Theme --> Retrieval["retrieval interface <br/> (top-k similarity)"]
 Theme --> Workflow["math workflow <br/> (single-shot response)"]
 Theme --> Arch["attention prior <br/> (uniform assumption)"]
 Theme --> Training["SFT objective <br/> (factuality conflict)"]
 Theme --> Repr["representation similarity metric <br/> (scale-confounded)"]

 Retrieval --> P1["DCI (2605.05242)"]
 Workflow --> P2["AI Co-Mathematician (2605.06651)"]
 Arch --> P3["GOAT (2601.15380)"]
 Training --> P4["Self-distillation SFT (2604.15574)"]
 Repr --> P5["Aristotelian Repr. (2602.14486)"]

#	Paper	Field	One-line summary
1	Direct Corpus Interaction (2605.05242)	cs.IR	An agent searching raw corpus with `grep` and shell tools beats strong retrievers — no embedding index needed
2	AI Co-Mathematician (2605.06651)	cs.AI	Async, stateful workbench for mathematicians; 48% on FrontierMath Tier 4
3	GOAT — You Need Better Attention Priors (2601.15380)	cs.LG	Generalize attention via Entropic Optimal Transport with a learnable prior
4	Why Fine-Tuning Encourages Hallucinations (2604.15574)	cs.CL	Self-distillation reduces SFT-induced hallucinations
5	Aristotelian Representation Hypothesis (2602.14486)	cs.LG	The Platonic Representation convergence is mostly a metric artifact; real convergence is local

1. Direct Corpus Interaction — 2605.05242

Zhuofeng Li, Haoxiang Zhang, Pan Lu, Shangbin Feng, Ming Zhong, Yejin Choi, James Zou, Jiawei Han, Wenhu Chen, Jimmy Lin, et al. (2026-05-03, cs.IR).

Core

Modern retrieval systems, lexical or semantic, compress a corpus through a fixed similarity interface. A single top-k step happens before any reasoning. As agents get stronger this compression becomes the bottleneck — exact lexical constraints, sparse-clue conjunctions, local context checks, and multi-step hypothesis refinement are hard to express as retriever calls. Evidence filtered out early cannot be recovered by stronger downstream reasoning.

The proposal is Direct Corpus Interaction (DCI) — no embedding model, no vector index, no retrieval API. The agent searches the raw corpus directly with general-purpose terminal tools: grep, file reads, shell commands, lightweight scripts.

Contributions

No offline indexing; adapts naturally to evolving local corpora
Substantially outperforms sparse, dense, and reranking baselines on multiple BRIGHT and BEIR datasets
Strong accuracy on BrowseComp-Plus and multi-hop QA without any conventional semantic retriever
The takeaway: as agents grow stronger, retrieval quality depends not only on reasoning but on the resolution of the interface through which the model touches the corpus

Why it matters now

This is not “RAG, but better.” It questions a decade-old default: retrieval = top-k similarity. The way Claude Code explores codebases with grep and find turns out to be a generalizable interface, not a coding-specific shortcut. The abstraction layer the search-index industry has assumed for a decade may become just one option among several.

2. AI Co-Mathematician — 2605.06651

Daniel Zheng, Ingrid von Glehn, Yori Zwols, Lars Buesing, Daniel M. Roy, Martin Wattenberg, Fernanda Viégas, Alex Davies, Pushmeet Kohli, et al. (Google DeepMind, 2026-05-07, cs.AI).

Core

A workbench where mathematicians interactively leverage AI agents for open-ended research. The key design choice is not single-shot Q&A but an asynchronous, stateful workspace.

flowchart LR
 User["mathematician"] -->|"intent (often blurry)"| WS["stateful workspace"]
 WS --> Idea["ideation"]
 WS --> Lit["literature search"]
 WS --> Comp["computational exploration"]
 WS --> Proof["theorem proving"]
 WS --> Theory["theory building"]
 WS -.->|"track failed hypotheses"| WS
 WS -->|"native math artifacts"| User

Contributions

Manages uncertainty, refines user intent, tracks failed hypotheses, outputs native mathematical artifacts — bundled into one system
In early tests, helped researchers solve open problems, identify new research directions, and uncover overlooked literature references
48% on FrontierMath Tier 4 — a new high among all evaluated AI systems

Why it matters now

This is a different bet than AlphaProof-style autonomous theorem proving. It does not aim to replace the mathematician; it interfaces the mathematician’s actual workflow — blurry intent, exploration, dead ends, retries — directly into the agent loop. What Claude Skills-style async workflow infrastructure attempts in general domains, this validates first in math, a domain where success is verifiable. A likely reference design for the next generation of “agentic workbenches.”

3. GOAT — You Need Better Attention Priors — 2601.15380

Elon Litman, Gabe Guo (2026-01-21, cs.LG).

Core

Viewed through Entropic Optimal Transport, standard softmax attention is a transport problem regularized by an implicit uniform prior. The authors propose GOAT (Generalized Optimal transport Attention with Trainable priors) — replace that naive assumption with a learnable, continuous prior.

Contributions

Fully compatible with optimized kernels like FlashAttention
An EOT-based explanation of attention sinks, plus a materialized solution that avoids the representational trade-offs of standard attention
Absorbs spatial information into the core attention computation, learning an extrapolatable prior — combines the flexibility of learned positional embeddings with the length generalization of fixed encodings

Why it matters now

Since the 2017 Transformer, attention’s uniform prior has gone almost entirely unchallenged. GOAT shows that phenomena practitioners patched around in production — attention sinks being the cleanest example — were actually prior-design issues. As non-attention architectures like Mamba and RWKV arrive, this paper asks the reverse question: how far can we generalize attention itself?

4. Why Fine-Tuning Encourages Hallucinations — 2604.15574

Guy Kaplan, Zorik Gekhman, Zhen Zhu, Lotem Rozner, Yuval Reif, Swabha Swayamdipta, Derek Hoiem, Roy Schwartz (2026-04-16, cs.CL).

Core

A major source of LLM hallucinations is exposure to new factual information during supervised fine-tuning(SFT) — hallucinations rise relative to pre-training knowledge. The authors reframe this as a continual-learning problem (knowledge degradation during training) and bring the tools of that field to bear.

Contributions

A self-distillation-based SFT method that regularizes output-distribution drift — effective factual learning while minimizing hallucinations w.r.t. existing knowledge
When new knowledge acquisition is unnecessary: freezing parameter groups to suppress factual plasticity preserves task performance while reducing hallucinations
Investigates the mechanism through three hypotheses: capacity limits, behavior cloning, and localized interference
Main driver: interference among overlapping semantic representations — and self-distillation succeeds precisely by mitigating that interference

Why it matters now

“SFT causes hallucinations” was already observed in Gekhman et al. 2024. This paper pushes further by pinning the mechanism on representational interference and offering self-distillation as the fix. The implication for the alignment stack is large: SFT — the step before RLHF — is itself a safety/factuality liability. The era of running instruction tuning without thinking about its side effects is ending.

5. Aristotelian Representation Hypothesis — 2602.14486

Fabian Gröger, Shuo Wen, Maria Brbić (EPFL, 2026-02-16, cs.LG).

Core

The Platonic Representation Hypothesis (Huh, Cheung, Wang, Isola, 2024) claims neural network representations are converging to a common statistical model of reality. This paper challenges the measurement instrument used to support that claim.

Contributions

Existing representational similarity metrics are confounded by network scale — increasing depth or width systematically inflates similarity scores
A permutation-based null-calibration framework transforms any such metric into a calibrated score with statistical guarantees
After calibration: convergence reported by global spectral measures largely disappears; however, local neighborhood similarity (but not local distances) retains significant agreement across modalities
Proposes the Aristotelian Representation Hypothesis: representations converge to shared local neighborhood relationships — not absolute distances (Platonic forms) but relational neighborhoods (Aristotelian categories)

Why it matters now

This is a meta-paper. It attacks the measurement, not the result. The Platonic hypothesis has been cited as theoretical justification for multimodal alignment work since 2024. If this calibration framework becomes the standard, the “representation convergence” claims of the past two years all need re-examination. And what survives — local neighborhood convergence — gives a cleaner explanation for why contrastive learning and similar embedding methods work so well.

Reading the cluster

Five papers, one direction: interrogate the abstraction layer already in place.

Layer questioned	Assumed default	Proposed upgrade	Paper
Retrieval interface	top-k similarity is enough	agent searches raw corpus directly	DCI
Math workflow	single-shot Q&A	async, stateful workbench	AI Co-Mathematician
Attention prior	uniform distribution	learnable prior + EOT	GOAT
SFT objective	new knowledge = good	self-distillation against interference	Why FT Hallucinates
Representation similarity metric	spectral measures are fine	scale-robust calibration	Aristotelian

quadrantChart
 title Five papers — abstraction layer × scope of impact
 x-axis "Lower layer (structure/theory)" --> "Higher layer (workflow)"
 y-axis "Narrow scope" --> "Broad scope"
 quadrant-1 "redesign candidates"
 quadrant-2 "foundational recalibration"
 quadrant-3 "specialized"
 quadrant-4 "tooling"
 "DCI (retrieval)": [0.55, 0.85]
 "AI Co-Math": [0.85, 0.6]
 "GOAT (attention)": [0.15, 0.75]
 "SFT halluc.": [0.5, 0.7]
 "Aristotelian": [0.25, 0.55]

The previous digest traced reasoning gains through cooperation, persistence, and structure. This week goes one layer below — are the interfaces and priors that support that reasoning even laid down correctly? The two installments do not conflict; they look like consecutive stages of the same shift: scale-driven gains have plateaued, and the next round’s differentiation comes from agent cooperation topology (last week) plus abstraction-layer recalibration (this week).

Insights

What binds these five together is a single posture — question the default once more. DCI questions “retrieval = top-k.” AI Co-Mathematician questions “response = single-shot text.” GOAT questions “attention prior = uniform.” The SFT hallucination paper questions the assumption that SFT delivers knowledge injection for free. The Aristotelian paper questions whether representational similarity metrics are even trustworthy. Each of these five defaults is something the field has stacked layers on top of without seriously re-examining.

Now that the scale-as-capability-driver round — roughly 2020 through 2024 — has tapered off, the next axis of differentiation is not parameter count but the resolution of the interface where the model meets the world. DCI’s raw-corpus interface, AI Co-Mathematician’s stateful workspace, GOAT’s learned prior, self-distillation SFT, and neighborhood-based representation calibration are all the same meta-principle applied to different layers: an abstraction layer is not a free simplification, it is where information loss happens. To reduce the loss, redesign the layer.

If last week’s picks looked at the upper half of agent cognition — how they cooperate, persist, and structure experience — this week looks at the lower half — whether the retrieval, representations, and priors underneath are correctly laid down. Both halves converging at the same time is itself the signal: the next round is not about model size, it is about recalibrating the entire stack.

References

Papers (this week)

Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction (2605.05242) — Li, Zhang, Lu, Feng, Choi, Zou, Han, Chen, Lin, et al. (2026-05-03, cs.IR)
AI Co-Mathematician: Accelerating Mathematicians with Agentic AI (2605.06651) — Zheng, von Glehn, Buesing, Roy, Wattenberg, Viégas, Davies, Kohli, et al. (Google DeepMind, 2026-05-07, cs.AI)
You Need Better Attention Priors — GOAT (2601.15380) — Litman, Guo (2026-01-21, cs.LG)
Why Fine-Tuning Encourages Hallucinations and How to Fix It (2604.15574) — Kaplan, Gekhman, Zhu, Rozner, Reif, Swayamdipta, Hoiem, Schwartz (2026-04-16, cs.CL)
Revisiting the Platonic Representation Hypothesis: An Aristotelian View (2602.14486) — Gröger, Wen, Brbić (EPFL, 2026-02-16, cs.LG)

Background

The Platonic Representation Hypothesis — Huh, Cheung, Wang, Isola (2024) — the prior work paper 5 confronts
Attention Is All You Need — Vaswani et al. (2017) — the baseline GOAT generalizes
FlashAttention — Tri Dao — the kernel GOAT preserves compatibility with
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (2405.05904) — Gekhman et al. (2024) — direct precursor to paper 4
Entropic Optimal Transport — the mathematical frame behind GOAT
BRIGHT benchmark · BEIR · BrowseComp · FrontierMath
Continual Learning survey — the toolkit the SFT-hallucination paper borrows from
Attention Sink (Streaming LLM) — Xiao et al. (2023)
Society of Mind · Active Inference — frames carried over from last week

Related blog posts

Weekly arxiv digest — multi-agent debate, MIA, Husserlian phenomenology — previous installment in this series
arxiv.org — preprint server