How StateCore Performs
The short version: on LongMemEval, at an equal context budget, StateCore answers 87.6% of questions correctly at a 64,000-token budget and 80.9% at 16,000. Pasting the entire conversation history into the prompt with no memory layer at all scores 70.1%.
That last comparison is the one worth sitting with. A memory layer only earns its place if what it selects beats what it replaces.
The numbers
194 questions from LongMemEval,
gpt-5 answering, the benchmark's official gpt-4o judge. Every system was
given the same questions, the same answerer, the same judge, and the same number
of tokens to fill. Intervals are 95% Wilson.
| system | 4,000 tok | 16,000 tok | 64,000 tok |
|---|---|---|---|
| StateCore | 51.0% ±7.0 | 80.9% ±5.5 | 87.6% ±4.6 |
| mem0 OSS | 61.3% ±6.9 | 59.8% ±6.9 | 61.3% ±6.9 |
| No memory (most recent messages) | 9.3% ±4.1 | 22.7% ±5.9 | 53.6% ±7.0 |
| No memory (entire history) | — | — | 70.1% ±6.4 |
Run 2026-08-08 against StateCore
96b853d and mem0
mem0ai==2.0.17, unmodified.
What this means for your integration
Give it room and it will use it. StateCore filled 100% of every budget it was given, at every size. If you are seeing weak recall, the first thing to check is how much of your prompt you are actually spending on memory — going from 4,000 to 16,000 tokens moved the score 30 points.
POST /v1/memory/retrieve takes an optional maxChars. Pass your budget and the
engine packs within it and reports what it left out, rather than handing you a
payload you have to trim yourself:
curl -s -X POST https://api.statecore.io/v1/memory/retrieve \
-H "x-api-key: $STATECORE_API_KEY" \
-H "content-type: application/json" \
-d '{"scopeId":"...","query":"what camera was I looking at","maxChars":16000}'
The response carries a budget block: how much was used, and every item that did
not fit, with the reason. Budgets mean dropping things; you should be able to ask
what was dropped.
Above ~64,000 tokens, expect diminishing returns. The entire test corpus is about 103,000 tokens. At 64,000 the engine is already selecting well enough to beat reading all of it, so the remaining headroom is small.
We lose at 4,000 tokens, and you should know why. In a very narrow budget, systems that store many small fragments cover more ground than systems that store fewer, larger, structured units — mem0 fits 142 short memories where StateCore fits 4 sessions plus 23 facts. If your budget is genuinely that tight, test both.
One thing shifts that comparison in practice: the benchmark writes one whole conversation as one memory event. If you write one event per turn — which most chat integrations do — the shape at 4,000 tokens is different from what this test measured.
What this benchmark does not tell you
LongMemEval measures needle-in-a-haystack recall: can the system find the one message that answers the question. It does not measure the things StateCore is built around — whether a corrected fact actually replaces the old one, whether you can trace a stored fact back to its source, whether state stays stable as history grows.
If those matter to you, the numbers on this page are not the evidence you want. Auditing Memory covers what the engine records and how to query it, and that is the surface to evaluate instead.
Not of a system. Quote the setup with the number or it will not reproduce.
Checking it yourself
The harness, the raw retrievals, and every per-question judge verdict are public: memory-budget-bench. You can read exactly what each system handed the answerer for any question rather than trusting the summary. Two earlier runs of this comparison were withdrawn after we found defects in the harness; both are kept in that repository with notices explaining what was wrong.
Full write-up, including the per-question-type breakdown and how much of each
corpus survived ingestion:
docs/longmemeval.md.