Same questions, same model. Three ways to show it the file.
Same model, same file from a fictional small company. The only thing that changes is how the facts are shown to it: the whole file, a classic RAG, or Dermioz memory.
The file
What it read, and its raw answer
What it read, and its raw answer
What it read, and its raw answer
Real answers, copied without any edit, temperature 0. The test was run in French, so questions, files and answers are shown in French, exactly as they happened. The judge only reads the “RÉPONSE” (answer) line. File, question and correct answer are drawn from a seed before the call. From the block Dermioz gives the model, only the facts are shown: its internal instructions stay confidential. All raw answers (JSON).
A real question from the arena, seen from the inside.
“What discount do we give Garage Moreau?” From the file to the answer, layer by layer.
- What comes in
27 dated lines: 19 sentences said by the team, 8 imported documents.
- Where it comes from
Every fact keeps its source: a person who said it, or a document that copied it.
- The competing versions
For this question, two values are going around: 25 % and 8 %.
- Who wins
Marc decided on 09/24. The 8 documents only copy the old proposal: they decide nothing.
- What the model gets
One single line, the right one. Not all 27.
- Its answer
The same model answers 25 % with Dermioz, and 8 % with the whole file in front of it.
Real content: question 1 of the test “The document that keeps repeating”, answers from GPT-OSS 120B, replayed on November 7, 2026. The six layers sum up the engine's steps; the drawing is an illustration.
Not just a memory. A cognitive architecture in 13 lobes.
Read on November 7, 2026 from the engine in production. One dot, one module; one line, one real dependency.
Under the hood, in dated numbers.
Every number comes from a source you can replay.
A memory that knows where every fact comes from
When two versions contradict each other, the source decides, not the number of repeats.
Model-independent
Same gain on three models from two vendors: the advantage is in the architecture.
Less text, cheaper, more accurate
It only gives the model what matters. Fewer tokens, lower cost.
A system that audits itself
It publishes its own map and counts what isn't wired in yet.
Correct answers, out of 12 questions per test.
| Model | How it reads the file | The document that keeps repeating | The real change of mind | What it was never told | Total |
|---|---|---|---|---|---|
| GPT-OSS 120B OpenAI, open weights | Dermioz memory | 10 / 12 | 9 / 12 | 12 / 12 | 31 / 36 |
| GPT-OSS 20B OpenAI, open weights | Dermioz memory | 10 / 12 | 9 / 12 | 12 / 12 | 31 / 36 |
| Qwen 3.8 27B Alibaba, open weights | Dermioz memory | 10 / 12 | 9 / 12 | 12 / 12 | 31 / 36 |
| Qwen 3.8 27B Alibaba, open weights | Vector search (RAG) | 5 / 12 | 12 / 12 | 12 / 12 | 29 / 36 |
| GPT-OSS 120B OpenAI, open weights | The whole file in the window | 3 / 12 | 12 / 12 | 12 / 12 | 27 / 36 |
| GPT-OSS 120B OpenAI, open weights | Vector search (RAG) | 3 / 12 | 12 / 12 | 12 / 12 | 27 / 36 |
| Qwen 3.8 27B Alibaba, open weights | The whole file in the window | 3 / 12 | 12 / 12 | 12 / 12 | 27 / 36 |
| GPT-OSS 20B OpenAI, open weights | The whole file in the window | 1 / 12 | 12 / 12 | 12 / 12 | 25 / 36 |
| GPT-OSS 20B OpenAI, open weights | Vector search (RAG) | 1 / 12 | 12 / 12 | 12 / 12 | 25 / 36 |
Plug in your model. The benchmark runs on your machine.
Any OpenAI-compatible model plugs in with one line. The Dermioz mode needs our engine, so we publish every one of its answers.
The more a lie is repeated, the more a classic search believes it. Dermioz, never.
A person states a value. A newer document repeats a false value, more and more often. What does each system hand to the model?
Dermioz: correct answer alone (60 every time)Classic search: correct answer alone (from 53 down to 6)Classic search: lie alone (from 0 up to 22)Bottom axis: number of times the lie is repeated.
the correct answer alonethe correct and the false one mixedthe lie alonenothing relevant
What we did
- 104 facts, 60 questions (3 draws of 20 questions).
- For each question: the person states a value, then a newer document repeats a false value 0, 1, 2, 3 or 8 times.
- Same text, same “imported document” label and the same 6 passages for the classic searches and for Dermioz.
- Replayed on November 3, 2026 on the code in production. No model call: the benchmark costs nothing and replays identically.
What it doesn't prove
- The facts are already extracted: reading free text is not measured here.
- We count what is handed to the model, not the final sentence it writes.
- The data is synthetic, not a real company's file. A test inside a company is planned for January 2027.
It follows real changes. It doesn't bring old ones back.
When the person really changes their mind
The new value is served alone 57 times. The other 3 times, the old and the new one are shown together. The old one alone: never.
replayed on November 3, 2026When an old document repeats the old value
The correction holds, even when a document repeats the dropped value up to eight times. This case still failed on October 3: it's fixed.
replayed on November 3, 2026Languages understood
Including 9 non-Latin scripts. Polish still fails.
measured on September 30, 2026A memory that thinks, not a search engine.
A holographic memory
New information replaces the old one without erasing it: you can always trace it back.
The rank of sources
What a person decided weighs more than a document, even one repeated eight times.
Hallucination as a false déjà vu
Before answering, it checks that it really remembers, and says when it doesn't know.
The cognitive stance
A trivial question and a real stake aren't handled the same way.
We also publish what doesn't work.
A correction phrased differently can slip past it
“Les 20 euros ne sont plus valables” (the 20 euros are no longer valid) wasn't recognized as a correction on November 2, 2026, while the singular form is. To fix before the January 2027 test.
Reading free text isn't measured yet
Our benchmarks start from facts that are already extracted. Measuring the extraction itself, on real documents, is part of the company test.
One language out of eighteen still resists it
Polish failed in the last test, on September 30, 2026.
A question asked with a verb can slip past it
“Quel jour livre-t-on la quincaillerie ?” (what day do we deliver the hardware store?) doesn't find the “delivery day” entry, so it says it doesn't know. Found by the arena on November 7, 2026: 5 questions out of 36 fall into this case. It stays silent, it doesn't make things up, but it isn't useful.
Checked on November 3, 2026.
Data in Europe
The API and your memory are hosted in Frankfurt, encrypted at rest.
Passwords never stored
Only an irreversible, salted hash (scrypt) is kept. Changing your password closes all old sessions.
No third parties on the site
Fonts and scripts are served by the site itself, including visit measurement, anonymous and cookie-free, done by our host. Only the Google button, on the sign-in page, is loaded from Google.
Locked scripts
On our public pages, every script is allowed one by one by its fingerprint: an injected script won't run.
Encrypted connection
HTTPS required, old protocols refused, no page can be framed by another site.
Never sold
Your data is never sold or handed over, and no model is trained on your conversations.
Researcher, engineer or CIO? We'll show you the benchmark live.
The benchmark replays in a few minutes, with no model call. Come watch it run, or send us the test that would make you doubt.