A threshold is measured, not chosen

The last post was about the memory service attached to an agent. This one is about a single number buried inside it.

A user says "I'm moving in December" and the model writes that down. But last month it already stored "moving in November". Write a new one, overwrite the old one, or keep both?

The question is whether the two sentences are the same fact, and that is answered with embedding similarity. So: same from what number up?

Where the number sits

Two thresholds are needed: one for merging duplicates within a category, another for overwriting an older, automatically collected memory. The second is the dangerous one. The first risks leaving a duplicate; the second risks erasing somebody else's fact.

What happens if you guess

We started at 0.9, on the theory that a high bar is a safe bar.

Production disagreed immediately. A user corrected their moving date to December, and the stale "November" memory kept coming back as the top search result. The correction had not failed to save; both were alive, and the stale one simply retrieved better.

A high threshold is not safe. It means wrong memories live longer.

So we measured

We collected sentence pairs and scored them with the same embedding model production uses. The pairs fall into four groups, because what we want differs by group.

• Same fact — different wording. Must overwrite
• Correction — the value changed. Must overwrite, or the failure above happens
• Added detail — the same fact with reasoning attached. Overwriting loses the elaboration
• Different fact — same category, different content. Overwriting destroys someone else's memory

The first two belong above the line, the last two below it. Setting a threshold means laying these four distributions on one axis and finding where to cut.

Press 0.80 and two of the four added-detail pairs get overwritten. Push to 0.90 and nothing is wrongly overwritten, but six of twenty same-fact pairs are missed. Only at 0.85 are both protected groups intact.

The margins are deliberately uneven

0.85 was not picked for being in the middle. The two directions of failure do not cost the same.

The thin margin is on the less dangerous side. Losing an elaboration loses the why but keeps the what, and history can restore it. Losing a different fact destroys the fact itself, silently.

So the margin was given to the dangerous axis. The asymmetry between 0.22 and 0.006 is a decision, not an oversight.

The same principle set the merge threshold, measured separately over 28 pairs.

Where similarity stops working

Measuring surfaced a region no threshold reaches.

Same kind of correction: caught when the frame is preserved, missed when it is not. Lowering the threshold does not fix this — going lower starts overwriting genuinely different facts.

This region is not a similarity problem; it is a question similarity cannot be asked. The answer is to let the model name the memory it is replacing, not to tune a number. The real yield of the measurement was not 0.85, but knowing the boundary of where 0.85 stops applying.

On the read path we dropped thresholds entirely

Thresholds worked on the write path, so the obvious move was to add one on the read path too, cutting low-scoring search results. The usual thing.

The measurement said otherwise.

Embedding scores are not absolute. The same 0.3 is a top hit for one query and noise for another. On the write path two sentences are compared directly, so the value means something; in search the axis moves with every query.

So search has no absolute threshold, only ordering. Narrowing is done with conditions instead of scores.

What this does not prove

Stated plainly.

It is not a significance test. It is decision-support: does the empirical distribution of real pairs separate? Embeddings are deterministic, so the same pair always scores the same. Repeating the measurement means nothing, and there is exactly one source of uncertainty — whether this sample represents the population.

One of the protected groups is thin. Added-detail has four pairs. That shape of memory is genuinely rare in the corpus and a second mining pass could not grow it. The 0.006 margin cannot be tightened with sampling. It has to be corrected against the score distribution in production logs, and frequent hits near 0.85 are the signal to raise the line.

The domain is narrow. The pairs come from three work personas. Sentence shapes from other domains were never measured.

What is left

A threshold is measured, not chosen. 0.85 is an observation, not a judgement. When someone asks, there is a table.

Decide the cost of failing in each direction first. Once you know which way it hurts less to be wrong, where to put the margin follows. Skip that step and the threshold just drifts to the middle.

The yield of measuring is a boundary, not a number. Learning that reframed corrections escape every threshold was worth more than arriving at 0.85. Knowing where something stops working tells you where to build something else.

The same instrument inverts in a different position. The absolute threshold that worked for writing cut the correct answers first in search. A fixed axis and an axis that moves per query are not the same problem.