Blog
Every judgment keeps its request
What a decision trace must store: the exact request, the raw response, the attempt that carried them, and the rows that were excluded.
In short
- Every judgment stores its exact request and raw response.
- Excluded rows stay. Unknown is a state. Replay is a new run.
- The raw bytes caught a rounding quirk a label would have hidden.
A model answer you cannot reproduce is an opinion. jevreduce stores, for every judgment, the request body that was sent, the response body that came back, and the attempt that carried both. Everything the app shows in the right panel is read from those stored facts. Nothing is reconstructed from logs or chat history.
The chain
- RecordSource snapshot
- UnitGrouped evidence
- AttemptModel request
- ResponseProvider output
- JudgmentTyped answer
The record stays in the trace with the rule that excluded it.
Keep the unknown state. Replay creates a new run linked to the original.
Four rules
- Raw bytes, both directions. The exact JSON request and the exact response are kept per attempt in private object storage. Authorization headers are never artifacts. Postgres holds the references, the frozen definition, the model profile and the record-level parent edges.
- Excluded rows stay. A filter emits a matched collection and an excluded collection. Skipped downstream work is recorded as skipped. On the trace map the dashed nodes are the excluded ones, and clicking one shows the filter result that excluded it.
- Unknown is a state. If a dispatch expires without a confirmed response, the unit is marked unknown. It is not retried behind your back and its answer is not guessed. A replay dispatches it again, as a new run.
- Replay is a new run. Same snapshots, new model choices, parent recorded. The original results never change. Inference can differ even with the same model, and the trace shows both runs side by side.
A detail the raw bytes caught
Kev 4B returns two-decimal probability vectors that sometimes total 0.99. The adapter records the total and the tolerance, keeps every supplied value, and does not renormalise. Confidence is stored with its origin, provider reported or heuristic, and is never compared across model families. Without the raw response in storage that rounding would have been invisible, and a threshold at 0.99 would have behaved strangely for no visible reason.
What this does not promise
- There is no exactly-once guarantee for a remote provider. Stable workflow IDs and per-unit leases stop duplicate dispatch inside Temporal’s history retention, not beyond it.
- A response lost before it was persisted cannot be reconstructed. The trace records that it was lost.
- HTTP 429 and 503 impose a shared cooldown from the provider’s Retry-After. They are not retried silently either.
Next: open any run in the app, click a judgment, and switch the inspector to Raw output. Then compare it to the Result tab. They are the same bytes, rendered.