dsh-blackhole

English | 简体中文

A DSH port of k0valik/pi-blackhole's
brief compiler (VCC conversation-compiler), wrapped in a dual-pipeline
compaction engine.

Drop-in replacement for the official @deepseek-ai/dsh-compaction-basic
backend that routes inside the single summarize() hook:

• OpenAI (or any provider with a configured Responses endpoint) — the
  official remote path: summarization is delegated to the model server via
  the Responses API compaction_trigger (the same protocol Codex's remote
  compaction uses), and the opaque replacement history is persisted per
  session;
• remote failure (no endpoint, missing key, timeout, protocol error) —
  by default the region is compiled locally by a deterministic VCC-style
  compiler (zero model calls, near-lossless (seq N) pointers back into
  the durable session log), with the native one-shot LLM summarizer available
  via fallbackMode: "llm";
• every other provider — the local compiler, straight away, no network.

Remote compaction branch ports
algal/pi-openai-server-compaction
(MIT). The local compiler is a port of pi-blackhole's brief.ts compiler
(VCC conversation-compiler, MIT; rule lineage: lllyasviel/VCC →
dsh-compaction-instant) — the differential suite pins content-level
equivalence against the real pi-blackhole sources (see Equivalence
below).

Why

How it works

1. DSH decides to compact (pressure / overflow / /compact) and calls the
   engine's summarize() hook with the replayed conversation region.
2. The routed provider/model is resolved (durable request header first, agent
   options second) and matched against the configured endpoints.
3. Remote branch (matched endpoint): the region is converted into
   Responses input items (text, reasoning, tool calls/results), a
   compaction_trigger is appended, and the compaction response streams from
   {baseUrl}/responses. The assistant-authored summary becomes the readable
   checkpoint; the opaque compaction item plus the retained tail of user
   messages (Codex-style 20K token budget) is persisted per session. A later
   remote compaction for the same provider/model replays that opaque window
   and appends only the new trailing surface.
4. Fallback: any remote failure logs a warning and hands the region to
   the local compiler (instant, default) or the native LLM summarizer
   (llm). instant-then-llm tries the compiler first and only then the LLM.
5. Local compiler branch (no endpoint match, or fallback): a deterministic
   zero-model pass over the region that keeps ONLY original tokens — every
   tool call becomes one line * bash "ls" (seq 2 -> result 3), tool results
   never occupy entries (one recall away via the pointer), reasoning is
   elided, long text truncates with ...(truncated from seq N), noise XML is
   stripped, and a token cap drops the oldest rows first with an explicit
   [N entries elided: seqs a-b] note. The (seq N) pointers resolve against
   the REAL durable session log (each flattened message is matched to its
   stored event by id), so a recall tool mounted next to this engine restores
   the exact original content.

The engine subclasses the official BasicCompactionEngine, so all pressure /
retention / policy knobs, the token-meter pricing, and the context-overflow
recovery behave exactly as upstream.

Recall and file recovery

The bundle mounts a dedicated tools unit on both the host and standard-preset
planes. It waits for the tools service and registers recall through the
official defineTool() adapter; the compaction engine does not register tools.
@deepseek-ai/dsh-tools is a required peer supplied by DSH. The optional
/blackhole-recall command still uses the host's dsh-commands service.
Both surfaces read DSH's lossless
in-memory event log and the original tool-call arguments, so they continue to
work after a local or remote compaction checkpoint without depending on pi's
JSONL files.

Typical tool input:

{
"query": "SecretNeedle",
"mode": "hybrid",
"scope": "lineage",
"page": 1,
"limit": 5
}

The same search is available interactively as:

/blackhole-recall SecretNeedle mode:hybrid scope:lineage page:1 limit:5

Supported recall forms are:

Write/Edit extraction recognizes content, edits, oldText, and newText
along with common path keys. Ordinary ranked pages clip prose blocks to about
300 characters, tool results to about 200 characters, and call arguments to 300.
Queries use keyword search unless explicitly wrapped as /pattern/flags;
punctuation in model names and paths does not enable regex. Existing bare
regex queries must add delimiters, for example hook|inject → /hook|inject/.
Transcript hits outside the head preview include up to three bounded verbatim
excerpts from the redacted search corpus. These are partial context; use #N
or expand: [N] before relying on the complete message. File-body match notes
remain separate and full Write/Edit bodies still require file drill-down.
Write/Edit transcript arguments retain string paths and scalar metadata;
both standard body fields and unknown string/object payloads are redacted,
including mixed provider schemas. Known file bodies remain indexed separately.
#N restores the complete prose/result blocks and 2,000 characters of each
call's arguments — the preview's 300-character cap truncated 222 of 1,004 command
lines in the readback, which defeated the point of drilling into an address. File
bodies are available only through #N:path. Search indexing clips each file
operation to 10 KB; every recall response, including exact expansion and
offset paging, is capped at 50 KB. Search/touched limit values are capped at
100 entries and explicit expansion at 50 entries. Unsafe, oversized, or
backtracking-prone regular expressions fall back to ordinary token search.
Ambiguous paths are reported with candidate paths instead of being silently
selected. A 12-character lowercase hexadecimal query is reserved for
observational-memory IDs and returns an explicit integration message.

This is intentionally close to pi-blackhole's transcript/file recall, but it
is not OM-complete yet: the OM observation/reflection ledger, OM-ID lookup, and
automatic reverse recall from OM records are a TODO. VCC-style summary output
and recall are therefore coupled through durable DSH sequence numbers, not
through OM memory IDs. scope:all always stays within the current DSH session
and never loads sibling fork/session logs. Stock DSH sessions do not currently
expose the optional lineageSeqs projection, so lineage and all differ only
when a host supplies that projection.

Install

Requires DSH 0.1.0-rc.6 (web or headless profile).

# into the profile that serves your sessions
dsh plugin --profile web add dsh-blackhole
# or from a local tarball:
npm pack dsh-blackhole && dsh plugin --profile web add ./dsh-blackhole-0.2.0.tgz

The bundle patch:

• disables the official compaction-basic row (already disabled by the
  web-app bundle — idempotent),
• re-enables command-compact (the /compact command, which the web-app
  bundle disables; it is backend-independent),
• inserts this engine under its own row.

Exactly one compaction service ends up mounted; the dual pipeline lives
inside the engine.

Configure

In the profile's settings (or ~/.dsh/settings.yaml under the plugin key):

dsh-blackhole:
# inherited BasicCompactionEngine knobs (all optional, same defaults as
# @deepseek-ai/dsh-compaction-basic):
thresholdRatio: 0.75
retainRatio: 0.25
auto: true

# Responses-assisted knobs:
endpoints:
# direct OpenAI Responses
- provider: openai
baseUrl: https://api.openai.com/v1
apiKeyEnv: OPENAI_API_KEY
# any OpenAI-compatible gateway that forwards the Responses API
- provider: '*'
baseUrl: https://my-gateway.example.com/v1
apiKeyEnv: MY_GATEWAY_KEY
# official Codex backend (needs the beta-feature header)
- provider: openai-codex
baseUrl: https://chatgpt.com/backend-api
apiKeyEnv: CODEX_TOKEN
headers:
OpenAI-Beta: responses=experimental
x-codex-beta-features: remote_compaction_v2
timeoutMs: 120000      # abort the remote attempt after 2 min
fallbackMode: instant  # instant | instant-then-llm | llm
format: blackhole      # blackhole | ledger | index (see "Checkpoint formats")
retainedMessageTokenBudget: 20000
persistRemoteHistory: true
stateDir: ~/.dsh/plugins/dsh-blackhole/state

# Local compiler knobs (all optional):
compileMaxTokens: 8192     # floor; scaled by checkpointScale on big regions
checkpointScale: 0.1       # cap = max(compileMaxTokens, shadowed x scale)
checkpointCap: 65536       # hard ceiling

# ledger format knobs (all optional; defaults are calibrated to this
# plugin's tokenizer, which runs ~1.094x cl100k):
ledgerBudget: 656          # fact section (P/F/C), partitioned F35 / P45 / C20
ledgerTimelineBudget: 438  # write timeline (W rows)
ledgerAtomChars: 150       # per-atom character cap
textTokens: 512            # per assistant text/reasoning block
userTextTokens: 1024       # per user text block
toolCallTokens: 128        # per tool one-liner
includeReasoning: false
stripNoiseXml: true
noisePatterns: []          # empty = built-in VCC patterns
toolArgTools: []           # empty = built-in whitelist
hideTools: []              # e.g. [TodoWrite, ToolSearch]

# Index pool sizes are minimums; automatic sizing includes carried state.
# The overall compileMaxTokens/checkpointScale/checkpointCap limit still applies.
indexBudget: 2048          # one `#seq tool@call arg` row per tool result
diagnosticsBudget: 768     # F section: verbatim failure lines
requirementBudget: 256     # U section: the user's own sentences
requirementCount: 8        # optional goals only; 0 disables goals, active rules stay whole
requirementChars: 160     # optional goal character cap, not an active-rule cap

Endpoint matching: provider matches the DSH provider route the latest request
ran on (exact, or * for any); model matches exact id or a * glob.
Authorization uses apiKeyEnv (env var name) or apiKey (inline — never
commit a real key).

fallbackMode semantics (applies to remote failures only):

The legacy fallbackToLlm boolean still maps: false forces instant;
true behaves like the default. Targets with no matching endpoint always
take the local compiler (or, in llm mode, the native LLM).

Runtime switching: /blackhole-mode

The bundle patch mounts two dedicated plugin units inside the preset
compaction group, mirroring the official shapes (dsh-command-compact /
dsh-tool-todo):

• dsh-blackhole/commands — inject = ['commands', 'compaction'] — registers
  /blackhole-mode and /blackhole-recall;
• dsh-blackhole/tools — inject = ['tools'] — registers the recall tool
  through the official defineTool() wrapper (enforced JSON-Schema
  arguments, structured output render).

They appear wherever /compact and the built-in tools do — no config edit,
no restart to take effect:

/blackhole-mode                 # status: current chain, remote gate, format, endpoints
/blackhole-mode instant         # switch the fallback chain (instant | instant-then-llm | llm)
/blackhole-mode llm             # e.g. hand this session's checkpoints to the LLM summarizer
/blackhole-mode remote off      # close the remote gate; every target uses the local chain
/blackhole-mode remote on       # reopen it
/blackhole-mode reset           # drop all runtime overrides, back to config defaults

Switches are sticky: each one persists into runtime-mode.json inside the
plugin state dir (default $DSH_HOME/plugins/dsh-blackhole/state) and is
re-applied over the config defaults on every DSH start — so an override made
yesterday survives today's restart. The plugin config (fallbackMode /
endpoints) remains the durable source of defaults; the override file only
carries runtime deltas, and the plugin never writes the host's profile YAML.
/blackhole-mode reset (or deleting the file) returns everything to the
config defaults. Disabling remote — or any local/LLM checkpoint landing —
removes the session's persisted remote replacement history, keeping the
"local checkpoint supersedes the remote window" invariant intact.

Checkpoint formats: blackhole, ledger and index

The format is a second, orthogonal axis: it decides what a checkpoint
keeps, not where it comes from. Both formats run on the same engine, and the
ledger path makes no model call at all — it short-circuits both the remote gate
and the fallback chain.

> Those retention numbers are for 1–3k-token regions (the probe set they were
> calibrated on). They do not transfer to regions an order of magnitude
> bigger: on five real 97k–175k-token regions the default ledgerBudget 656
> keeps 44.8% of paths and 42.1% of failure lines, reaching 95.9% / 65.8% only
> at ledgerBudget 1968. Full numbers: artifacts/dsh-real-session-test.md.
>
> Those figures are lower than the 49.0% / 53.4% this README used to carry — not
> a regression but an honest denominator. The old extractor missed every typed
> error (TypeError:, ReferenceError: — the ^ anchored only the first
> alternation) and silently dropped failure lines over 200 characters, so a real
> region counted 58 failures where there are 76. More failure lines compete
> for the same F/P/C budget, which is why the path column moves too.

Three properties are deliberate and easy to get wrong:

1. The budget is partitioned (F 35% / P 45% / C 20%, remainder
   backfilled), never filled in priority order. Order-filling starves failure
   lines — measured 81.8% vs 100% retention at the same 600-token budget,
   because paths always win the race.
2. Ledgers accumulate. A second compaction parses the superseded
   checkpoint back into atoms and merges them, so paths recorded by the first
   compaction retain their source anchors. The old checkpoint is parsed as state,
   rather than extracted again as fresh evidence.
3. The user's instructions are quoted, not summarized (U). A ledger has
   no prose of its own, which measured as 0% constraint retention: the paths
   survived and "only call the free model" did not. The first sentence of every
   user message plus any rule-shaped sentence is quoted verbatim (a paraphrase
   is exactly what loses a prohibition). Active rules and revocations are kept
   whole with their source seq and kind. Only optional goals use the three-entry,
   160-character limit. Rules are additive to F/P/C, within the overall
   checkpoint cap; insufficient mandatory space fails explicitly. Historical
   measurements before this carry/rule change, on two real windows with
   deepseek-v4.1-flash-coding (9 calls): constraint retention 0/4 → 3/4,
   temporal-state accuracy 0/5 → 2/5.
   ⚠️ It buys recall of a constraint, not compliance with it: in the same batch
   the record contained U 用中文回答我 and the model still answered an English
   question in English prose (prose probe CJK 0%).

The index format: buy addresses, not retellings

ledger proved that dropping prose is cheap; it also showed that a ledger
cannot point at a tool result, only at the call that produced it — 15 of 152
results were addressable on the real session. index spends its whole budget on
that one missing property. Shape:

## 结果索引（`#seq` 可用 recall 取回原文）
R: src/
#92 bash@91 tsc --noEmit !
#93 read@92 R:hybrid.ts
#omitted: 12 行结果未入索引（seqs 120-131），可用 #seq 直接回查

## 诊断（逐字，`!` 为含硬失败的调用）
F #92 src/hybrid.ts(95,19): error TS2304: Cannot find name 'Foo'.
F #250 Command exited with code 127

## 用户要求（逐字）
U (seq 1) [constraint] 只允许调用 free 模型

• Every tool result gets exactly one row, keyed by the seq the recall tool
  resolves (#92). A result whose seq cannot be resolved is omitted, never
  written as #-1 — an address the host rejects is worse than none.
• A row is # @. The @call half is what makes the
  checkpoint self-sufficient: recall #92 returns the output, and only the
  assistant message at #91 says what was run. Without it the model can read
  every result back and still not know which command produced it — the readback
  sweep found 225 of 1,456 addresses in that state.
• R: declares a shared path root; only explicit R:file references
  expand it. File tools may share a root, while shell commands and other labels
  stay literal. Unrelated relative paths and Windows paths with spaces round-trip.
  The parser also reads legacy R checkpoints.
• The pools are sized to the region (autoIndexOptions). The fixed
  2048/768/256 numbers were calibrated on a 152-result session and do not hold
  on a bigger one: a 314-result TypeScript session elided 91 rows and kept 21 of
  54 failure lines under them — the format's one promise, broken. Budgets are
  ceilings, not allocations: on a region that fits inside the default, a
  larger pool renders a byte-identical body, so sizing costs nothing
  anywhere else, while a region that does not fit loses rows. Now 0 elided on
  all six measured regions. The overall checkpoint cap can still elide optional
  records, with a visible recall notice; automatic pools are not an unbounded allowance.
• Elision is visible (#omitted:), never silent: the model's next question is
  always "how much is missing", and it can act on a span even if it cannot read
  it.
• F lines are verbatim and each carries the seq of the result that printed it,
  so a diagnostic with no address is impossible. An isError result with no
  regex-shaped failure line still contributes its top line — five of the
  real session's 18 failed calls are Could not find the exact text in,
  whose whole diagnosis is that line.
• U preserves source seq, kind and the user's own words. Active rules,
  preferences and revocations are never count/character clipped. Optional goals
  grow automatically from 8 to at most 32 unless requirementCount is explicitly
  configured; 0 disables goals. requirementChars only bounds optional goals.
• Like the ledger, the index accumulates: a second compaction parses the
  superseded checkpoint's rows back and merges them.

Measured on the 285-node real session (whole matrix: tests/eval-formats.mjs,
page: artifacts/zero-llm-matrix.html): 3,907 tokens — 3.5× smaller than
instant, 3.6× smaller than pi-brief — with 152/152 results addressable,
18/18 failed calls diagnosed, 14/14 hard-failure lines, and 6/6 user
instructions verbatim.

Does it hold off this repo?

The 285-node figure above comes from this project's own session, and index was
designed for exactly this shape, so it proves nothing on its own. Re-run on five
independent sessions in four different situations — a TypeScript
type-break, a Rust build break, a cargo test triage, a multi-file refactor,
and a pip install failure — index is the only format that is simultaneously
fully addressable, fully diagnosed and fully flagged, in 5 of 5
(tests/eval-sessions.mjs, page: artifacts/zero-llm-sessions.html):

Three readings of the table are load-bearing:

• n/a is not 0. dcp emits no #seq addresses at all, so its address
  space is empty; ledger has one and fills it with the wrong thing (W rows
  point at calls, so 0 of 1,304 results are reachable). Printing both as 0/0
  and 0/1,304 in one column made two different facts look like one.
• ledger is 5× cheaper than index and cannot address a single result; the
  fixed-pool index looks 2× cheaper only because it dropped 627 addresses and
  58 failure lines. Cost without coverage is not a cheaper format, it is a
  different, smaller checkpoint.
• index is not the best format on user turns, and the column says so. It
  quotes rather than copies: a message that fits the 160-character cap whole, and
  otherwise the goal sentence plus any rule-shaped or numbered sentence. Formats
  that carry whole messages (dcp wide, pi-brief) retain every opening; index
  keeps 35 of 37, and the two it misses are not lost constraints — one is a user
  message with an injected ⚠ Skill descriptions… notice glued to its tail (the
  request is quoted verbatim; the 40-character marker window crosses the join),
  the other is the same view-source: URL pasted twice. Where the loss would
  have been severe it no longer happens: a message like
  这些都修复了吗？1. build_dsp_chain … 2. 卷积注册表用 is_quiescent() 剪枝。 keeps
  both items instead of just the question, and
  别在整手机了好吗？直接电脑上操作 keeps the request that used to fall in
  sentence two.

Does an address actually give the thing back?

Resolving is not the same as returning the right thing, so every address the
above regions wrote — 1,456 of them, none sampled — was put back through the
real recallSession (tests/recall-verify.mjs, page:
artifacts/zero-llm-recall.html):

The readback had to be fixed twice to get there, and both were product defects
rather than test artifacts: the expanded renderer filled one byte budget in
block order, so a long thinking block consumed it and the * bash line
never appeared (225 addresses read back without their call); and #N drilling
reused the preview's 300-character argument cap, truncating 222 of 1,004 command
lines. Every row is now checked against the call seq it names, and the negative
controls keep "1,456 resolved" from being a claim about a resolver that never
refuses anything.

What it still does not do: it never explains why something was rejected. No
zero-LLM format does (decision consistency is 0/4 for all of them); that is the
one axis still reserved for the LLM path.

Switch the format globally, or per session:

/blackhole-mode format                  # report the format in effect
/blackhole-mode format ledger           # global; takes effect on the next compaction
/blackhole-mode format index            # addresses + diagnostics, no prose
/blackhole-mode format blackhole        # back to the prose format
/blackhole-mode format ledger session   # this session only (needs a session id, i.e. a turn)
/blackhole-mode format default          # drop the global override
/blackhole-mode format default session  # drop this session's override

Settings page

The bundle also ships a browser half (lib/client.js), registered both as a
Settings section and as the plugin manager's row page. It shows the live format,
the fallback chain and the remote gate, switches the global values, and lets you
revoke per-session overrides. Setting a per-session override stays in the
command — a settings section has no session, so it cannot name one.

Two copy decisions are load-bearing, because both failed before:

• The page never shows the switch values. The format options read
  VCC 编译摘要 / 逐字原子台账 (what the algorithm emits), and the chain
  options read 仅本地 / 本地优先 / 仅模型 (what happens to your context). A
  control labelled blackhole | ledger is a control nobody can choose from.
• Each control carries one line of "what this does", not a rationale. No
  measured numbers, no "why this exists": that belongs here. A paragraph on
  screen is re-read on every flip.

The last card is a demo rendered by the engine, not by the page: GET /blackhole-mode/preview runs both real renderers (compileInstantRegion +
compileBlackholeSummary, and renderLedger) over a fixed 14-message incident
triage, with the live configuration, and returns both checkpoints with their
measured token counts. The page draws them side by side with the ratio, so the
difference between the two formats is visible instead of described. It is
read-only, pure, and degrades to one dim line when the route is missing.

The page talks to five routes that the dsh-blackhole/http unit
(inject = ['webServer', 'compaction'], mounted only inside the preset group)
registers on the host web server:

There is no client→host RPC for plugin state in DSH, so this is the same
webServer.register() bridge dsh-market uses for its whole UI. Both halves
read the one modeController seam, so the page and the command cannot disagree.

Behavior notes

• Images: DSH stores image bytes in the attachment store, which a remote
  compaction call cannot resolve; image blocks are carried as an explicit
  "image content omitted" text visit instead of being silently dropped. The
  local compiler renders them as [image] (seq N) labels.
• Checkpoint framing: the engine returns plain text; the inherited
  backend wraps it in the standard `` checkpoint framing.
• Deployments with agent presets: presets mount their own compaction realm.
  If a preset also compacts, either disable auto on one side or drop the
  preset's compaction rows — two engines must not compact the same session.
• Normal-turn opaque replay remains adapter-owned: DSH's immutable
  llm/stream request has no message-rewrite hook. This package therefore
  provides the optional responsesCompactionReplay service. A Responses
  adapter should call prepareInput({ sessionId, provider, model, trailingMessages, endpoint }) immediately before building its HTTP payload;
  it returns [owned opaque replacement history, ...trailingMessages], including
  a final incomplete user/tool turn for the request being sent, or a portable
  conversion of the supplied DSH messages when no owned artifact is available.
  replayHistory(...) remains available for the compaction-side variant, which
  deliberately retains only completed trailing turns. Foreign provider/model
  state, legacy artifacts, endpoint mismatches, and fallback compactions are
  rejected or removed rather than replayed. The Responses compaction path uses
  the completed-turn contract automatically on the next compaction.

Adapter wiring contract (responsesCompactionReplay)

The seam is one method call inside the adapter that serializes an ordinary
Responses request. tests/protocol-smoke.mjs pins the whole matrix as
executable assertions; this is the shape an adapter must implement:

// Inside the Responses adapter fiber:
const replay = ctx.get('responsesCompactionReplay') as
import('dsh-blackhole').ResponsesCompactionReplay | undefined

// Immediately before building the HTTP payload for a NON-compaction turn:
const input = replay === undefined
? dshMessagesToResponseItems(trailing)
: await replay.prepareInput({
sessionId,            // agent/session id (the opaque window is keyed by it)
provider,             // durable route: e.g. 'openai'
model,                // durable model: e.g. 'gpt-5'
trailingMessages: trailing, // the DSH Messages being sent this turn
endpoint,             // optional; pass the base URL if known
})
// builder.send({ model, input, stream: true, store: false, ... })

Contract semantics (all enforced by RemoteCompactionStateStore):

• Owned artifact (same provider/model/endpoint, versioned details):
  returns [opaque replacement items, ...input items fortrailingMessages],
  keeping a final incomplete user/tool turn (includePending) so the request
  being sent is not lost.
• No state / foreign route / legacy artifact / endpoint mismatch: returns
  the plain DSH→Responses conversion (fail closed — never a stale opaque
  window). replayHistory() returns undefined in exactly these cases.
• Compaction turns: do not call prepareInput() with the compaction
  intent — RoutingCompactionEngine already replays the completed-turn
  window internally when it summarizes. The service exists for ordinary
  conversation turns only.
• Fallback compactions invalidate the stored window (state.remove), so
  a later same-route turn starts from DSH messages and cannot skip the new
  local checkpoint.
• Pure-text regions compress less than tool-dense ones (tool results cost
  zero tokens in the compiled view); a region the compiler cannot shrink is
  rejected fail-closed, exactly like the upstream instant engine's shrink
  guarantee.

Development

pnpm install
pnpm run typecheck   # tsc --noEmit against the rc.6 packages
pnpm run build       # esbuild → lib/index.js
node tests/smoke-load.mjs      # loader composition mounts the engine
node tests/protocol-smoke.mjs  # conversion/SSE/round-trip vs a local fake endpoint
node tests/hybrid-smoke.mjs    # compiler unit + routing (remote/instant/llm/compat)
node tests/recall.mjs          # transcript/file recall + optional command/tool seams
node tests/equivalence/build-algal.mjs && node tests/equivalence/differential.mjs
# protocol equivalence vs the REAL upstream bundle
node tests/equivalence/build-blackhole.mjs && node tests/equivalence/differential-brief.mjs
# CONTENT-level differential vs the REAL pi-blackhole
# brief compiler (BRIEF-DIFF.md: 11/12 vectors aligned)
node tests/instant-blackhole-align.mjs
# regression suite pinning the aligned brief rules
node tests/eval-quality.mjs    # checkpoint-QUALITY baseline: compression ratio,
# entity survival (verbatim vs #N-pointed), pointer
# integrity, determinism — floor thresholds fail on
# regression.  Fixtures: two synthetic sessions
# (tests/eval-fixtures.mjs) + the REAL recorded pi
# transcript (DSH_EVAL_SESSION pins one; skips if none)
node tests/eval-profile.mjs    # real-session composition profile: where shadow &
# checkpoint tokens sit, truncation/elision losses,
# tool-call redundancy, pointer visibility — the
# evidence base for selection-work priorities
node tests/eval-compare.mjs    # same real session through the VENDORED upstream
# pi-blackhole brief compiler AND this package's
# instant compiler, one yardstick: size, entity
# recall by class, refs, loss-channel counters
node tests/equivalence/differential-real-session.mjs
# content differential on a REAL recorded pi session
# (live transcript, not synthetic vectors)
node tests/equivalence/entity-fidelity.mjs
# entity-level retention audit (paths/symbols/errors)
DSH_EVAL_KEY=... DSH_EVAL_BASE=... node tests/equivalence/eval-fidelity-llm.mjs
# LLM A/B: answer accuracy with full vs brief context
# (see FIDELITY.md; key/base/model/session via env)

Equivalence to the upstream port

tests/equivalence/ snapshots the upstream source at a pinned commit
(algal/SOURCE.json) and bundles it with pi dependencies stubbed (only the
computation helpers are replaced; the protocol code runs unmodified). The
differential test then feeds identical vectors to both implementations:
request body, replacement history (retention, 20K truncation, image
exemption, empty-message filtering), SSE parsing (success + every error
path), message conversion (pi↔DSH semantic projection), identity headers,
and a full local-endpoint round trip. 15/15 vectors pass.

Differences that remain are platform adaptation, not protocol drift:

• Eligibility detection: upstream auto-detects direct OpenAI / Codex
  models from the pi-ai registry (host must be api.openai.com); this port
  matches a declared endpoints (provider + model glob), so it can also
  target your own gateway. The upstream host gate would reject exactly the
  deployments this plugin is for.
• Checkpoint text source: upstream runs a parallel local summary and
  stores the remote artifact as details.remoteCompaction (pi has that
  slot); DSH's summarize() returns one text, so this port surfaces the
  server-side summary text as the checkpoint and falls back per
  fallbackMode. One LLM call at most — and by default, none.
• Images: pi carries image bytes (→ input_image); DSH only has
  attachment ids, so image visits become explicit placeholder text.
• Exposed knobs: retainedMessageTokenBudget (upstream's fixed 20K),
  timeoutMs, sendIdentityHeaders, installationId are extra options;
  defaults reproduce upstream behavior exactly.

License

MIT. Portions derived from
algal/pi-openai-server-compaction
(MIT) and the compiler rules of
dsh-compaction-instant
(MIT, VCC conversation-compiler port) — see LICENSE.