dsh-gme-test-generator

English | 中文

GME Test Generator workflows inside DeepSeek Harness (dsh): three tools that pick interfaces, drive autonomous test generation and repair in a local Python backend, poll the task until review, and gate outward actions behind explicit user consent.

This is a community plugin, not an official DeepSeek package, and it needs an existing GME Test Generator checkout with its Python dependencies — it drives that project rather than replacing it.

Requirements

• DeepSeek Harness 0.1.2-alpha.1 or newer on the 0.1.x line, with a base-backed profile that provides tools and systemPrompt.
• A GME Test Generator checkout containing backend/run_backend.py, its config.local.json, its task database, and the GME repository and compiler toolchain it needs. The backend is not bundled with this plugin: its public copy — the framework, without GME-specific generated data — is nuaaweixinye/gme-agent, which documents cloning, configuration and how to generate the interface catalogs locally: clone it, run scripts\install.ps1 -GmeRepo, then scripts\run_web.ps1. The full checkout, which additionally carries those generated catalogs and the internal notes, is private: access is granted per person by @nuaaweixinye.
• A Python interpreter with that backend's dependencies installed, plus matching deepseek-harness-sdk and deepseek-harness-runtime-bin wheels in it, and clang-format 17.0.2 — the version behind GME's own check-format target; the backend refuses to judge formatting with another major version (section 2 of the setup doc).
• For automatic startup, a local subprocess service in the profile (every shipped profile has one).

The Python backend runs its coding work through the DeepSeek Harness Python SDK in a separate sdk profile. Build, tests and the memory audit run inside the backend as automatic stages of every task, not as chat actions. That coding profile includes file, search and PowerShell tools and excludes this plugin, so tasks never recurse.

The backend repository ships its own reference docs: docs/backend-overview.md covers how it works and what it guarantees (the job lifecycle, action preconditions, the authoritative what-counts-as-passing rules, and the config reference), and docs/knowledge-injection.md describes its optional, off-by-default capability of injecting historical divergences and knowledge-base references before generation. That capability is configured in the backend's own config.local.json; the one intersection with this plugin is the owned worker's environment — see section 6 of the setup doc.

Install

From the plugin market (Settings → Plugin Market) — one click — or:

dsh plugin --profile web add dsh-gme-test-generator

The command installs the package and appends it to the profile's dsh.profile.bundles; this package ships a dsh.bundle.patch layer, so no profile file needs editing. Restart dsh web afterwards.

Configure

backendRoot is a deployment path, so it has no default and is never a model argument. While it is unset the plugin is still mounted: it registers no tools, logs one warning, and publishes the setup procedure to the model, so an agent asked for a GME task can explain what is missing and how to finish the setup instead of answering "no such tool". It never throws — a row whose config fails validation would take the whole plugin tree down with it.

Option 1 — environment variables, read when Harness starts:

$env:GME_TEST_GENERATOR_ROOT  = 'D:/workspace/gme-test-generator'
$env:GME_TEST_GENERATOR_PYTHON = 'C:/ProgramData/Miniconda3/envs/agent/python.exe'   # optional, default 'python'
dsh web

Option 2 — a profile patch, in $DSH_HOME/profiles//cordis.patch.yml:

- id: gme-test-generator
config:
backendRoot: D:/workspace/gme-test-generator
pythonPath: C:/ProgramData/Miniconda3/envs/agent/python.exe
port: 8765
autoStart: true

A patch row replaces only the keys it names, entire config included, so restate every field you keep. Restart dsh web afterwards; the tools appear as soon as a backendRoot resolves.

If an older profile separately enables tool-gme, disable that row; installing this plugin does not remove other packages.

Tools

For tests and extension, pass either catalog interface_ids or a free-form goal, never both: the backend discards a free-form goal when IDs are supplied, and the plugin rejects the combination. Batches require IDs. Query interfaces before choosing IDs, and use backend job IDs rather than Harness session IDs.

Generation runs autonomously from acceptance to needs_review; the model polls with gme_check and does not steer the intermediate build, test and memory-audit stages. Every response carries a suggested_next signpost whose phase moves poll → report → decide → done: keep polling while a task executes, report the summary, failures and diff at needs_review, and leave outward steps to the user. A gme_decide call without confirm: true fails with guidance and never reaches the backend.

Generation and decisions can return accepted: true; that means queued work, not successful validation. Report pages expose content (a slice of serialized JSON), total_characters and next_offset; repeat the same query with that offset. Growing lists may shift between pages; use bounded incremental events.after queries for live progress and completed artifacts for stable reports. A returned job with status: failed is a valid query result; infrastructure failures appear as tool errors.

Behaviour and limits

• Worker lifetime. The first request reuses a server only after an authenticated health response; otherwise autoStart: true starts the configured Python entrypoint, and concurrent calls share that startup. Authentication failure, or another service on the port, fails without starting a worker. Disposal terminates only a backend this plugin started, including its children — so closing or reloading Harness can interrupt owned jobs, while an independently started backend survives. An owned worker that exits is restarted by a subsequent request; the interrupted job is not retried for you. Aborting a tool stops waiting but does not cancel an accepted backend job, and POST requests are never retried automatically: inspect tasks after an uncertain submission.
• Credentials. The token comes from tokenFile. It never appears in tool arguments, and only the API token is explicitly forwarded to the managed child. The coding SDK uses the dsh_home and dsh_profile configured in the backend; credentials must exist there. The outer workflow dialogue and each backend coding session keep separate histories.
• Known limitations. The Python checkout and its toolchain stay required; the backend has no cancellation endpoint and no automatic restart recovery; the gme_decide consent gate is a plugin-side confirm: true check, while the backend still applies its own submission and cleanup rules; free-form tasks inherit the backend's own selection and validation behaviour; large reports are character windows, not immutable snapshots or structured tables.

Development

pnpm install
pnpm run verify        # typecheck + build + tests + packaged-artefact smoke
pnpm run test:live     # read-only smoke against a real backend (needs GME_TEST_GENERATOR_ROOT)

src/backend.ts owns authenticated transport and worker lifetime, src/index.ts owns tool schemas, route mapping, presentation and the unconfigured-mount guard, and src/next-step.ts maps backend statuses to the suggested_next signpost. tests/install.spec.ts composes the committed cordis.patch.yml through the include's real patch engine and mounts the resulting row into a real Loader tree. No invariant companion is published: backend state is authoritative and the plugin keeps no duplicate durable task state.

License

MIT