specgate/docs v0.20.0

The harness#

The harness delivers requirements unattended and stops when the gate says no. It sits between an archived specification and a person who merges.

The lifecycle of a requirement Three phases in order. Specify: keep the brief in the repository, draft a module as a change, check the draft, archive it. Build: status names the next requirement, the harness or a person builds it in its own branch, a person reviews and merges. Prove: done marks it Implemented only if the gate passes, check runs the whole gate in CI, report shows where the product stands. A requirement that arrives waiting for an answer stays out of Build until the client answers, then rejoins at status. SPECIFY BUILD PROVE 1 Brief docs/brief.md 2 Draft one module specgate draft <module> 3 Check the draft specgate draft <module> 4 Archive — all Draft specgate change archive 5 What is next specgate status 6 Build, one branch each specgate harness run 7 Review and merge a person, never the harness 8 Mark it done specgate done 9 The gate, in CI specgate check 10 Where it stands specgate report Waiting for an answer a question blocks it: the harness and done refuse it — answered → 5 Nothing is built before it is specified; nothing counts as done until the gate has checked it.

One turn, in sequence#

One turn of the harness, per requirement Five steps in a row. The harness picks the next pending requirement, never one waiting for an answer; creates a git worktree on a branch named after it; builds a prompt from its Gherkin, the rules and the boundaries; your agent writes code and tests; the gate runs strict validation, checks the specification was not touched, and runs your tests. Red: the failures go into the next prompt, up to the maximum number of attempts. Out of attempts: the work is committed as a failed attempt and the requirement stays Draft. Green: the requirement is marked done and committed on its branch, and the harness moves to the next pending requirement. It never merges. 01 Pick the next pending requirement — never one waiting for an answer 02 Worktree its own branch: harness/REQ-007 03 Prompt its Gherkin, the rules, the boundaries 04 Your agent writes code and tests — any CLI, no SDK 05 The gate validate --strict spec untouched your tests ✔ Green marked done, committed on its branch ✖ Out of attempts wip(…): FAILED the gate stays Draft — done never ran ✖ red — the failures go into the next prompt · up to max_attempts next pending requirement Then it reports. It never merges — the branch is yours to merge or throw away.

Each pending requirement (or the one --req names) gets a fresh worktree on harness/REQ-NNN and a prompt built from its Gherkin. The gate is validate --strict, an untouched specification, then your tests. Green marks it done on its branch; red feeds the failures into the next attempt, up to max_attempts, and after the last one the work is kept as a wip(REQ-NNN): FAILED the gate commit with the requirement still Draft. A requirement waiting for an answer is refused. Nothing is ever merged.

Set it up#

bash
specgate harness init

Writes harness.config.yaml and .harness/prompt-prefix.md, detects the gate from your build files, and leaves agent: unset — which agent runs the loop is your choice and your credentials.

Name the agent when you run it:

bash
specgate harness run --req REQ-001 \
  --agent "claude -p --allowedTools Read Write Edit Glob Grep 'Bash(npm:*)' < {prompt_file}"

The tool allowances matter. An agent in non-interactive mode has no way to ask permission, so without them it reads the prompt, cannot write a file, and the attempt is wasted. Scope them rather than reaching for --dangerously-skip-permissions: the agent needs Bash to run the project's build, and Bash(npm:*) is enough for a Node project. It works in a throwaway git worktree, which is the whole reason the harness uses one.

Other agents take the prompt differently — aider --yes --message-file {prompt_file}, opencode run "$(cat {prompt_file})". The harness only requires that the command contain {prompt_file}.

A tool that fits none of those shapes needs a wrapper, and a command without {prompt_file} is refused rather than silently run. → Choosing and wiring your agent covers both, with the commands for the tools people actually use.

The agent works in a fresh worktree, which carries only what git tracks — no node_modules, no target/, no .venv. An agent must not spend its attempt installing dependencies: put that in the gate command (--test-cmd "npm ci && npm test"), the only part that knows how. Found the expensive way, by an agent that spent its first attempt on npm install and timed out.

Or commit the commands your team uses and pick one by name:

yaml
# harness.config.yaml
agent_profile: local-claude

# .harness/profiles.yaml
profiles_version: 1
profiles:
  local-claude:
    agent: "claude -p < {prompt_file}"
  ci:
    agent: "aider --yes --message-file {prompt_file}"

Write profiles in block form, as above: the reader is a dependency-free YAML subset, and an inline mapping (local-claude: { agent: "…" }) parses as a string, leaving the profile with no agent.

A profile may also declare match: and select itself, so one run gives different requirements different agents — and different tool allowances — rather than the greatest common denominator of the whole plan. → A profile per requirement

Parallel branches and the traceability matrix#

Every harness branch flips its own row in docs/specs/traceability.md. Git merges by lines and needs an unchanged line between two changed regions to treat them as independent — and matrix rows are consecutive lines. So two branches that touched different requirements still collide, purely because their rows are neighbours. Measured on a three-requirement project: rows 1 and 2 conflict, rows 1 and 5 merge clean.

specgate harness init sets up a merge driver that merges the matrix by row instead of by line:

bash
specgate harness init --project-dir .   # writes .gitattributes, registers the driver
git add .gitattributes && git commit -m "merge the matrix by row"

It resolves the case it exists for — two branches, two different rows — and it still conflicts when two branches set the same row to different values, because quietly discarding somebody's decision is worse than asking.

Every clone registers it once. .gitattributes is committed, but merge.specgate-matrix.driver is local git config that nothing can commit, so a fresh checkout and every CI job needs specgate harness init too. Until then git falls back to its built-in merge — the conflict the project had before, never a silently wrong result — and specgate doctor reports the gap rather than leaving it to be found mid-merge.

Resolving one by hand#

When two branches really did change the same row, the file gets git's usual markers around that one row:

text
<<<<<<< ours
| REQ-002 | SCN-002 | … | Implemented |
=======
| REQ-002 | SCN-002 | … | Verified |
>>>>>>> theirs

Keep exactly one of the two lines and delete the three marker lines. Never keep both: a duplicated requirement is the one corruption this file must not have — it is also what a naive merge=union produces, which is why that is not what specgate configures. Then run specgate validate ., which checks the statuses are legal and that no scenario id repeats.

Change agent between attempts#

Three attempts with the same agent, the same model and only a different prompt is mostly a re-roll. A ladder gives each attempt its own profile, and can put an advisory reviewer in front of every retry:

yaml
# harness.config.yaml
attempt_profiles:
  - implementer # attempt 1
  - repairer # attempt 2
  - repairer-strong # attempt 3, and any beyond it
review_profile: reviewer

# .harness/profiles.yaml
profiles:
  implementer:
    agent: "claude -p < {prompt_file}"
  repairer:
    agent: "claude -p < {prompt_file}"
  repairer-strong:
    agent: "claude --model opus -p < {prompt_file}"
  reviewer:
    agent: "claude -p < {prompt_file}"
    advisory: true

A ladder shorter than max_attempts reuses its last rung. Declaring neither key keeps exactly the behaviour you have today: one agent, every attempt.

The reviewer advises; it never approves. It runs before each retry — never before the first attempt, since there is nothing to review yet — and its output is added to the next prompt as findings. Then everything it touched in the worktree is discarded, so it cannot reach the gate even if it tries to write code. validate --strict plus your test command stay the only judge, and a finding with the gate green does not block anything.

A profile named by review_profile must declare advisory: true. Without it the harness refuses to start, because a reviewer whose work was gated and committed is not a reviewer.

Every agent is still bound to exactly one requirement: an attempt may run two roles, but both work on the same REQ, in the same worktree, against the same gate. .harness/runs/<ts>.json names the roles that ran each attempt, and the branch carries one archived prompt per role — attempt-2-reviewer.md, attempt-2-implementer.md — so a reviewer can see exactly what each was asked.

Run more than one requirement at a time#

bash
specgate harness run --concurrency 4

Only requirements that do not depend on each other ever run together. The harness reads the dependency graph — the depends= keys in each requirement's specgate:trace comment — and processes the queue in levels: everything in a level is independent, so it can go out to the pool at once; the next level waits.

A failure blocks what waits on it; it does not fail it. Before dependencies were expressible, one broken predecessor produced a failure for every requirement behind it — N failures for one cause, and N agent invocations paid for work that could not have succeeded. Now the report says so:

text
  ❌ REQ-001  fail (1 attempt)     → harness/REQ-001
  ⛔ REQ-002  blocked (0 attempts) → harness/REQ-002
       Not attempted: REQ-001 did not pass.

  0 passed · 1 failed · 0 skipped · 1 blocked

A requirement caught in a dependency cycle is reported the same way and never attempted; specgate validate is what explains the cycle and how to break it.

The base is derived, not passed. A requirement is cut from the branch of the dependency it builds on, because that is the only place its code exists during a run. Nobody has to know the order and pass --base-branch by hand — which used to cost a whole agent run when the base turned out to be missing a fix that had landed on main. The harness now says so when it happens:

text
⚠️  REQ-002: base harness/REQ-001 is 3 commit(s) behind main.
    A fix that landed on main is not in this worktree —
    a gate failure may not be about REQ-002.

A requirement with several dependencies gets them merged. Their branches know nothing of each other, so cutting from one would silently omit the rest. The harness assembles a throwaway harness/base/REQ-NNN first. Two things worth knowing about it:

  • It resolves conflicts in docs/specs/traceability.md by keeping the base's version. Sibling branches always conflict there — each run ends with specgate done, editing the same table — and the integration base exists only so the agent can see code. Each real harness/REQ-NNN branch keeps its own row.
  • A conflict in an actual source file blocks the requirement and names the file. That is a genuine finding: two dependencies changed the same code in incompatible ways, and no automatic answer would be right.

This is not the merge the harness refuses to do. That one is into a branch a human reviews; this one assembles the context a requirement was declared to need, and it is thrown away afterwards.

The default is 1, deliberately. Two reasons, and neither is timidity:

  • Every step of a requirement — the gate, the agent, specgate done, git — is a blocking call, so above 1 each requirement runs in a worker process. That path is newer than the serial one, which has years of real agent runs behind it.
  • Parallelism moves the bottleneck rather than removing it. Four green branches are four reviews, and the harness still never merges.

concurrency: 4 in harness.config.yaml sets it for the project.

What the harness has cost#

specgate harness report reads every recorded run and prints two numbers: how often the first attempt worked, and the wall-clock cost per delivered requirement. → What the harness has cost

When a run fails#

The report prints the tail of the gate output and the command that failed. Some failures end it after one attempt: when retrying cannot help.

The attempt is committed on the branch with a wip(REQ-NNN): FAILED the gate subject, so the agent's work is there to read instead of discarded. The requirement stays Draft, because specgate done never ran.

bash
specgate harness run --req REQ-002 --format json      # the whole gate output
specgate harness run --req REQ-002 --keep-worktrees   # reproduce it in place

A requirement that builds on another needs its branch as the base:

bash
specgate harness run --req REQ-002 --base-branch harness/REQ-001

That branch also supplies the project configuration for the run, so a fix committed to main does not apply to a stacked run until the base has it. A false failure from that is indistinguishable from a real one — check the gate command the report prints.

Next#