diff --git a/scout/SPIKE.md b/scout/SPIKE.md new file mode 100644 index 0000000..beb022f --- /dev/null +++ b/scout/SPIKE.md @@ -0,0 +1,218 @@ +# Scout Host Spike (T041) + +Time-boxed, read-mostly investigation of this host (m2, the machine this repo lives on) +to pick where the Solution Scout (Story 5 / FR-011) should live. Evaluates the three +options named in research.md §R6: (A) standalone supervised watcher, (B) Hermes plugin, +(C) herdr plugin/hook — against (a) access to structured summaries without raw +keystrokes, (b) opt-in enforcement point, (c) on-box summarization cost, (d) notification +surface, (e) update/rollout story across the `primus` and `agent-latest` image classes. + +Note: `specs/S4-solution-scout.md` §5 already drafted a recommendation (Option A) before +this spike ran. This report re-derives the decision from concrete file/path evidence +gathered on the live host, and **overrides that draft** — see §5 for why. + +## 0. Fleet context confirmed on this host + +- Two desktop image classes exist, and they are **not** interchangeable for rollout: + - `primus` — built from `/home/m2/m2o/desktop/Dockerfile`, this repo's own supervisord + config, canary machine `chris-m2o`. System/image changes here mean re-baking primus + (`/home/m2/m2o/desktop/README.md:74`, `:95-104`). + - `agent-latest` — `ghcr.io/machine-machine/m2-desktop:agent-latest`, described + repeatedly in `/home/m2/m2o/.planning/federated-learning/*` as "legacy/Coolify", + a **different image lineage** this repo does not build + (`m2o/.planning/federated-learning/proposals/A4-propagation-rollout.md:5,17`). + - The federated-learning propagation ADR is explicit: *"image/system changes are baked + into `m2-desktop:primus`... This avoids pretending that `primus` and legacy/Coolify + `agent-latest` machines can be updated the same way"* (A4:5). Constitution VI says the + same: "runtime-sync is the only universal install path across mixed desktop images." + - Concretely on this host, every desktop-level supervised program (`hermes-gateway`, + `herdr-integrations`, `ttyd`) is wired in by `Dockerfile` `COPY ... >> supervisord.conf` + steps at **build time** (`m2o/desktop/Dockerfile:73-126`) into + `/etc/supervisor/conf.d/supervisord.conf`. That file is not part of any volume that + runtime-sync touches — it only exists post-bake, inside the image. + +This single fact is the crux of the spike: **adding a new supervised process is an +image-bake operation**, reachable on `primus` (we own the Dockerfile) but not on +`agent-latest` (we don't own that image's build) without a separate, per-machine, +non-idempotent `docker exec` patch to a running container's supervisord config — exactly +the kind of fleet-wide-blast / non-reversible operation constitution VI rules out. + +## 1. Option A — Standalone supervised watcher + +Evidence: + +- Pattern to copy: `hermes-gateway.supervisor.conf` (`m2o/desktop/hermes/hermes-gateway.supervisor.conf`) + — `[program:hermes-gateway]`, `autorestart=true`, `priority=45`, `sleep 12` boot stagger, + appended into the active `supervisord.conf` via `Dockerfile:79-82`. +- Adding `m2-solution-scout` the same way means a third `COPY ... >> supervisord.conf` + block in `m2o/desktop/Dockerfile`, i.e. a `primus` image cut (`VERSION=vN ./build.sh`, + README.md:99). +- **Rollout gap (the finding S4 missed):** there is no equivalent Dockerfile for + `agent-latest` in this repo — it's pulled pre-built from `ghcr.io`. Reaching it means + either waiting for whoever owns that image to add the program, or `docker exec`-patching + each running `agent-latest` container's `/etc/supervisor/conf.d/supervisord.conf` and + `supervisorctl update` by hand per machine. That is not idempotent/reversible/runtime-sync + in the sense constitution VI and the "Inherited Hard Constraints" section require. + +Scoring: strongest privacy isolation (a dedicated process only reads what it's told to), +but the update/rollout story (e) is the weakest of the three on THIS fleet, not the +strongest as S4 assumed. + +## 2. Option B — Hermes plugin + +Evidence: + +- Hermes already runs a real plugin on this host: + `/home/m2/.hermes/plugins/herdr-agent-state/{__init__.py,plugin.yaml}`, installed by + `herdr integration install hermes` (confirmed via `herdr integration` subcommand list; + this exact plugin ships with `# HERDR_INTEGRATION_ID=hermes` header). It registers + hooks via `ctx.register_hook(...)` for `on_session_start`, `pre_llm_call`, + `pre_tool_call`, `post_tool_call`, `pre_approval_request`, `post_approval_response`, + `post_llm_call`, `on_session_end`, `on_session_finalize` — reporting only lifecycle + state (`working|blocked|idle`) over the herdr Unix socket, never message content. +- Plugin hook docs (`/home/m2/.hermes/hermes-agent/website/docs/user-guide/features/hooks.md`) + show the two hooks a Scout would actually want: + - `post_llm_call(session_id, user_message, assistant_response, conversation_history, + model, platform, **kwargs)` — fires once per turn, **already gives structured, + per-turn intent content** (not raw keystrokes — it's the resolved message/response + text, the same thing Hermes itself just processed). + - `pre_llm_call(...) -> {"context": str}` — could be reused later to inject a + "here's a matching Solution" hint directly into the next turn instead of only toasting. +- Deployment is a pure file drop: `~/.hermes/plugins/m2-solution-scout/{__init__.py,plugin.yaml}`. + No supervisord, no image bake — this is exactly the "runtime fleet-sync agent into + persisted home volumes" path the federated-learning ADR (A4) calls out as the one + path that reaches BOTH image classes, **because Hermes itself is already supervised + on both** (hermes-gateway is the canonical per-desktop supervision pattern referenced + by this very task, and its plugin directory lives under `$HERMES_HOME` regardless of + which image booted it). +- Opt-in enforcement point: hooks are in-process Python: a `pull-policy.toml` read at + plugin `register()` time can make every hook a no-op when `scout.enabled=false`, + exactly like the CLAUDE.md-documented pattern for other opt-in config. +- On-box summarization cost: cheapest of the three — the plugin receives already-clean + turn text; no separate log-tailing/parsing/redaction pass over external files is needed + before summarizing, only redaction of the text it's handed directly (same + `redact_secrets`/`redact_client_identifiers` obligations from S4's `pull-policy.toml` draft). +- Notification surface: none built in — the plugin still needs to shell out to + `notify-send` (XFCE) or the herdr toast socket (see Option C) for the actual popup. + This is a small, one-file bridge either way. + +Cons confirmed from the doc: plugin hooks run **in-process** with Hermes (same trust +boundary — a crashing hook is caught and logged per the docs, "never crashing the agent", +but a slow hook still adds latency to every turn). And it only sees Hermes sessions — +OpenClaw-only or bare-herdr (Claude Code/Codex panes with no Hermes chat active) sessions +are invisible to it. + +## 3. Option C — herdr plugin/hook + +Evidence: + +- `herdr agent list` (run once, per task instructions) returns a live, structured JSON + lifecycle feed today: `{agent, agent_status: working|idle|done, cwd, pane_id, + workspace_id, ...}` for every pane across every workspace on the box — genuinely + "structured run/lifecycle summary," zero raw keystrokes, already running, no code to write. +- `~/.herdr/runs/*.md` (e.g. `2026-07-02-m2-market-first-wedge-gate.md`) are rich, + human/agent-authored narrative summaries — but they are a **manual convention** written + by an orchestrating session at the end of a herd run (see this repo's own recent runs), + not an automatic per-session artifact. Most single-agent sessions never produce one. + This makes Option C's "structured summary" source sparse and bursty rather than + continuous — a poor fit for "an operator is mid-session, starts building something... an + agent proposes the link right then" (CONCEPT.md §5). +- `~/.claude/hooks/herdr-agent-state.sh` (installed by `herdr integration install claude`) + shows the wire protocol: a Claude Code hook posts `pane.report_agent_session` over + `$HERDR_SOCKET_PATH` — again lifecycle-only, no content. +- `~/.config/herdr/config.toml` already has `[ui.toast] delivery = "herdr"` — confirms + herdr owns a native toast/notification surface today (this is also directly reusable + by Options A/B as the notification backend, independent of which host wins). +- `herdr integration` subcommands (`install/uninstall {pi,omp,claude,codex,copilot, + opencode,hermes,qodercli}`, `status`) show integrations are themselves file-drops under + `~/.claude/hooks/`, `~/.hermes/plugins/`, etc. — same runtime-sync-friendly deploy story + as Option B. + +Scoring: best-in-class for (b) opt-in enforcement (a single socket message gate) and (d) +notification, but weakest for (a) — the only *continuous* signal herdr exposes natively is +coarse lifecycle state (working/idle/done + cwd), not intent. Getting real intent out of +herdr alone means depending on the sparse `runs/*.md` convention, which most sessions +don't produce. + +## 4. Scored comparison + +| Criterion | A: Standalone watcher | B: Hermes plugin | C: herdr plugin/hook | +|---|---|---|---| +| (a) structured intent w/o raw keystrokes | Depends entirely on tap sources it's given — none exist continuously today | **Best** — `post_llm_call` gives per-turn content already, every turn | Weak — only continuous signal is lifecycle state; real intent needs sparse `runs/*.md` | +| (b) opt-in enforcement point | Own config file, own gate — clean but net-new | Clean — gate inside `register()` before any hook does work | **Best** — one socket message, herdr already gates its own integrations | +| (c) on-box summarization cost | Highest — must tail/parse/redact multiple external files first | **Lowest** — handed clean text directly | Low for lifecycle, but high if it also has to parse `runs/*.md` | +| (d) notification surface | Must shell out (herdr toast or XFCE) | Must shell out (herdr toast or XFCE) | **Native** — herdr toast is its own surface, `[ui.toast]` already configured | +| (e) update/rollout, both image classes | **Weakest** — new supervisor program = image bake; no equivalent Dockerfile for `agent-latest` | **Best** — pure file drop under `$HERMES_HOME`, works wherever Hermes already runs (both classes) | Good — file drop under `~/.claude/hooks`/`~/.hermes/plugins` via `herdr integration install`, but only reaches herdr-managed panes | + +## 5. Recommendation + +**DECISION: Hermes plugin (Option B)** + +Rationale: FR-011 and Story 5's hardest constraints are (1) real per-turn intent +without raw keystrokes, and (5) a rollout story that doesn't require rebuilding either +image class. Concrete evidence on this host shows Option A fails (5) outright — the only +mechanism this repo has for registering a new supervised program is a `Dockerfile` COPY +into `supervisord.conf`, which has no `agent-latest` equivalent, making that option +image-bake-only on one of the two live classes. Option C is native for opt-in/notification +but its only continuous, low-cost signal is coarse lifecycle state — it would need Option +B's or Option A's tap to get real intent anyway. Option B satisfies (1) directly +(`post_llm_call` hands over already-resolved turn text, cheapest to summarize-on-box), and +(5) matches proven fact: this host already runtime-installs a real Hermes plugin +(`herdr-agent-state`) as a pure file drop, no image rebuild, and Hermes is the one +component already supervised identically across both `primus` and `agent-latest` +(that's the same "hermes-gateway pattern" this task was pointed at as reference). + +Caveat worth carrying into T042: a Hermes-hosted Scout is blind to bare-herdr sessions +that never touch Hermes chat (a Claude Code/Codex pane with no Hermes turn). This does +**not** violate the "Hermes first, never Hermes-only" inherited constraint — that +constraint is about marketplace touchpoints in general (Store/CLI/catalog remain +Hermes-independent), and Story 5's own acceptance criteria only require discovery "via +Scout **or** Hermes/CLI." It is, however, a real coverage gap: pure-herdr desktops get no +push-side discovery in v0. Cheap mitigation for a later phase, not blocking: also register +`herdr integration install hermes`'s existing socket report as a secondary opt-in trigger +so the same plugin can request the herdr toast surface (`[ui.toast] delivery = "herdr"`) +instead of shelling to `notify-send`, without adding a second host. + +## 6. v0 implementation sketch + +Components: + +- `~/.hermes/plugins/m2-solution-scout/` + - `plugin.yaml` — `name: m2-solution-scout`, `description: Solution Scout intent watch`. + - `__init__.py` — `register(ctx)` wires `on_session_start` (load `pull-policy.toml`, + no-op immediately if `scout.enabled=false` or `tenant_id` missing — fail closed, per + S4's policy rules) and `post_llm_call` (the intent tap). +- Config: reuse S4's existing `pull-policy.toml` schema verbatim + (`~/.config/m2-market/pull-policy.toml`, `[scout]` block, `raw_keystrokes`/ + `raw_client_data` reserved-deny keys, `max_proposals_per_day`, cooldowns). No new + config format — the plugin is a new *host* for the same contract S4 already specified. +- State/telemetry: same paths as S4 §3 (`~/.local/share/m2-market/scout/state.json`, + `outbox/YYYY-MM-DD.jsonl`) — the plugin process just runs inside Hermes instead of as + its own daemon. + +Event flow (per turn, inside `post_llm_call`): + +1. Gate: `scout.enabled` true, tenant_id present, rate cap and per-session cap not yet + hit, cooldown not active for the last dismissed listing → else return immediately. +2. Take `user_message` + `assistant_response` (bounded to `summary.max_source_chars`), + redact secrets/client identifiers per `pull-policy.toml`, produce + `m2.scout.intent_summary.v1` (S4 schema, unchanged) — deterministic extractive summary + in v0, no extra LLM call needed since the plugin already has clean turn text. +3. `POST /memory/search` to memory-api, `agent_id=market:catalog`, tenant-filtered (S4 §3 + catalog query, unchanged). +4. Score/coverage threshold check (S4's `m2.scout.match.v1`) → if it clears, build + `m2.scout.proposal.v1` and fire the toast (`notify-send` primary, herdr socket toast as + fallback/secondary surface) with the M2 Store deep link. +5. Write `proposal_shown` telemetry to local outbox (batched flush to `market:evidence`, + same as S4 §3/§4) — Scout never calls `m2-market install` itself; Store/CLI own + debit→grant→apply and emit the accept/dismiss events this plugin later folds back in. +6. All of steps 2-5 run off the hot path where possible (background thread, mirroring the + `boot-md` hook tutorial's pattern of not blocking the turn on non-essential work) so a + slow catalog query never adds latency to the operator's actual turn. + +Guardrails carried over unchanged from FR-011 / S4: default `enabled=false`; opt-in is +per-desktop `pull-policy.toml`; `raw_keystrokes`/`raw_client_data` keys hard-refuse start +if true; rate cap default 5/day; propose-only — install always requires the human's click +through M2 Store. + +DECISION: Hermes plugin