m2-market/specs/S4-solution-scout.md
m2 (AI Agent) 8a8db9f2cc specs: m2-market spec discovery — 6 subsystem specs + synthesis index
herdr workforce output (6 codex angle-specialists -> claude synthesis):
S1 ledger/economy, S2 catalog/registry, S3 storefront (cargstore->M2 Store),
S4 Solution Scout (standalone watcher recommended), S5 packaging + 3 seed
Solutions (mm-pdf 25cr, agent-scaffold 35cr, competitor-scan 60cr),
S6 proposal engine + evidence loop.

SPEC-INDEX is the tie-break: 23-row cross-cutting contract table (16 mismatches
resolved, e.g. /v1 ledger paths win, m2store://listing/<id> deep link, one
m2.market.telemetry.v1 envelope, op_<slug> operator ids, tenant ids from m2-gpt),
8 deduped clarifications with defaults, 4-phase build order where the paid-install
wedge does NOT block on fedlearn (local ApplyAdapter until rails land), risk register.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 02:43:50 +02:00

537 lines
21 KiB
Markdown

# S4 — Solution Scout Spec
## 1. Scope & non-goals
### MVP scope
Solution Scout v0 is a per-desktop, propose-only service that notices when an operator is already working on a task that a published M2 Marketplace listing may cover, then surfaces a non-blocking proposal.
Recommended host: **standalone supervised watcher** inside each desktop container, deployed canary-first on `chris-m2o`, then one `agent-latest` canary such as `gunnar-m2o`. It reads only already-structured local summaries, writes only proposal telemetry, and calls the existing catalog/store/install contracts.
MVP responsibilities:
- Watch safe input taps:
- herdr run summaries from `~/.herdr/runs/*.md` and herdr session metadata from `~/.config/herdr/session.json` when present.
- Hermes/OpenClaw session summaries only if the harness writes an explicit summary file or emits a summary event. No raw chat logs unless another subsystem has already summarized and scrubbed them.
- Optional window title summaries only when explicitly enabled in policy; default off.
- Summarize on-box before any external call. The Scout sends a short intent summary, not raw workspace/client text.
- Query the semantic catalog through memory-api using partition/agent namespace `market:catalog`, tenant-filtered by the caller tenant.
- Decide whether to propose using confidence, coverage, recency, rate-limit, dismissal cooldown, and policy thresholds.
- Surface proposals as:
- herdr/XFCE toast with a deep link into M2 Store.
- Hermes chat surfacing when Hermes is active and exposes the market-propose bridge.
- Persist local proposal state and write accept/dismiss/show telemetry to `market:evidence`.
- Respect opt-in config in `~/.config/m2-market/pull-policy.toml`.
### Later scope
- Full Hermes-native agent reasoning for build-vs-install proposals belongs to S6. S4 only hands Hermes a candidate proposal event.
- Automatic install is out of scope. Human clicks through M2 Store and `m2-market install` handles debit/grant/apply.
- Raw keystroke capture, screen recording OCR, shell history capture, browser DOM scraping, and unreviewed client data extraction are explicitly out of scope.
- Fleet-wide rollout, packaging format, catalog truth, storefront implementation, and ledger internals are owned by S5/S2/S3/S1 respectively.
## 2. User stories + acceptance criteria
### Story 1 — Opt-in canary proposal
Given `chris-m2o` has `~/.config/m2-market/pull-policy.toml` with `scout.enabled = true`
and a herdr summary describes a competitor research report,
when Scout runs its poll loop,
then it creates an on-box intent summary, queries `market:catalog` within the machine tenant, and shows at most one proposal for `competitor-scan` if score and coverage exceed thresholds.
Acceptance:
- The payload sent to memory-api contains no raw run body longer than `summary.max_chars`.
- The proposal contains listing name, price, confidence/coverage, install count/rating if present, and Store deep link.
- `~/.local/share/m2-market/scout/state.json` records the proposal id and cooldown.
### Story 2 — Human-controlled install path
Given a proposal toast is shown,
when the operator clicks Open,
then M2 Store opens `m2store://listing/<listing_id>?proposal_id=<proposal_id>&source=scout`.
Acceptance:
- Scout never calls `m2-market install` directly.
- Store/CLI remain responsible for ledger `/tx/install`, license grant, bundle verification, apply adapter, and refund on failure.
- Scout observes accept/installed telemetry only after Store/CLI emits an explicit event.
### Story 3 — Dismissal tunes future proposals
Given an operator dismisses a proposal,
when the same listing/task pair appears again within the configured cooldown,
then Scout suppresses the proposal and writes a dismiss event to evidence.
Acceptance:
- Dismissal is keyed by `{tenant_id, operator_id, desktop_id, listing_id, intent_hash}`.
- Dismiss telemetry is stored with a summary hash and reason, not client content.
- Listing evidence can fold `proposals_shown` and `proposals_accepted` into `listing.stats`.
### Story 4 — Tenant firewall
Given the desktop belongs to `sdjs`/GST or another client tenant,
when Scout queries the catalog,
then only listings with `tenant_visibility` containing that tenant or `"*"` are eligible.
Acceptance:
- Empty tenant filter is a hard error, not an unscoped search.
- Tenant-private summaries never become shared catalog data.
- Owner-initiated commercialization and double scrub remain packaging/catalog obligations, not Scout behavior.
## 3. Interfaces & data contracts
### Process and deployment
Process name: `m2-solution-scout`
MVP placement:
- Desktop binary: `/usr/local/bin/m2-solution-scout`
- Supervisor config: `/etc/supervisor/conf.d/m2-solution-scout.conf`
- User config: `~/.config/m2-market/pull-policy.toml`
- Local state: `~/.local/share/m2-market/scout/state.json`
- Local event outbox: `~/.local/share/m2-market/scout/outbox/YYYY-MM-DD.jsonl`
- Logs: `~/.local/share/m2-market/scout/scout.log`
Deploy via runtime-sync / apply adapter, not image bake, until canary acceptance proves it. The service must tolerate missing herdr/Hermes/OpenClaw sources and run as degraded rather than crash-looping.
### `pull-policy.toml`
```toml
[scout]
enabled = false
mode = "suggest" # enum: off|suggest. No take-control mode in S4.
desktop_id = "chris-m2o"
operator_id = "op_chris"
tenant_id = "machine-machine"
min_score = 0.78
min_coverage = 0.55
poll_interval_seconds = 90
max_proposals_per_day = 4
max_proposals_per_session = 1
dismiss_cooldown_days = 14
proposal_cooldown_minutes = 45
[scout.sources]
herdr_runs = true
session_summaries = true
window_titles = false
raw_keystrokes = false
raw_client_data = false
[scout.summary]
max_source_chars = 12000
max_summary_chars = 900
redact_secrets = true
redact_client_identifiers = true
fail_closed_on_secret = true
[scout.catalog]
memory_api_url = "https://memory.machinemachine.ai"
agent_id = "market:catalog"
limit = 5
routing_strategy = "standard"
[scout.store]
deeplink_scheme = "m2store"
fallback_command = "m2-market show --json"
[scout.telemetry]
enabled = true
agent_id = "market:evidence"
batch_seconds = 60
```
Policy rules:
- `enabled=false` disables all watching and all outbound calls.
- `tenant_id` is required. If missing, Scout exits non-zero and writes no events.
- `raw_keystrokes` and `raw_client_data` are reserved deny keys. If true, Scout refuses to start.
- All keys are runtime-injected/configured. No secrets in this file except references to existing machine config; memory/ledger keys remain in `~/.m2-market/config.toml` with mode `0600`.
### Input event schema
Scout normalizes safe local taps into `ScoutObservation`.
```json
{
"schema_version": "m2.scout.observation.v1",
"observation_id": "obs_20260702_chris_01j1",
"desktop_id": "chris-m2o",
"operator_id": "op_chris",
"tenant_id": "machine-machine",
"source": "herdr_run",
"source_uri": "file:///home/developer/.herdr/runs/2026-07-02-fedlearn-seed.md",
"source_ts": "2026-07-02T00:15:00Z",
"session_id": "herdr:2026-07-02-fedlearn-seed",
"safe_excerpt": "Short scrubbed excerpt or heading only",
"source_hash": "sha256:...",
"metadata": {
"image_class": "primus",
"harness": "herdr"
}
}
```
Constraints:
- `safe_excerpt` is optional and capped by policy; raw source body is never persisted in telemetry.
- `source_hash` is computed from local source text for dedup, but only the hash leaves the box.
- For herdr MVP, source files are the live path verified on `chris-m2o`: `~/.herdr/runs/*.md`; config/session files are under `~/.config/herdr/`.
### On-box intent summary schema
```json
{
"schema_version": "m2.scout.intent_summary.v1",
"summary_id": "sum_20260702_chris_01j1",
"observation_ids": ["obs_20260702_chris_01j1"],
"desktop_id": "chris-m2o",
"operator_id": "op_chris",
"tenant_id": "machine-machine",
"session_id": "herdr:2026-07-02-fedlearn-seed",
"summary": "Operator is preparing a competitor research report with source gathering and final PDF/report output.",
"intent_terms": ["competitor research", "report", "sources", "PDF"],
"negative_terms": ["private payroll", "credentials"],
"scrub_status": {
"secrets_redacted": true,
"client_identifiers_redacted": true,
"failed_closed": false
},
"intent_hash": "sha256:...",
"created_at": "2026-07-02T00:35:00Z"
}
```
Summarization can be deterministic extractive in MVP. If an LLM is used later, it must call m2-gpt with the desktop tenant key and still send only the capped summary onward.
### Catalog query
Use existing memory-api `/memory/search` surface rather than a new Scout API.
Request:
```http
POST https://memory.machinemachine.ai/memory/search
X-API-Key: <memory api key from ~/.m2-market/config.toml>
Content-Type: application/json
```
```json
{
"query": "Operator is preparing a competitor research report with source gathering and final PDF/report output.",
"agent_id": "market:catalog",
"routing_strategy": "standard",
"limit": 5,
"tenant_id": ["machine-machine", "*"],
"filters": {
"tenant_id": ["machine-machine", "*"]
}
}
```
Response is the live `RoutedSearchResponse` shape from memory-api: `{results, routing}`. Each result must carry listing payload metadata matching `schemas/listing.schema.json`. The matcher discards any hit whose listing payload is missing, delisted, wrong tenant, or below threshold.
### Match decision schema
```json
{
"schema_version": "m2.scout.match.v1",
"match_id": "mat_20260702_chris_01j1",
"summary_id": "sum_20260702_chris_01j1",
"listing_id": "lst_competitor-scan",
"solution_id": "sol_competitor-scan",
"tenant_id": "machine-machine",
"score": 0.84,
"coverage": 0.72,
"reasons": [
"intent overlaps listing intent",
"listing supports report generation",
"price/evidence present"
],
"suppressed": false,
"suppression_reason": null,
"created_at": "2026-07-02T00:35:03Z"
}
```
Coverage is Scout's local estimate of how much of the current intent is covered by the listing. It is not authoritative pricing. S6 owns full build-vs-install composition.
### Proposal schema
```json
{
"schema_version": "m2.scout.proposal.v1",
"proposal_id": "prp_20260702_chris_01j1",
"match_id": "mat_20260702_chris_01j1",
"desktop_id": "chris-m2o",
"operator_id": "op_chris",
"tenant_id": "machine-machine",
"listing": {
"listing_id": "lst_competitor-scan",
"name": "Competitor Scan",
"summary": "Research report package for competitor positioning.",
"price": {"amount": 40, "currency": "m2cr", "model": "fixed"},
"stats": {"installs": 12, "rating": 4.6, "proposals_shown": 20, "proposals_accepted": 7}
},
"confidence": 0.84,
"coverage": 0.72,
"deeplink": "m2store://listing/lst_competitor-scan?proposal_id=prp_20260702_chris_01j1&source=scout",
"shown_at": "2026-07-02T00:35:04Z",
"expires_at": "2026-07-02T01:20:04Z"
}
```
Toast copy should be non-blocking and specific:
`This looks like Competitor Scan: ~72% coverage, 40 m2cr. Open in M2 Store?`
### Proposal UX interfaces
MVP toast command:
```bash
notify-send "M2 Solution Scout" "This looks like Competitor Scan: ~72% coverage, 40 m2cr. Open in M2 Store?" --action=open="Open" --action=dismiss="Dismiss"
```
If `notify-send` actions are unreliable in XFCE/VNC, fallback to herdr toast delivery. The existing herdr config supports `[ui.toast] delivery = "herdr"`, verified in `/home/m2/m2o/desktop/herdr/config.toml`.
M2 Store deep link:
```text
m2store://listing/{listing_id}?proposal_id={proposal_id}&source=scout
```
Store fallback command:
```bash
m2-market show {listing_id} --json
```
Hermes surfacing event, written to a local bridge file/socket for the `market-propose` skill:
```json
{
"schema_version": "m2.hermes.market_proposal.v1",
"event_type": "market.proposal.available",
"proposal_id": "prp_20260702_chris_01j1",
"session_id": "herdr:2026-07-02-fedlearn-seed",
"message": "I found an existing Solution that may cover this task.",
"deeplink": "m2store://listing/lst_competitor-scan?proposal_id=prp_20260702_chris_01j1&source=scout"
}
```
The Hermes bridge is best-effort in MVP. If unavailable, toast + Store link is sufficient.
### Telemetry event schema
Telemetry is append-only, locally spooled, then written to memory-api `market:evidence` via `/memory/store`.
```json
{
"schema_version": "m2.scout.telemetry.v1",
"event_id": "evt_20260702_chris_01j1",
"event_type": "proposal_shown",
"proposal_id": "prp_20260702_chris_01j1",
"listing_id": "lst_competitor-scan",
"solution_id": "sol_competitor-scan",
"operator_id": "op_chris",
"desktop_id": "chris-m2o",
"tenant_id": "machine-machine",
"session_id": "herdr:2026-07-02-fedlearn-seed",
"intent_hash": "sha256:...",
"score": 0.84,
"coverage": 0.72,
"reason": null,
"created_at": "2026-07-02T00:35:04Z"
}
```
Allowed `event_type`:
- `proposal_shown`
- `proposal_opened`
- `proposal_dismissed`
- `proposal_accepted`
- `install_started`
- `install_succeeded`
- `install_failed`
- `proposal_suppressed`
Memory write:
```http
POST https://memory.machinemachine.ai/memory/store
X-API-Key: <memory api key>
Content-Type: application/json
```
```json
{
"content": "proposal_shown lst_competitor-scan score=0.84 coverage=0.72",
"agent_id": "market:evidence",
"memory_type": "semantic",
"importance": 0.4,
"source": "system",
"tenant_id": "machine-machine",
"session_id": "herdr:2026-07-02-fedlearn-seed",
"metadata": {
"schema_version": "m2.scout.telemetry.v1",
"event_id": "evt_20260702_chris_01j1",
"event_type": "proposal_shown",
"listing_id": "lst_competitor-scan",
"proposal_id": "prp_20260702_chris_01j1",
"intent_hash": "sha256:..."
}
}
```
Telemetry must be batchable and retryable. If memory-api is down, Scout keeps local JSONL outbox and continues proposing within local rate limits.
### CLI verbs
Scout management CLI:
```bash
m2-solution-scout run
m2-solution-scout once --json
m2-solution-scout status --json
m2-solution-scout dismiss <proposal_id> [--reason not-relevant]
m2-solution-scout open <proposal_id>
m2-solution-scout flush-telemetry
```
`once --json` is the canary acceptance hook: read sources, summarize, query, and emit a proposal or suppression decision without daemonizing.
## 4. Integration contract with other subsystems
### S1 ledger
Scout does not debit credits. It includes listing price in the proposal and links to Store. Install remains:
`M2 Store -> m2-market install -> POST /tx/install -> license grant -> apply -> telemetry`
Scout listens for Store/CLI install result events and writes evidence only. Refund behavior remains S1/S3/CLI.
### S2 catalog
Scout reads `market:catalog` only through memory-api `/memory/search` using `agent_id = "market:catalog"`. Listing payload must validate against `/home/m2/m2-market/schemas/listing.schema.json`.
Scout writes proposal evidence to `market:evidence`. S2/indexer folds proposal counts into `listing.stats.proposals_shown` and `listing.stats.proposals_accepted`.
### S3 store
Scout proposal opens:
`m2store://listing/{listing_id}?proposal_id={proposal_id}&source=scout`
Store shows evidence, price, permission diff, and calls `m2-market install`. Store should emit local result events:
- `~/.local/share/m2-market/events/install-events.jsonl`, or
- a local socket declared by S3.
Until S3 finalizes this, Scout can infer only `proposal_opened` and `proposal_dismissed`.
### S5 packaging
Scout is installed as a Solution/fleet component via the same runtime-sync path and canary policy. It must not require rebuilding primus or agent-latest images. Package should include:
- binary/script
- supervisor conf
- default disabled `pull-policy.toml`
- rollback removing supervisor entry and disabling the daemon
### S6 proposal engine
Scout is push-side discovery. Hermes/S6 is pull-side composition. Scout hands candidate listing evidence to Hermes but does not produce full alternatives such as install+adapt vs operator-assisted vs full custom.
S6 may consume `m2.hermes.market_proposal.v1` and `market:evidence`; it must preserve Scout's privacy boundary.
### Existing live system touchpoints
- Desktop canary: `chris-m2o` is running on `coolify`, has `/usr/local/bin/herdr`, `~/.herdr/runs/*.md`, `~/.config/herdr/`, and supervised `hermes-gateway`.
- Agent-latest canaries exist on `coolify`, including `gunnar-m2o-...`, `nasr-m2o-...`, `parlobyg-m2o-...`, and `peter-desktop-...`.
- memory-api public health is live at `https://memory.machinemachine.ai/health`; internal desktop alias is `http://memory-api:8000` after fedlearn resolver hardening.
- m2-gpt public health is live at `https://gpt.machinemachine.ai/health`; Hermes config points at `https://gpt.machinemachine.ai/v1` with `M2_GPT_API_KEY` injected at runtime.
- New services/components must be canary-first, idempotent, reversible, and use Coolify/runtime envs for secrets.
## 5. Options compared
### Option A — Standalone supervised watcher
What it is: a small daemon/process on each desktop, managed like `hermes-gateway`, reading safe local summary files/events and calling catalog APIs directly.
Pros:
- Harness-agnostic: works for Hermes, herdr, OpenClaw, and future OpenAI-wire clients.
- Least coupling to harness release cycles and plugin APIs.
- Clear privacy boundary: only the watcher owns taps, summaries, policy, and telemetry.
- Can be deployed by runtime-sync to mixed primus/agent-latest images.
- Failure is isolated; if Scout breaks, Hermes and herdr keep working.
Cons:
- Needs its own supervisor config, local state, and policy file.
- Less direct access to Hermes intent than a Hermes plugin.
- Needs a small bridge to surface in Hermes chat.
### Option B — Hermes plugin
What it is: implement Scout inside Hermes or as a Hermes skill/plugin.
Pros:
- Sees user intent directly in the primary conversational surface.
- Natural chat UX for explaining build-vs-install options.
- Reuses m2-gpt identity and model path.
Cons:
- Hermes-specific; OpenClaw and non-Hermes OpenAI-wire clients lose push discovery.
- Higher surveillance risk because it is closer to raw chat/tool context.
- Plugin failures can degrade the primary agent experience.
- herdr run lifecycle signals still need a separate bridge.
### Option C — herdr plugin
What it is: implement Scout as a herdr extension using herdr lifecycle events and run summaries.
Pros:
- Strongest access to structured lifecycle/run context.
- Aligns with live `~/.herdr/runs/*.md` capture source and toast delivery.
- Lowest risk of raw chat access if limited to run summaries.
Cons:
- Misses Hermes-only or OpenClaw-only sessions that do not run through herdr.
- Couples marketplace discovery to herdr's extension surface and lifecycle.
- Still needs Hermes and Store bridges.
### Recommendation
Choose **Option A: standalone supervised watcher** for MVP.
Reason: S4's hard requirements are privacy, tenant isolation, mixed-fleet installability, and harness-agnostic discovery. A standalone watcher best satisfies those while still consuming herdr summaries and surfacing through Hermes. Hermes and herdr should be integrations, not the host boundary.
## 6. Risks/edge cases
- Surveillance creep: prohibit raw keystrokes/client data in config and startup validation; default disabled; cap source and summary sizes; make summary/outbound payload inspectable with `once --json`.
- Tenant leakage: require tenant_id and use tenant-filtered catalog queries. Empty tenant filters fail closed. Telemetry stores intent hashes and short synthetic content only.
- Bad recommendations/spam: enforce daily/session limits, cooldowns, min score, min coverage, repeated-dismiss suppression, and proposal expiry.
- Junk listing amplification: proposal acceptance/dismiss telemetry should affect listing evidence but not bypass S2 curation. Published listing status remains registry-controlled.
- Memory-api split-brain/outage: Scout treats catalog search as best-effort. No proposal is better than unscoped or stale proposals. Telemetry spools locally.
- Secrets: Scout reads credentials from existing `~/.m2-market/config.toml`; never commits or logs keys. Log redaction is mandatory.
- Host fragility: deploy canary-first, supervisor autorestart with backoff, health/status command, rollback by disabling supervisor and leaving state intact.
- Client tenant desktops: client identifiers can appear in herdr summaries. Summarizer must redact known tenant/client names where policy says shared telemetry; for tenant-private telemetry, still avoid raw text.
- Duplicate proposals: dedup by `{listing_id, intent_hash, session_id}` plus cooldown.
- False accept telemetry: Store/CLI install result events must include `proposal_id` to count as accepted. Merely opening Store is not accepted.
## 7. [NEEDS CLARIFICATION]
1. Exact M2 Store deep-link registration for Linux desktop: confirm whether `m2store://` is final or whether S3 wants an `m2-store --open-listing` command-only contract.
2. Hermes chat bridge location: confirm whether Hermes should consume a local JSONL file, Unix socket, or m2-gpt subconscious event for `m2.hermes.market_proposal.v1`.
3. Catalog tenant wildcard representation: confirm whether public listings use `tenant_visibility: ["*"]` in registry payload and whether memory-api tenant filter should include `"*"` or a normalized `tenant:m2-shared` namespace.