m2-market/specs/S4-solution-scout.md

S4 — Solution Scout Spec

1. Scope & non-goals

MVP scope

Solution Scout v0 is a per-desktop, propose-only service that notices when an operator is already working on a task that a published M2 Marketplace listing may cover, then surfaces a non-blocking proposal.

Recommended host: standalone supervised watcher inside each desktop container, deployed canary-first on chris-m2o, then one agent-latest canary such as gunnar-m2o. It reads only already-structured local summaries, writes only proposal telemetry, and calls the existing catalog/store/install contracts.

MVP responsibilities:

• Watch safe input taps:
  • herdr run summaries from ~/.herdr/runs/*.md and herdr session metadata from ~/.config/herdr/session.json when present.
  • Hermes/OpenClaw session summaries only if the harness writes an explicit summary file or emits a summary event. No raw chat logs unless another subsystem has already summarized and scrubbed them.
  • Optional window title summaries only when explicitly enabled in policy; default off.
• Summarize on-box before any external call. The Scout sends a short intent summary, not raw workspace/client text.
• Query the semantic catalog through memory-api using partition/agent namespace market:catalog, tenant-filtered by the caller tenant.
• Decide whether to propose using confidence, coverage, recency, rate-limit, dismissal cooldown, and policy thresholds.
• Surface proposals as:
  • herdr/XFCE toast with a deep link into M2 Store.
  • Hermes chat surfacing when Hermes is active and exposes the market-propose bridge.
• Persist local proposal state and write accept/dismiss/show telemetry to market:evidence.
• Respect opt-in config in ~/.config/m2-market/pull-policy.toml.

Later scope

• Full Hermes-native agent reasoning for build-vs-install proposals belongs to S6. S4 only hands Hermes a candidate proposal event.
• Automatic install is out of scope. Human clicks through M2 Store and m2-market install handles debit/grant/apply.
• Raw keystroke capture, screen recording OCR, shell history capture, browser DOM scraping, and unreviewed client data extraction are explicitly out of scope.
• Fleet-wide rollout, packaging format, catalog truth, storefront implementation, and ledger internals are owned by S5/S2/S3/S1 respectively.

2. User stories + acceptance criteria

Story 1 — Opt-in canary proposal

Given chris-m2o has ~/.config/m2-market/pull-policy.toml with scout.enabled = true and a herdr summary describes a competitor research report, when Scout runs its poll loop, then it creates an on-box intent summary, queries market:catalog within the machine tenant, and shows at most one proposal for competitor-scan if score and coverage exceed thresholds.

Acceptance:

• The payload sent to memory-api contains no raw run body longer than summary.max_chars.
• The proposal contains listing name, price, confidence/coverage, install count/rating if present, and Store deep link.
• ~/.local/share/m2-market/scout/state.json records the proposal id and cooldown.

Story 2 — Human-controlled install path

Given a proposal toast is shown, when the operator clicks Open, then M2 Store opens m2store://listing/<listing_id>?proposal_id=<proposal_id>&source=scout.

Acceptance:

• Scout never calls m2-market install directly.
• Store/CLI remain responsible for ledger /tx/install, license grant, bundle verification, apply adapter, and refund on failure.
• Scout observes accept/installed telemetry only after Store/CLI emits an explicit event.

Story 3 — Dismissal tunes future proposals

Given an operator dismisses a proposal, when the same listing/task pair appears again within the configured cooldown, then Scout suppresses the proposal and writes a dismiss event to evidence.

Acceptance:

• Dismissal is keyed by {tenant_id, operator_id, desktop_id, listing_id, intent_hash}.
• Dismiss telemetry is stored with a summary hash and reason, not client content.
• Listing evidence can fold proposals_shown and proposals_accepted into listing.stats.

Story 4 — Tenant firewall

Given the desktop belongs to sdjs/GST or another client tenant, when Scout queries the catalog, then only listings with tenant_visibility containing that tenant or "*" are eligible.

Acceptance:

• Empty tenant filter is a hard error, not an unscoped search.
• Tenant-private summaries never become shared catalog data.
• Owner-initiated commercialization and double scrub remain packaging/catalog obligations, not Scout behavior.

3. Interfaces & data contracts

Process and deployment

Process name: m2-solution-scout

MVP placement:

• Desktop binary: /usr/local/bin/m2-solution-scout
• Supervisor config: /etc/supervisor/conf.d/m2-solution-scout.conf
• User config: ~/.config/m2-market/pull-policy.toml
• Local state: ~/.local/share/m2-market/scout/state.json
• Local event outbox: ~/.local/share/m2-market/scout/outbox/YYYY-MM-DD.jsonl
• Logs: ~/.local/share/m2-market/scout/scout.log

Deploy via runtime-sync / apply adapter, not image bake, until canary acceptance proves it. The service must tolerate missing herdr/Hermes/OpenClaw sources and run as degraded rather than crash-looping.

pull-policy.toml

[scout]
enabled = false
mode = "suggest"              # enum: off|suggest. No take-control mode in S4.
desktop_id = "chris-m2o"
operator_id = "op_chris"
tenant_id = "machine-machine"
min_score = 0.78
min_coverage = 0.55
poll_interval_seconds = 90
max_proposals_per_day = 4
max_proposals_per_session = 1
dismiss_cooldown_days = 14
proposal_cooldown_minutes = 45

[scout.sources]
herdr_runs = true
session_summaries = true
window_titles = false
raw_keystrokes = false
raw_client_data = false

[scout.summary]
max_source_chars = 12000
max_summary_chars = 900
redact_secrets = true
redact_client_identifiers = true
fail_closed_on_secret = true

[scout.catalog]
memory_api_url = "https://memory.machinemachine.ai"
agent_id = "market:catalog"
limit = 5
routing_strategy = "standard"

[scout.store]
deeplink_scheme = "m2store"
fallback_command = "m2-market show --json"

[scout.telemetry]
enabled = true
agent_id = "market:evidence"
batch_seconds = 60

Policy rules:

• enabled=false disables all watching and all outbound calls.
• tenant_id is required. If missing, Scout exits non-zero and writes no events.
• raw_keystrokes and raw_client_data are reserved deny keys. If true, Scout refuses to start.
• All keys are runtime-injected/configured. No secrets in this file except references to existing machine config; memory/ledger keys remain in ~/.m2-market/config.toml with mode 0600.

Input event schema

Scout normalizes safe local taps into ScoutObservation.

{
  "schema_version": "m2.scout.observation.v1",
  "observation_id": "obs_20260702_chris_01j1",
  "desktop_id": "chris-m2o",
  "operator_id": "op_chris",
  "tenant_id": "machine-machine",
  "source": "herdr_run",
  "source_uri": "file:///home/developer/.herdr/runs/2026-07-02-fedlearn-seed.md",
  "source_ts": "2026-07-02T00:15:00Z",
  "session_id": "herdr:2026-07-02-fedlearn-seed",
  "safe_excerpt": "Short scrubbed excerpt or heading only",
  "source_hash": "sha256:...",
  "metadata": {
    "image_class": "primus",
    "harness": "herdr"
  }
}

Constraints:

• safe_excerpt is optional and capped by policy; raw source body is never persisted in telemetry.
• source_hash is computed from local source text for dedup, but only the hash leaves the box.
• For herdr MVP, source files are the live path verified on chris-m2o: ~/.herdr/runs/*.md; config/session files are under ~/.config/herdr/.

On-box intent summary schema

{
  "schema_version": "m2.scout.intent_summary.v1",
  "summary_id": "sum_20260702_chris_01j1",
  "observation_ids": ["obs_20260702_chris_01j1"],
  "desktop_id": "chris-m2o",
  "operator_id": "op_chris",
  "tenant_id": "machine-machine",
  "session_id": "herdr:2026-07-02-fedlearn-seed",
  "summary": "Operator is preparing a competitor research report with source gathering and final PDF/report output.",
  "intent_terms": ["competitor research", "report", "sources", "PDF"],
  "negative_terms": ["private payroll", "credentials"],
  "scrub_status": {
    "secrets_redacted": true,
    "client_identifiers_redacted": true,
    "failed_closed": false
  },
  "intent_hash": "sha256:...",
  "created_at": "2026-07-02T00:35:00Z"
}

Summarization can be deterministic extractive in MVP. If an LLM is used later, it must call m2-gpt with the desktop tenant key and still send only the capped summary onward.

Catalog query

Use existing memory-api /memory/search surface rather than a new Scout API.

Request:

POST https://memory.machinemachine.ai/memory/search
X-API-Key: <memory api key from ~/.m2-market/config.toml>
Content-Type: application/json

{
  "query": "Operator is preparing a competitor research report with source gathering and final PDF/report output.",
  "agent_id": "market:catalog",
  "routing_strategy": "standard",
  "limit": 5,
  "tenant_id": ["machine-machine", "*"],
  "filters": {
    "tenant_id": ["machine-machine", "*"]
  }
}

Response is the live RoutedSearchResponse shape from memory-api: {results, routing}. Each result must carry listing payload metadata matching schemas/listing.schema.json. The matcher discards any hit whose listing payload is missing, delisted, wrong tenant, or below threshold.

Match decision schema

{
  "schema_version": "m2.scout.match.v1",
  "match_id": "mat_20260702_chris_01j1",
  "summary_id": "sum_20260702_chris_01j1",
  "listing_id": "lst_competitor-scan",
  "solution_id": "sol_competitor-scan",
  "tenant_id": "machine-machine",
  "score": 0.84,
  "coverage": 0.72,
  "reasons": [
    "intent overlaps listing intent",
    "listing supports report generation",
    "price/evidence present"
  ],
  "suppressed": false,
  "suppression_reason": null,
  "created_at": "2026-07-02T00:35:03Z"
}

Coverage is Scout's local estimate of how much of the current intent is covered by the listing. It is not authoritative pricing. S6 owns full build-vs-install composition.

Proposal schema

{
  "schema_version": "m2.scout.proposal.v1",
  "proposal_id": "prp_20260702_chris_01j1",
  "match_id": "mat_20260702_chris_01j1",
  "desktop_id": "chris-m2o",
  "operator_id": "op_chris",
  "tenant_id": "machine-machine",
  "listing": {
    "listing_id": "lst_competitor-scan",
    "name": "Competitor Scan",
    "summary": "Research report package for competitor positioning.",
    "price": {"amount": 40, "currency": "m2cr", "model": "fixed"},
    "stats": {"installs": 12, "rating": 4.6, "proposals_shown": 20, "proposals_accepted": 7}
  },
  "confidence": 0.84,
  "coverage": 0.72,
  "deeplink": "m2store://listing/lst_competitor-scan?proposal_id=prp_20260702_chris_01j1&source=scout",
  "shown_at": "2026-07-02T00:35:04Z",
  "expires_at": "2026-07-02T01:20:04Z"
}

Toast copy should be non-blocking and specific:

This looks like Competitor Scan: ~72% coverage, 40 m2cr. Open in M2 Store?

Proposal UX interfaces

MVP toast command:

notify-send "M2 Solution Scout" "This looks like Competitor Scan: ~72% coverage, 40 m2cr. Open in M2 Store?" --action=open="Open" --action=dismiss="Dismiss"

If notify-send actions are unreliable in XFCE/VNC, fallback to herdr toast delivery. The existing herdr config supports [ui.toast] delivery = "herdr", verified in /home/m2/m2o/desktop/herdr/config.toml.

M2 Store deep link:

m2store://listing/{listing_id}?proposal_id={proposal_id}&source=scout

Store fallback command:

m2-market show {listing_id} --json

Hermes surfacing event, written to a local bridge file/socket for the market-propose skill:

{
  "schema_version": "m2.hermes.market_proposal.v1",
  "event_type": "market.proposal.available",
  "proposal_id": "prp_20260702_chris_01j1",
  "session_id": "herdr:2026-07-02-fedlearn-seed",
  "message": "I found an existing Solution that may cover this task.",
  "deeplink": "m2store://listing/lst_competitor-scan?proposal_id=prp_20260702_chris_01j1&source=scout"
}

The Hermes bridge is best-effort in MVP. If unavailable, toast + Store link is sufficient.

Telemetry event schema

Telemetry is append-only, locally spooled, then written to memory-api market:evidence via /memory/store.

{
  "schema_version": "m2.scout.telemetry.v1",
  "event_id": "evt_20260702_chris_01j1",
  "event_type": "proposal_shown",
  "proposal_id": "prp_20260702_chris_01j1",
  "listing_id": "lst_competitor-scan",
  "solution_id": "sol_competitor-scan",
  "operator_id": "op_chris",
  "desktop_id": "chris-m2o",
  "tenant_id": "machine-machine",
  "session_id": "herdr:2026-07-02-fedlearn-seed",
  "intent_hash": "sha256:...",
  "score": 0.84,
  "coverage": 0.72,
  "reason": null,
  "created_at": "2026-07-02T00:35:04Z"
}

Allowed event_type:

• proposal_shown
• proposal_opened
• proposal_dismissed
• proposal_accepted
• install_started
• install_succeeded
• install_failed
• proposal_suppressed

Memory write:

POST https://memory.machinemachine.ai/memory/store
X-API-Key: <memory api key>
Content-Type: application/json

{
  "content": "proposal_shown lst_competitor-scan score=0.84 coverage=0.72",
  "agent_id": "market:evidence",
  "memory_type": "semantic",
  "importance": 0.4,
  "source": "system",
  "tenant_id": "machine-machine",
  "session_id": "herdr:2026-07-02-fedlearn-seed",
  "metadata": {
    "schema_version": "m2.scout.telemetry.v1",
    "event_id": "evt_20260702_chris_01j1",
    "event_type": "proposal_shown",
    "listing_id": "lst_competitor-scan",
    "proposal_id": "prp_20260702_chris_01j1",
    "intent_hash": "sha256:..."
  }
}

Telemetry must be batchable and retryable. If memory-api is down, Scout keeps local JSONL outbox and continues proposing within local rate limits.

CLI verbs

Scout management CLI:

m2-solution-scout run
m2-solution-scout once --json
m2-solution-scout status --json
m2-solution-scout dismiss <proposal_id> [--reason not-relevant]
m2-solution-scout open <proposal_id>
m2-solution-scout flush-telemetry

once --json is the canary acceptance hook: read sources, summarize, query, and emit a proposal or suppression decision without daemonizing.

4. Integration contract with other subsystems

S1 ledger

Scout does not debit credits. It includes listing price in the proposal and links to Store. Install remains:

M2 Store -> m2-market install -> POST /tx/install -> license grant -> apply -> telemetry

Scout listens for Store/CLI install result events and writes evidence only. Refund behavior remains S1/S3/CLI.

S2 catalog

Scout reads market:catalog only through memory-api /memory/search using agent_id = "market:catalog". Listing payload must validate against /home/m2/m2-market/schemas/listing.schema.json.

Scout writes proposal evidence to market:evidence. S2/indexer folds proposal counts into listing.stats.proposals_shown and listing.stats.proposals_accepted.

S3 store

Scout proposal opens:

m2store://listing/{listing_id}?proposal_id={proposal_id}&source=scout

Store shows evidence, price, permission diff, and calls m2-market install. Store should emit local result events:

• ~/.local/share/m2-market/events/install-events.jsonl, or
• a local socket declared by S3.

Until S3 finalizes this, Scout can infer only proposal_opened and proposal_dismissed.

S5 packaging

Scout is installed as a Solution/fleet component via the same runtime-sync path and canary policy. It must not require rebuilding primus or agent-latest images. Package should include:

• binary/script
• supervisor conf
• default disabled pull-policy.toml
• rollback removing supervisor entry and disabling the daemon

S6 proposal engine

Scout is push-side discovery. Hermes/S6 is pull-side composition. Scout hands candidate listing evidence to Hermes but does not produce full alternatives such as install+adapt vs operator-assisted vs full custom.

S6 may consume m2.hermes.market_proposal.v1 and market:evidence; it must preserve Scout's privacy boundary.

Existing live system touchpoints

• Desktop canary: chris-m2o is running on coolify, has /usr/local/bin/herdr, ~/.herdr/runs/*.md, ~/.config/herdr/, and supervised hermes-gateway.
• Agent-latest canaries exist on coolify, including gunnar-m2o-..., nasr-m2o-..., parlobyg-m2o-..., and peter-desktop-....
• memory-api public health is live at https://memory.machinemachine.ai/health; internal desktop alias is http://memory-api:8000 after fedlearn resolver hardening.
• m2-gpt public health is live at https://gpt.machinemachine.ai/health; Hermes config points at https://gpt.machinemachine.ai/v1 with M2_GPT_API_KEY injected at runtime.
• New services/components must be canary-first, idempotent, reversible, and use Coolify/runtime envs for secrets.

5. Options compared

Option A — Standalone supervised watcher

What it is: a small daemon/process on each desktop, managed like hermes-gateway, reading safe local summary files/events and calling catalog APIs directly.

Pros:

• Harness-agnostic: works for Hermes, herdr, OpenClaw, and future OpenAI-wire clients.
• Least coupling to harness release cycles and plugin APIs.
• Clear privacy boundary: only the watcher owns taps, summaries, policy, and telemetry.
• Can be deployed by runtime-sync to mixed primus/agent-latest images.
• Failure is isolated; if Scout breaks, Hermes and herdr keep working.

Cons:

• Needs its own supervisor config, local state, and policy file.
• Less direct access to Hermes intent than a Hermes plugin.
• Needs a small bridge to surface in Hermes chat.

Option B — Hermes plugin

What it is: implement Scout inside Hermes or as a Hermes skill/plugin.

Pros:

• Sees user intent directly in the primary conversational surface.
• Natural chat UX for explaining build-vs-install options.
• Reuses m2-gpt identity and model path.

Cons:

• Hermes-specific; OpenClaw and non-Hermes OpenAI-wire clients lose push discovery.
• Higher surveillance risk because it is closer to raw chat/tool context.
• Plugin failures can degrade the primary agent experience.
• herdr run lifecycle signals still need a separate bridge.

Option C — herdr plugin

What it is: implement Scout as a herdr extension using herdr lifecycle events and run summaries.

Pros:

• Strongest access to structured lifecycle/run context.
• Aligns with live ~/.herdr/runs/*.md capture source and toast delivery.
• Lowest risk of raw chat access if limited to run summaries.

Cons:

• Misses Hermes-only or OpenClaw-only sessions that do not run through herdr.
• Couples marketplace discovery to herdr's extension surface and lifecycle.
• Still needs Hermes and Store bridges.

Recommendation

Choose Option A: standalone supervised watcher for MVP.

Reason: S4's hard requirements are privacy, tenant isolation, mixed-fleet installability, and harness-agnostic discovery. A standalone watcher best satisfies those while still consuming herdr summaries and surfacing through Hermes. Hermes and herdr should be integrations, not the host boundary.

6. Risks/edge cases

• Surveillance creep: prohibit raw keystrokes/client data in config and startup validation; default disabled; cap source and summary sizes; make summary/outbound payload inspectable with once --json.
• Tenant leakage: require tenant_id and use tenant-filtered catalog queries. Empty tenant filters fail closed. Telemetry stores intent hashes and short synthetic content only.
• Bad recommendations/spam: enforce daily/session limits, cooldowns, min score, min coverage, repeated-dismiss suppression, and proposal expiry.
• Junk listing amplification: proposal acceptance/dismiss telemetry should affect listing evidence but not bypass S2 curation. Published listing status remains registry-controlled.
• Memory-api split-brain/outage: Scout treats catalog search as best-effort. No proposal is better than unscoped or stale proposals. Telemetry spools locally.
• Secrets: Scout reads credentials from existing ~/.m2-market/config.toml; never commits or logs keys. Log redaction is mandatory.
• Host fragility: deploy canary-first, supervisor autorestart with backoff, health/status command, rollback by disabling supervisor and leaving state intact.
• Client tenant desktops: client identifiers can appear in herdr summaries. Summarizer must redact known tenant/client names where policy says shared telemetry; for tenant-private telemetry, still avoid raw text.
• Duplicate proposals: dedup by {listing_id, intent_hash, session_id} plus cooldown.
• False accept telemetry: Store/CLI install result events must include proposal_id to count as accepted. Merely opening Store is not accepted.

7. [NEEDS CLARIFICATION]

1. Exact M2 Store deep-link registration for Linux desktop: confirm whether m2store:// is final or whether S3 wants an m2-store --open-listing command-only contract.
2. Hermes chat bridge location: confirm whether Hermes should consume a local JSONL file, Unix socket, or m2-gpt subconscious event for m2.hermes.market_proposal.v1.
3. Catalog tenant wildcard representation: confirm whether public listings use tenant_visibility: ["*"] in registry payload and whether memory-api tenant filter should include "*" or a normalized tenant:m2-shared namespace.