factory-loop(plan): 001-market-first-wedge — plan, research, data model, 5 contracts, quickstart
Rail-independence is the plan's spine: fedlearn rails verified NOT landed (no m2/m2-core, no capture/curator CLIs), so the wedge builds against frozen interfaces — m2/market-registry as registry of record, ApplyAdapter seam with local adapter now / m2core-sync stub, schemas frozen here with a fedlearn coordination note. Constitution check passed (1 justified deviation). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
e8ee3920de
commit
8757c6b620
9 changed files with 719 additions and 0 deletions
43
specs/001-market-first-wedge/contracts/apply-adapter.md
Normal file
43
specs/001-market-first-wedge/contracts/apply-adapter.md
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
# Contract: Apply Adapter (v1, frozen) — the fedlearn integration seam
|
||||
|
||||
The ONLY interface through which anything installs a Solution bundle onto a machine.
|
||||
Freezing this seam is what lets the wedge ship before the fedlearn rails land (research.md
|
||||
R5) and swap them in later without touching CLI or Store.
|
||||
|
||||
## Protocol (Python, `cli/src/m2_market/apply/base.py`)
|
||||
|
||||
```python
|
||||
class ApplyAdapter(Protocol):
|
||||
name: str # "local" | "m2core-sync"
|
||||
def plan(self, bundle: Path, state: InstallState) -> Changeset: ...
|
||||
def apply(self, changeset: Changeset) -> ApplyResult: ... # idempotent
|
||||
def verify(self, bundle: Path, state: InstallState) -> bool: ...
|
||||
def rollback(self, changeset: Changeset) -> None: ... # reversible (constitution VI)
|
||||
```
|
||||
|
||||
- `Changeset`: declarative list of file placements / config merges / post-install checks
|
||||
derived from the bundle's `recipe.yaml`, plus a `changeset_hash` recorded in
|
||||
InstallState.
|
||||
- `apply` MUST be idempotent (re-apply of an applied changeset = no-op success) and MUST
|
||||
NOT require elevated fleet-wide operations (canary principle).
|
||||
- `rollback` restores pre-apply state from the changeset's recorded backups.
|
||||
|
||||
## v1 implementations
|
||||
|
||||
### `local` (ships in this wedge)
|
||||
Applies `payload/` into the operator's agent-home per `recipe.yaml` targets
|
||||
(skills, prompts, configs, scripts), runs recipe post-install checks, records
|
||||
InstallState. Works identically inside primus and agent-latest desktops (targets are
|
||||
volume paths, not image paths) — satisfying "runtime-sync is the only universal path" in
|
||||
spirit until the real one exists.
|
||||
|
||||
### `m2core-sync` (stub in this wedge)
|
||||
Raises `RailsNotLanded` (CLI exit 6) with pointer to this contract. When fedlearn's
|
||||
`m2-core-sync`/`m2-core pull --apply` lands, this adapter shells to it and merges its
|
||||
state.json with InstallState. Acceptance for the swap (row 10 of plan sequencing): all
|
||||
quickstart.md scenarios pass with `adapter = m2core-sync` unchanged.
|
||||
|
||||
## Selection
|
||||
|
||||
`config.toml: apply_adapter = "local"` (default in v1). Per-install override
|
||||
`--adapter` for testing. The Store never selects an adapter — it inherits the CLI's.
|
||||
41
specs/001-market-first-wedge/contracts/catalog-index.md
Normal file
41
specs/001-market-first-wedge/contracts/catalog-index.md
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
# Contract: market:catalog partition (memory-api) (v1)
|
||||
|
||||
The semantic index over published listings. Derived data only — rebuildable from
|
||||
`m2/market-registry` at any time.
|
||||
|
||||
## Partition
|
||||
|
||||
- Namespace: `market:catalog` (memory-api partition, same mechanism as
|
||||
`fedlearn:core-index` design).
|
||||
- Document = one published listing: embedded text = `name + summary + keywords + intent +
|
||||
evidence_summary` (BGE-M3 hybrid dense+sparse); payload = full `listing.json` +
|
||||
`tenant_visibility`.
|
||||
- Endpoint: the stack behind `memory.machinemachine.ai` (split-brain caveat: research.md R4
|
||||
— pinned until fedlearn T13 resolves; recovery = reindex).
|
||||
|
||||
## Writers
|
||||
|
||||
ONLY `m2-market-indexer`:
|
||||
- `watch` — on registry merge (webhook or poll ≤5 min), upsert changed listings, remove
|
||||
delisted.
|
||||
- `reindex` — drop partition, rebuild from registry `main` (constitution II gate; must
|
||||
complete < 5 min at wedge scale).
|
||||
|
||||
## Readers
|
||||
|
||||
- CLI `m2-market search` → memory-api semantic query, **server-side filter**
|
||||
`tenant_visibility ⊇ {caller tenant | "*"}` (FR-007; Scout edge case "impossible by
|
||||
construction").
|
||||
- M2 Store front page/search → same query path (via CLI or direct HTTP with desktop creds).
|
||||
- Scout matcher → same query path, additionally thresholded by confidence.
|
||||
|
||||
## Query contract (via memory-api HTTP)
|
||||
|
||||
Request: `{query_text, partition: "market:catalog", tenant, limit}` →
|
||||
Response items: `{listing_id, score, listing: <payload>}` ordered by hybrid score.
|
||||
|
||||
## Telemetry write-back (FR-012, FR-014)
|
||||
|
||||
Install + proposal events land as memory records in `market:evidence` partition keyed by
|
||||
listing_id; a periodic indexer pass folds counts into `stats` in the registry (PR-less
|
||||
stats commit or scheduled batch PR — implementation's choice, registry remains truth).
|
||||
44
specs/001-market-first-wedge/contracts/cli.md
Normal file
44
specs/001-market-first-wedge/contracts/cli.md
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
# Contract: m2-market CLI (v1)
|
||||
|
||||
Python (uv) package `m2-market`; installed on desktops + host. Config
|
||||
`~/.m2-market/config.toml`: `operator_id`, `ledger_url` + key, `memory_api_url` + key,
|
||||
`forgejo_url` + token, `tenant`. State: `~/.m2-market/state.json` (data-model.md
|
||||
InstallState). All commands support `--json` for machine consumption (the Store shells
|
||||
these).
|
||||
|
||||
## Commands
|
||||
|
||||
### `m2-market search <query> [--type solution] [--limit N]`
|
||||
Semantic catalog query (contracts/catalog-index.md), tenant-filtered.
|
||||
Output: table (or JSON): listing_id, name, summary, price, installs, rating.
|
||||
|
||||
### `m2-market show <listing_id>`
|
||||
Full listing detail: evidence summary, price, permissions diff (from solution.permissions
|
||||
vs current state), seller, version, install_ref.
|
||||
|
||||
### `m2-market install <listing_id> [--yes]`
|
||||
The FR-005 sequence, in order:
|
||||
1. Resolve listing from catalog; confirm price + permissions diff (interactive unless
|
||||
`--yes`).
|
||||
2. `POST /tx/install` to ledger (ref = `listing_id:machine:uuid`) → grant. On 402: exit 3
|
||||
"insufficient funds", nothing applied (Story 1 AC-2).
|
||||
3. Fetch bundle release from registry; verify `content_hash`.
|
||||
4. `AdapterProtocol.plan → apply → verify` (contracts/apply-adapter.md). On apply/verify
|
||||
failure: `POST /tx/refund {ref}`, exit 4 with remediation text (Story 4 AC-3 semantics).
|
||||
5. Record InstallState; print entrypoint invocation hint (Story 1 AC-4).
|
||||
Re-run with existing grant + applied state → idempotent no-op, exit 0 (Story 1 AC-3).
|
||||
|
||||
### `m2-market wallet [--operator X]`
|
||||
Balance + recent tx from ledger.
|
||||
|
||||
### `m2-market publish <bundle-dir>` (builder-side, v1 minimal)
|
||||
Validate against schemas → push branch + open listing PR on the registry (Story 2 step 1–2).
|
||||
Does NOT merge; humans/veto do.
|
||||
|
||||
### `m2-market reindex` (admin)
|
||||
Invokes indexer full rebuild (constitution II operational check).
|
||||
|
||||
## Exit codes
|
||||
|
||||
0 ok/no-op · 1 usage · 2 not found · 3 insufficient funds · 4 apply failed (refunded) ·
|
||||
5 validation/firewall rejection · 6 rails-not-landed (m2core_sync stub invoked)
|
||||
47
specs/001-market-first-wedge/contracts/ledger-api.md
Normal file
47
specs/001-market-first-wedge/contracts/ledger-api.md
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
# Contract: m2-ledger HTTP API (v1, frozen)
|
||||
|
||||
Base: `http://m2-ledger:8000` (coolify network alias) / public via Traefik later if needed.
|
||||
Auth: `X-API-Key` header. Two key classes: `service` (CLI/Store/indexer) and `admin`
|
||||
(grants, snapshots). All bodies JSON. Amounts are integer credits.
|
||||
|
||||
## Endpoints
|
||||
|
||||
### `GET /health`
|
||||
→ `200 {"status":"healthy"}`
|
||||
|
||||
### `GET /balance/{operator_id}` (service)
|
||||
→ `200 {"operator_id","balance","as_of"}` — balance derived from tx log.
|
||||
|
||||
### `POST /tx/install` (service) — the FR-005 money move
|
||||
Body: `{"buyer","seller","amount","platform_pct"?,"ref"}` (`ref` = listing_id + install id)
|
||||
Semantics: single SQLite transaction — verify `balance(buyer) ≥ amount`, insert
|
||||
buyer→seller net row + buyer→platform cut row, insert license grant, return grant.
|
||||
→ `200 {"tx_ids":[..],"grant":{"grant_id","listing_id","operator_id",...}}`
|
||||
→ `402 {"error":"insufficient_funds","balance"}` — nothing written.
|
||||
→ `409 {"error":"duplicate_ref"}` — same `ref` already settled (idempotency guard).
|
||||
|
||||
### `POST /tx/refund` (service)
|
||||
Body: `{"ref"}` → compensating refund rows for that install ref (edge case + FR-005).
|
||||
→ `200 {"tx_ids":[..]}`; `404` unknown ref; `409` already refunded.
|
||||
|
||||
### `POST /grant/starter` (admin)
|
||||
Body: `{"operator_id","amount"?}` (default from config, 100 cr UNCONFIRMED) → mint→operator
|
||||
`grant` row. → `200 {"tx_id","balance"}`
|
||||
|
||||
### `GET /tx?operator_id=&reason=&since=` (service)
|
||||
→ `200 {"transactions":[...]}` append-only log view, paginated.
|
||||
|
||||
### `POST /snapshot` (admin, also daily cron)
|
||||
Dumps all balances → commits `audit/YYYY-MM-DD.json` to `m2/market-registry` (Forgejo API,
|
||||
token from env). → `200 {"path","commit"}`
|
||||
|
||||
### `GET /license/{operator_id}/{listing_id}` (service)
|
||||
→ `200 {"grant":...}` or `404` — the apply path's pre-check.
|
||||
|
||||
## Invariants (testable)
|
||||
|
||||
1. No endpoint UPDATEs/DELETEs tx rows.
|
||||
2. `balance == fold(tx log)` for every operator after any call sequence (SC-004).
|
||||
3. Every grant has a non-null paying tx (SC-003).
|
||||
4. `/tx/install` is all-or-nothing and idempotent by `ref`.
|
||||
5. Config (not code): `platform_pct` default 0.10, `starter_grant` default 100.
|
||||
45
specs/001-market-first-wedge/contracts/registry-layout.md
Normal file
45
specs/001-market-first-wedge/contracts/registry-layout.md
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
# Contract: m2/market-registry repo layout + PR/veto protocol (v1, frozen)
|
||||
|
||||
Registry of record on Forgejo (`git.machinemachine.ai/m2/market-registry`). Truth for
|
||||
listings and audit; everything downstream (catalog, store, CLI) is derived and rebuildable.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
listings/<listing_id>/
|
||||
├── listing.json # conforms to schemas/listing.schema.json
|
||||
├── solution.json # conforms to schemas/solution.schema.json (current version)
|
||||
└── evidence/ # provenance files referenced by solution.evidence[]
|
||||
audit/
|
||||
└── YYYY-MM-DD.json # daily ledger balance snapshots (ledger-api /snapshot)
|
||||
schemas/ -> mirrored copy of frozen v1 schemas (CI validates listings against them)
|
||||
```
|
||||
|
||||
Solution bundles (tarballs: `solution.json` + `payload/` + `recipe.yaml`) attach as
|
||||
**repo releases** tagged `<listing_id>-v<semver>`; `listing.json.install_ref` names the tag.
|
||||
|
||||
## Publish protocol (Story 2)
|
||||
|
||||
1. Builder branch `listing/<listing_id>` adds/updates `listings/<listing_id>/`.
|
||||
2. PR opened with template: evidence summary, price, permissions diff, rollback note.
|
||||
Status: `in_review`.
|
||||
3. CI validation on PR (schema-validate, evidence non-empty, tenant-firewall rules,
|
||||
content_hash matches release asset) — hard-fails before human review (FR-015, spec
|
||||
Story 2 AC-2).
|
||||
4. Human review window; label `veto` → close (back to draft). Label `approved` + merge →
|
||||
`published`. (Same veto semantics the fedlearn pipeline uses; when fedlearn's curation
|
||||
`commercialize` disposition lands, it opens THIS PR shape — that is the integration seam.)
|
||||
5. Merge webhook (or indexer poll) triggers catalog index (contracts/catalog-index.md).
|
||||
|
||||
## Delist protocol
|
||||
|
||||
Commit setting `listing.json.status = "delisted"` (PR, lighter review). Licenses and
|
||||
installed bundles remain valid; catalog entry removed on next index; release assets kept.
|
||||
|
||||
## Rules
|
||||
|
||||
- No secrets anywhere in the repo (constitution VI); evidence is scrubbed before commit.
|
||||
- Registry wins over index on divergence; `m2-market-indexer reindex` rebuilds the
|
||||
partition from `main` (constitution II).
|
||||
- Tenant-derived listings require `owner_initiated` + double-scrub attestations in the PR
|
||||
body; CI rejects `tenant_scope` violations mechanically.
|
||||
112
specs/001-market-first-wedge/data-model.md
Normal file
112
specs/001-market-first-wedge/data-model.md
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
# Data Model — 001-market-first-wedge (Phase 1)
|
||||
|
||||
Entities from spec.md §Key Entities, concretized. Schemas frozen in `schemas/` are the
|
||||
normative source; this file is the narrative view.
|
||||
|
||||
## Solution (`schemas/solution.schema.json`, v1 frozen)
|
||||
|
||||
The installable bundle. Manifest-core fields kept field-compatible with the fedlearn
|
||||
submission/manifest design (coordination item R3).
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| `schema_version` | const `m2.solution.v1` | frozen |
|
||||
| `solution_id` | string `sol_<slug>` | unique across registry |
|
||||
| `name`, `summary`, `description` | string | presentation |
|
||||
| `intent` | string | what outcome this delivers |
|
||||
| `behavior` | object | prompts / skills / playbooks refs (paths in payload) |
|
||||
| `tools` | array | MCP/API/connector requirements |
|
||||
| `runtime` | object | required surfaces: hermes / desktop / browser / openclaw |
|
||||
| `memory_schema` | object? | partitions/keys the solution reads/writes |
|
||||
| `permissions` | array | declarative permission requests (shown as diff pre-install) |
|
||||
| `deployment` | object | `recipe.yaml` ref + entrypoint + verify command |
|
||||
| `applicability` | object | `image_classes[]`, `roles[]`, `tool_requirements[]` (fedlearn-compatible) |
|
||||
| `tenant_scope` | string | `m2-core` (shared) or tenant id; drives firewall checks |
|
||||
| `evidence` | array | provenance refs: `{source, excerpt?, machine?, session?, tokens?, wall_time?}` — ≥1 REQUIRED |
|
||||
| `content_hash` | string sha256 | bundle integrity |
|
||||
| `price` | object | `{amount: int>0, currency: "m2cr", model: "fixed"}` — `model` enum extensible (job/metered later, headroom per operator steer) |
|
||||
| `seller` | string operator_id | |
|
||||
| `license` | object | `{terms: string, major_version_coverage: true}` |
|
||||
| `revenue_split` | object | `{platform_pct: number}` (default from ledger config) |
|
||||
|
||||
**Validation**: evidence non-empty; price.amount ≥ 1; tenant_scope=tenant-id requires
|
||||
`owner_initiated: true` + `scrub_status.double_scrubbed: true`; content_hash matches bundle.
|
||||
|
||||
## Listing (`schemas/listing.schema.json`, v1 frozen)
|
||||
|
||||
Catalog-facing record; truth in `m2/market-registry`, derived copy in `market:catalog`.
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| `schema_version` | const `m2.listing.v1` | |
|
||||
| `listing_id` | string `lst_<slug>` | |
|
||||
| `solution_id` + `solution_version` | refs | what's being sold |
|
||||
| `inventory_type` | enum `solution|capability|resource|service` | only `solution` transacted in v1 |
|
||||
| `name`, `summary`, `category`, `keywords[]`, `icon?` | | cargstore-catalog compatible |
|
||||
| `price` | object | denormalized from solution at listing time |
|
||||
| `seller` | operator_id | |
|
||||
| `evidence_summary` | string | rendered from solution evidence |
|
||||
| `tenant_visibility` | array | tenant ids or `["*"]`; catalog queries filter on this |
|
||||
| `stats` | object | `{installs: int, rating: number?, proposals_shown: int, proposals_accepted: int}` — conversion signals (FR-012), updated by telemetry, start zeroed |
|
||||
| `status` | enum `draft|in_review|published|delisted` | delisted keeps licenses valid |
|
||||
| `install_ref` | string | registry release tag `<listing_id>-v<semver>` |
|
||||
|
||||
**State transitions**: `draft → in_review` (PR opened) `→ published` (PR merged, no veto
|
||||
label) `→ delisted` (delist commit; grants unaffected). Veto label on PR → back to `draft`.
|
||||
|
||||
## Transaction (m2-ledger, SQLite `tx` table — append-only)
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | integer PK autoincrement | |
|
||||
| `ts` | text ISO-8601 UTC | set by service |
|
||||
| `from_id` | text | operator_id or `platform` or `mint` |
|
||||
| `to_id` | text | operator_id or `platform` |
|
||||
| `amount` | integer > 0 | whole credits in v1 |
|
||||
| `reason` | enum `install|payout|grant|route|earn|refund` | `refund` added for FR-005 compensation |
|
||||
| `ref` | text | listing_id / grant note / install id |
|
||||
|
||||
**Invariants**: rows never UPDATEd or DELETEd; `balance(op) = Σ to_id=op − Σ from_id=op`;
|
||||
service refuses tx that would take a non-`mint` balance negative. An install produces 2–3
|
||||
rows atomically (one SQLite transaction): buyer→seller (net), buyer→platform (cut), and on
|
||||
apply-failure a compensating seller/platform→buyer `refund` pair referencing the same
|
||||
install id.
|
||||
|
||||
## LicenseGrant (m2-ledger, `grant` table)
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `grant_id` | text `gr_<uuid>` | |
|
||||
| `operator_id` | text | buyer |
|
||||
| `listing_id` | text | |
|
||||
| `solution_version_major` | int | license covers this major (edge case: updates free within major) |
|
||||
| `tx_id` | int FK → tx | the debit that paid for it; NOT NULL (no grant without payment — SC-003) |
|
||||
| `ts` | text | |
|
||||
|
||||
## Wallet (derived — no table)
|
||||
|
||||
Balance is always computed from `tx`. A `balance_cache` may exist for performance but is
|
||||
rebuildable and never authoritative (constitution V). Daily snapshot = full balances dump
|
||||
committed to `m2/market-registry:audit/YYYY-MM-DD.json`.
|
||||
|
||||
## Operator
|
||||
|
||||
Provisioned mapping, not a service: `operator_id` from fleet.json tenancy; CLI config
|
||||
(`~/.m2-market/config.toml`) carries `operator_id` + API keys per machine. Ledger admin
|
||||
endpoint issues starter grants against any operator_id.
|
||||
|
||||
## Proposal (Scout v0 — local JSONL + telemetry event)
|
||||
|
||||
| Field | Notes |
|
||||
|---|---|
|
||||
| `proposal_id`, `ts`, `desktop`, `listing_id`, `confidence` | |
|
||||
| `outcome` | `shown|dismissed|accepted|suppressed_rate|suppressed_optout` |
|
||||
|
||||
Outcomes are appended locally and shipped (summarized, on-box) to memory as listing
|
||||
evidence; `stats.proposals_*` on the listing updated via the indexer telemetry path.
|
||||
|
||||
## InstallState (per machine, `~/.m2-market/state.json`)
|
||||
|
||||
Mirrors fedlearn state.json design: `{installs: {listing_id: {version, grant_id, ts,
|
||||
changeset_hash, status: applied|failed|rolled_back}}}` — the idempotency + re-apply guard
|
||||
for the local adapter, and the file m2core_sync will merge into when fedlearn lands.
|
||||
170
specs/001-market-first-wedge/plan.md
Normal file
170
specs/001-market-first-wedge/plan.md
Normal file
|
|
@ -0,0 +1,170 @@
|
|||
# Implementation Plan: M2 Marketplace — First Wedge
|
||||
|
||||
**Branch**: `001-market-first-wedge` | **Date**: 2026-07-02 | **Spec**: [spec.md](./spec.md)
|
||||
|
||||
**Input**: Feature specification from `/specs/001-market-first-wedge/spec.md`
|
||||
|
||||
## Summary
|
||||
|
||||
Close the commercial loop once: schemas (frozen v1 contract) → `m2-ledger` (append-only
|
||||
credits, FastAPI+SQLite Coolify app) → listings registry on Forgejo (`m2/market-registry`,
|
||||
PR+veto review) → semantic catalog (`market:catalog` Qdrant partition on the existing
|
||||
memory-api) → `m2-market` CLI (`search|show|install`, install = debit→grant→apply) → 3 seeded
|
||||
Solutions → M2 Store (cargstore revival, canaries) → Scout v0 (one canary, after an
|
||||
exploration spike). **Critical sequencing constraint (verified 2026-07-02): the fedlearn
|
||||
rails are NOT landed** — no `m2/m2-core` repo, no capture/curator CLIs. Therefore every
|
||||
component in this wedge is built rail-independent against frozen interfaces; the two
|
||||
fedlearn touchpoints (sync/apply install backend, `commercialize` curation disposition) are
|
||||
isolated behind adapters and integrate when fedlearn lands.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Python 3.12 (ledger, CLI, indexer — uv-managed); TypeScript/React +
|
||||
Electron (M2 Store, inherited from cargstore)
|
||||
|
||||
**Primary Dependencies**: FastAPI + uvicorn (ledger), httpx + click (CLI), memory-api HTTP
|
||||
API (catalog, existing), Forgejo REST API (registry), cargstore codebase (store)
|
||||
|
||||
**Storage**: SQLite (ledger, WAL mode, single-writer service); Forgejo repo
|
||||
`m2/market-registry` (registry of record + daily ledger snapshots); Qdrant partition
|
||||
`market:catalog` via memory-api (derived index only)
|
||||
|
||||
**Testing**: pytest (ledger unit + contract, CLI integration against a local ledger +
|
||||
mocked Forgejo/memory-api); scripted end-to-end on canary per quickstart.md
|
||||
|
||||
**Target Platform**: m2 host via Coolify (`coolify` docker network) for services; m2o
|
||||
desktops (primus + agent-latest) for CLI/Store/Scout
|
||||
|
||||
**Project Type**: multi-component monorepo (this repo) + one evolved external repo
|
||||
(cargstore → M2 Store)
|
||||
|
||||
**Performance Goals**: internal-fleet scale — tens of operators, hundreds of installs;
|
||||
ledger correctness ≫ throughput; catalog search < 2s
|
||||
|
||||
**Constraints**: append-only ledger, balances always derived; no secrets in repos (keys at
|
||||
runtime); tenant firewall enforced at submission and at catalog query; canary-first
|
||||
(chris-m2o, gunnar-m2o); idempotent + reversible applies; fedlearn rails absent → adapters +
|
||||
frozen interfaces (no blocking dependency)
|
||||
|
||||
**Scale/Scope**: 6 components, ~5 new Python packages/services + 1 Electron app revival;
|
||||
3 seeded Solutions; 2 canary desktops
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: evaluated against Constitution v1.0.0 before Phase 0; re-checked after Phase 1.*
|
||||
|
||||
| Principle | Check | Status |
|
||||
|---|---|---|
|
||||
| I. Assembly over greenfield | Catalog = memory-api partition; store = cargstore evolution; registry = Forgejo; identity = fleet.json operators; the ONLY new service is m2-ledger (sanctioned) | ✅ |
|
||||
| II. Protocol + registry, not one app | Truth in `m2/market-registry` (PRs/labels); `market:catalog` rebuildable via `m2-market-indexer reindex`; CLI/Store/Scout are surfaces | ✅ |
|
||||
| III. Evidence-backed listings | `listing.schema.json` requires `evidence[]` + provenance; validation rejects before human review; conversion signals fields present from v1 | ✅ |
|
||||
| IV. Tenant firewall | `tenant_scope` on solutions; catalog queries tenant-filtered server-side; submission validator enforces owner-initiated for tenant-derived work | ✅ |
|
||||
| V. Ledger integrity | Append-only tx table, derived balances, X-API-Key, daily snapshot committed to registry repo, debit-before-apply with compensating refund | ✅ |
|
||||
| VI. Canary-first, reversible | Store + Scout on chris/gunnar only; installs idempotent via apply adapter contract; ledger is a Coolify app | ✅ |
|
||||
| VII. Simplicity, sequenced wedges | SQLite, fixed pricing, manual payouts; Scout last; deferred items listed in spec; fedlearn integration deferred behind adapters instead of half-built | ✅ |
|
||||
|
||||
**Deviation from CONCEPT.md §3 asset map (justified)**: CONCEPT assumed the fedlearn
|
||||
`m2-core-manifest` + `m2/m2-core` repo as the package/registry substrate. Verified absent.
|
||||
Rather than block the wedge or fork fedlearn's job, this plan (a) freezes
|
||||
`solution.schema.json` v1 in THIS repo as the contract fedlearn's manifest must be
|
||||
superset-compatible with (coordination note in research.md), and (b) uses a dedicated
|
||||
`m2/market-registry` repo — which was the right registry-of-record shape anyway, since
|
||||
listings are commercial artifacts, not core commons. Post-design re-check: ✅ no violations.
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/001-market-first-wedge/
|
||||
├── plan.md # This file
|
||||
├── research.md # Phase 0 output
|
||||
├── data-model.md # Phase 1 output
|
||||
├── quickstart.md # Phase 1 output
|
||||
├── contracts/ # Phase 1 output
|
||||
│ ├── ledger-api.md # m2-ledger HTTP contract
|
||||
│ ├── registry-layout.md # m2/market-registry repo layout + PR/veto protocol
|
||||
│ ├── catalog-index.md # market:catalog partition contract (memory-api)
|
||||
│ ├── cli.md # m2-market CLI command contract
|
||||
│ └── apply-adapter.md # install/apply adapter interface (fedlearn integration seam)
|
||||
└── tasks.md # Phase 2 output (/speckit-tasks — NOT created by plan)
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
|
||||
```text
|
||||
schemas/ # frozen v1 JSON Schemas (the contract)
|
||||
├── solution.schema.json
|
||||
└── listing.schema.json
|
||||
|
||||
ledger/ # m2-ledger service (FastAPI + SQLite)
|
||||
├── src/m2_ledger/
|
||||
│ ├── api.py # routes: tx, balance, grant, install-debit
|
||||
│ ├── models.py # tx record, derived-balance queries
|
||||
│ ├── snapshot.py # daily balance snapshot → registry repo commit
|
||||
│ └── auth.py # X-API-Key
|
||||
├── tests/
|
||||
├── Dockerfile # Coolify app, coolify network
|
||||
└── pyproject.toml
|
||||
|
||||
cli/ # m2-market CLI (uv package)
|
||||
├── src/m2_market/
|
||||
│ ├── cli.py # search | show | install | wallet
|
||||
│ ├── catalog.py # memory-api client (market:catalog)
|
||||
│ ├── registry.py # Forgejo client (listings, bundles, releases)
|
||||
│ ├── ledger.py # m2-ledger client
|
||||
│ └── apply/ # apply-adapter implementations
|
||||
│ ├── base.py # AdapterProtocol (frozen interface, see contracts/)
|
||||
│ ├── local.py # v1: local bundle apply (files + recipe), idempotent
|
||||
│ └── m2core_sync.py # stub → real when fedlearn lands
|
||||
├── tests/
|
||||
└── pyproject.toml
|
||||
|
||||
indexer/ # registry → market:catalog sync (small, cron/webhook)
|
||||
├── src/m2_market_indexer/
|
||||
│ ├── reindex.py # full rebuild (constitution II)
|
||||
│ └── watch.py # incremental on merge
|
||||
└── pyproject.toml
|
||||
|
||||
solutions/ # seeded Solution bundles (sources for the 3 listings)
|
||||
├── mm-pdf-report/
|
||||
├── agent-scaffold/
|
||||
└── competitor-scan/
|
||||
|
||||
scout/ # Scout v0 (AFTER exploration spike; canary-only)
|
||||
└── SPIKE.md # spike findings → host decision (operator: "explore")
|
||||
|
||||
store/ # NOT here — cargstore fork evolves in its own repo
|
||||
└── README.md # pointer + integration contract references
|
||||
```
|
||||
|
||||
**Structure Decision**: single monorepo (this repo) for schemas/ledger/CLI/indexer/solutions
|
||||
— they version together against the frozen schemas. The M2 Store stays in the cargstore
|
||||
repo (github.com/machine-machine/cargstore, branch `m2-store`) because it is an evolution of
|
||||
a living codebase (constitution I); it consumes the same public contracts (`contracts/`).
|
||||
`m2/market-registry` on Forgejo is data, not code — created by a task, not a directory here.
|
||||
|
||||
## Component Plan & Sequencing (rail-independence explicit)
|
||||
|
||||
| # | Component | Depends on | fedlearn coupling |
|
||||
|---|---|---|---|
|
||||
| 1 | `schemas/` v1 frozen | — | NONE now; coordination note → fedlearn manifest superset-compat |
|
||||
| 2 | `m2/market-registry` repo + PR/veto protocol | Forgejo (live) | replaces curation-pipeline dependency; `commercialize` disposition plugs in later |
|
||||
| 3 | `m2-ledger` service + wallets + grants | Coolify (live) | NONE |
|
||||
| 4 | `market:catalog` partition + indexer | memory-api (live), #2 | NONE |
|
||||
| 5 | `m2-market` CLI (search/show/install) | #1–#4 | apply via AdapterProtocol; v1 `local` adapter, `m2core_sync` stub |
|
||||
| 6 | Seed 3 Solutions + listings | #1, #2, #5 | NONE (manual packaging from proven outcomes) |
|
||||
| 7 | **GATE: SC-001 paid install** (CLI path) | #1–#6 | NONE |
|
||||
| 8 | M2 Store (cargstore → `m2-store` branch), canaries | #4, #5 contracts | NONE |
|
||||
| 9 | Scout spike → Scout v0, one canary | #4; herdr run summaries (live) | NONE |
|
||||
| 10 | fedlearn integration pass | fedlearn lands | swap `m2core_sync` adapter real; wire `commercialize` disposition → registry PR |
|
||||
|
||||
Row 7 is the wedge's success criterion and does not wait for rows 8–10.
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
> Constitution Check passed — one recorded deviation (registry substrate), justified above.
|
||||
|
||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
||||
|-----------|------------|-------------------------------------|
|
||||
| New repo `m2/market-registry` instead of fedlearn `m2/m2-core` | m2-core does not exist yet; listings are commercial artifacts, not core commons | Waiting for fedlearn blocks the whole wedge on an unlanded dependency; writing into m2-core would entangle free-core and paid-market boundaries (constitution IV) |
|
||||
81
specs/001-market-first-wedge/quickstart.md
Normal file
81
specs/001-market-first-wedge/quickstart.md
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
# Quickstart — validate 001-market-first-wedge end-to-end
|
||||
|
||||
Runnable validation scenarios proving the wedge works. These are the converge-stage gates;
|
||||
SC references map to spec.md Success Criteria.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- m2-ledger deployed on Coolify (`GET /health` → healthy), admin + service keys in env.
|
||||
- `m2/market-registry` exists on Forgejo with schemas mirrored and CI validation active.
|
||||
- memory-api reachable; `market:catalog` partition created.
|
||||
- `m2-market` CLI installed on two machines mapped to two DIFFERENT operator_ids
|
||||
(e.g. m2bd and sdjs-operator); `apply_adapter = "local"`.
|
||||
- ≥1 Solution bundle packaged from `solutions/` and listed (published) with real evidence.
|
||||
|
||||
## Scenario A — publish with evidence (Story 2, SC-002)
|
||||
|
||||
```bash
|
||||
m2-market publish solutions/mm-pdf-report # opens listing PR
|
||||
# expect: PR with evidence/price/permissions/rollback template; CI green
|
||||
# reviewer merges (no veto label)
|
||||
m2-market search "branded pdf report" # expect: listing returned with price
|
||||
```
|
||||
Negative: strip `evidence[]` from a copy → `m2-market publish` exits 5 (or CI hard-fails).
|
||||
|
||||
## Scenario B — funded install, the money move (Story 1, SC-001/003/005)
|
||||
|
||||
```bash
|
||||
# seller: sdjs-operator (listing owner) · buyer: m2bd
|
||||
curl -X POST $LEDGER/grant/starter -H "X-API-Key: $ADMIN" -d '{"operator_id":"m2bd"}'
|
||||
m2-market wallet # expect: 100 cr
|
||||
time m2-market install lst_mm_pdf_report --yes # expect: exit 0, entrypoint hint printed
|
||||
m2-market wallet # expect: 100 - price
|
||||
curl $LEDGER/balance/sdjs-operator # expect: +price*0.9
|
||||
curl $LEDGER/balance/platform # expect: +price*0.1
|
||||
curl $LEDGER/license/m2bd/lst_mm_pdf_report # expect: 200 grant
|
||||
# invoke the installed solution's documented entrypoint → produces its outcome (AC-4)
|
||||
```
|
||||
`time` for the whole discover→install path < 10 min (SC-005).
|
||||
|
||||
## Scenario C — refusal, idempotency, refund (Story 1 AC-2/3, Story 4 AC-3)
|
||||
|
||||
```bash
|
||||
# fresh operator, no grant:
|
||||
m2-market install lst_mm_pdf_report --yes # expect: exit 3, no tx, no grant, no files
|
||||
# re-run a completed install:
|
||||
m2-market install lst_mm_pdf_report --yes # expect: exit 0 no-op, no second debit
|
||||
# force an apply failure (recipe check that fails on this machine):
|
||||
m2-market install lst_broken_fixture --yes # expect: exit 4, refund rows present:
|
||||
curl "$LEDGER/tx?reason=refund" # expect: compensating pair for that ref
|
||||
```
|
||||
|
||||
## Scenario D — ledger audit properties (SC-003/004)
|
||||
|
||||
```bash
|
||||
curl -X POST $LEDGER/snapshot -H "X-API-Key: $ADMIN"
|
||||
# in m2/market-registry: audit/<today>.json committed
|
||||
pytest ledger/tests/test_properties.py # balance==fold(log); grants all have tx
|
||||
```
|
||||
|
||||
## Scenario E — catalog rebuildability (constitution II)
|
||||
|
||||
```bash
|
||||
m2-market reindex # drop + rebuild market:catalog
|
||||
m2-market search "branded pdf report" # same results as before reindex
|
||||
```
|
||||
|
||||
## Scenario F — M2 Store on canary (Story 4) [after row 8]
|
||||
|
||||
On chris-m2o: open M2 Store → seeded listings render → install from a Solution page →
|
||||
Scenario B post-conditions hold; UI reports the result truthfully.
|
||||
|
||||
## Scenario G — Scout v0 on canary (Story 5, SC-006) [after row 9 + spike]
|
||||
|
||||
On the opted-in canary, run a session resembling a seeded listing's intent → one proposal
|
||||
toast with deep link; opt-out desktop shows nothing; 6th proposal of the day suppressed;
|
||||
outcomes appear in `market:evidence`.
|
||||
|
||||
## The wedge gate
|
||||
|
||||
SC-001 is met when Scenario B (and C's guards) pass between two real operators with a real
|
||||
listing — via CLI alone. Scenarios F/G extend the same loop to Store and Scout surfaces.
|
||||
136
specs/001-market-first-wedge/research.md
Normal file
136
specs/001-market-first-wedge/research.md
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
# Research — 001-market-first-wedge (Phase 0)
|
||||
|
||||
All NEEDS CLARIFICATION items from Technical Context resolved below. Sources: CONCEPT.md
|
||||
v0.2, context/SYSTEM-MAP.md (verified 2026-07-01), live probes 2026-07-02, operator steer
|
||||
(README §open-forks edit 2026-07-02), Constitution v1.0.0.
|
||||
|
||||
## R1. Ledger substrate
|
||||
|
||||
- **Decision**: standalone `m2-ledger` — FastAPI + SQLite (WAL), single-writer Coolify app
|
||||
on the `coolify` network, X-API-Key auth (same pattern as memory-api). Ledger API kept
|
||||
strictly separable from storage (repository layer) so a future migration into extended
|
||||
m2-gpt billing — including the operator's noted crypto-ledger direction — is a backend
|
||||
swap, not an API break.
|
||||
- **Rationale**: operator confirmed standalone-now (README steer); m2-gpt gateway already
|
||||
meters tokens/tenants, so the ledger *federates* (reads identity, later maybe settlement)
|
||||
rather than duplicating; SQLite is constitution-VII-simple and one service is the only
|
||||
sanctioned new component.
|
||||
- **Alternatives considered**: extend m2-gpt billing (rejected for v1: couples marketplace
|
||||
release cadence to gateway deploys; revisit post-wedge per operator note);
|
||||
Forgejo-as-ledger (rejected: git is the audit trail, not a transactional store — race
|
||||
conditions on concurrent installs).
|
||||
|
||||
## R2. Registry of record (fedlearn absence)
|
||||
|
||||
- **Decision**: create Forgejo repo **`m2/market-registry`**: `listings/<listing_id>/`
|
||||
directories (listing.json + evidence/ + bundle ref), Solution bundles attached as repo
|
||||
releases (`<listing_id>-v<semver>`), review = PR, veto = labels (`veto`, `approved`),
|
||||
merge = published. Daily ledger snapshots commit to `audit/` in the same repo.
|
||||
- **Rationale**: verified 2026-07-02 that `m2/m2-core` does not exist and the fedlearn herd
|
||||
has not started (fleet busy on m2-gpt work) — blocking on it would stall the wedge.
|
||||
Listings are commercial artifacts; a dedicated registry keeps the free-core/paid-market
|
||||
boundary physical (constitution IV). Forgejo is live today (forgejo skill available for
|
||||
API access).
|
||||
- **Alternatives considered**: wait for `m2/m2-core` (rejected: unlanded dependency);
|
||||
listings inside m2-core (rejected: erodes commons boundary); one-repo-per-listing
|
||||
(rejected for v1: heavier to administer; releases-in-one-repo suffices at this scale).
|
||||
- **Integration seam**: when fedlearn's curation pipeline lands, its `commercialize`
|
||||
disposition emits a PR against `m2/market-registry` — the registry's PR/veto protocol is
|
||||
the frozen interface (contracts/registry-layout.md).
|
||||
|
||||
## R3. Schema freeze order (manifest superset problem)
|
||||
|
||||
- **Decision**: freeze `solution.schema.json` v1 **in this repo now**, embedding the
|
||||
manifest-shaped core fields observed in the fedlearn plan's submission example
|
||||
(`schema_version`, `applicability{image_classes,roles,tool_requirements}`, `tenant_scope`,
|
||||
`evidence[]`, `content_hash`, scrub status) plus commercial fields (`price`, `seller`,
|
||||
`license`, `revenue_split`). File a coordination note to the fedlearn effort: its final
|
||||
manifest must remain field-compatible with this core, or a v1→v2 schema migration is
|
||||
planned at integration.
|
||||
- **Rationale**: the spec (FR-001) requires a frozen contract; fedlearn's own schema task
|
||||
(their Task 1.1) is unexecuted, so "superset of the manifest" is only satisfiable by
|
||||
freezing our side and coordinating. Frozen interfaces are exactly how the constitution
|
||||
says to sequence around unlanded rails.
|
||||
- **Alternatives considered**: wait for fedlearn's freeze (rejected: unbounded wait);
|
||||
schema-less v1 (rejected: FR-001/FR-002 are explicit).
|
||||
|
||||
## R4. Catalog partition mechanics
|
||||
|
||||
- **Decision**: `market:catalog` as a memory-api partition (agent_id-style namespace, same
|
||||
BGE-M3 hybrid path as `fedlearn:core-index` was designed for). Writes go only through
|
||||
`m2-market-indexer`, which reads merged listings from `m2/market-registry`;
|
||||
`reindex` rebuilds the partition from scratch (constitution II rebuildability). Tenant
|
||||
filtering applied at query time via listing `tenant_scope` payload field.
|
||||
- **Rationale**: SYSTEM-MAP confirms memory-api live with partition model; one more
|
||||
partition is the assembly move. Known risk: the memory-api split-brain (two Coolify
|
||||
stacks) — fedlearn task T13 was to resolve it; until then the indexer targets the stack
|
||||
behind `memory.machinemachine.ai` and `reindex` makes recovery cheap.
|
||||
- **Alternatives considered**: separate Qdrant collection managed directly (rejected:
|
||||
bypasses memory-api auth + embedding pipeline); SQL full-text in the ledger DB (rejected:
|
||||
no semantic match, Scout needs embeddings).
|
||||
|
||||
## R5. Install/apply path without m2-core-sync
|
||||
|
||||
- **Decision**: define `AdapterProtocol` (contracts/apply-adapter.md): `plan(bundle) →
|
||||
changeset`, `apply(changeset) → result`, `verify(bundle) → bool`, `rollback(changeset)`,
|
||||
all idempotent. v1 ships the `local` adapter: bundle = tarball release containing
|
||||
`solution.json` + `payload/` + `recipe.yaml` (declarative file placements + post-install
|
||||
checks) applied into the operator's home/agent volume, with a state file
|
||||
(`~/.m2-market/state.json`) mirroring the fedlearn state.json design. `m2core_sync`
|
||||
adapter is a stub that errors with "fedlearn rails not landed".
|
||||
- **Rationale**: runtime-sync is the only universal path fleet-wide, but its implementation
|
||||
(m2-core-sync) doesn't exist yet; the adapter seam lets the wedge close SC-001 with the
|
||||
local adapter and swap in m2-core-sync without touching CLI or Store.
|
||||
- **Alternatives considered**: implement m2-core-sync ourselves (rejected: forks fedlearn's
|
||||
in-plan work, constitution's "consume, not fork"); flatpak-style backend from cargstore
|
||||
(rejected: desktop-app-shaped, not agent-bundle-shaped).
|
||||
|
||||
## R6. Scout host (operator: "explore")
|
||||
|
||||
- **Decision**: DEFERRED to a time-boxed spike (task in Phase 2; output `scout/SPIKE.md`)
|
||||
comparing: (a) standalone supervised watcher (hermes-gateway pattern), (b) Hermes plugin,
|
||||
(c) herdr plugin/hook. Spike evaluates: access to structured run summaries, opt-in policy
|
||||
enforcement point, on-box summarization cost, notification surface, and update/rollout
|
||||
story on both image classes. Scout v0 implements the spike's winner on ONE canary.
|
||||
- **Rationale**: operator explicitly marked this fork "explore"; Scout is P3/last-sequenced
|
||||
so the spike doesn't block the wedge gate (SC-001).
|
||||
|
||||
## R7. Store surface
|
||||
|
||||
- **Decision**: fork-branch `m2-store` on cargstore; keep Electron+React shell, replace
|
||||
Flatpak catalog source with `market:catalog` (via memory-api HTTP) and add an install
|
||||
backend that shells to `m2-market install` (one code path for FR-005 semantics). Auth:
|
||||
desktop's existing m2-gpt-issued credentials pattern (operator steer) — the store reads
|
||||
the same key material provisioned on the desktop, never stores its own secrets.
|
||||
- **Rationale**: operator steer ("MVP electron app... credentials from m2-gpt"); reusing
|
||||
the CLI for install guarantees ledger semantics are implemented once.
|
||||
- **Alternatives considered**: web-first (deferred per steer); re-implementing install in
|
||||
the store's Node side (rejected: two implementations of debit→grant→apply invites drift).
|
||||
|
||||
## R8. Identity & wallets
|
||||
|
||||
- **Decision**: `operator_id` = the operator identity from fleet.json tenancy (e.g.
|
||||
`m2bd`, `sdjs-operator`); wallets keyed by operator_id in m2-ledger; machine→operator
|
||||
resolution shipped as a small mapping in the CLI config (provisioned, not guessed).
|
||||
Starter grants issued via an admin-authenticated ledger endpoint (default 100 cr —
|
||||
UNCONFIRMED default, flagged).
|
||||
- **Rationale**: fleet.json is the existing identity substrate (constitution I); v1 needs
|
||||
no new auth system — API keys per service, operator mapping per machine.
|
||||
|
||||
## R9. Testing strategy
|
||||
|
||||
- **Decision**: pytest for ledger (property-style tests on derived balances: balance ==
|
||||
fold(tx log) under generated tx sequences) and CLI (integration against a real local
|
||||
ledger instance + recorded Forgejo/memory-api fixtures); a scripted quickstart
|
||||
(quickstart.md) as the canary E2E gate; schema validation tests for all seeded bundles.
|
||||
- **Rationale**: SC-003/SC-004 are audit properties — property tests are the cheap way to
|
||||
hold them; canary E2E is the constitution-VI rollout gate.
|
||||
|
||||
## Open coordination items (not blockers)
|
||||
|
||||
1. Coordination note → fedlearn effort: manifest/schema field-compat with frozen
|
||||
`solution.schema.json` v1 core (R3).
|
||||
2. memory-api split-brain (fedlearn T13): until resolved, indexer pins the public endpoint;
|
||||
reindex is the recovery path (R4).
|
||||
3. Operator confirmations outstanding: platform cut 10%, starter grant 100 cr (spec
|
||||
Assumptions; ledger ships them as config, not constants).
|
||||
Loading…
Reference in a new issue