artenschutz-router

2 commits 1 branch 0 tags 63 KiB

Author	SHA1	Message	Date
m2 (AI Agent)	04d2f5b9e4	fix: align router with memory-api's actual payload shape (smoke findings) End-to-end smoke against the live coolify memory stack surfaced three gaps between the spec and what memory-api actually writes: 1. memory-api wraps the /ingest payload under `metadata.*` in Qdrant — our domain fields (fact_type, scope.states, …) don't live at the top of the payload, they're nested. `build_qdrant_filter` now auto-prefixes any non-top-level key with `metadata.`. Callers can write `{"fact_type": "legal"}` and it just works; top-level fields (`agent_id`, `memory_type`, `source`, …) pass through unprefixed. 2. The deployed BGE-M3 TEI image doesn't serve /embed_sparse (returns 424 Failed Dependency). Stored points have `has_sparse=false`. `BGEClient.embed_sparse` now returns `{}` on 4xx and logs a warning; `QdrantReader.hybrid_query` falls back to a dense-only query when sparse is empty. RRF fusion is skipped in that path. 3. `scripts/smoke.sh` had a missing `shift` after reading the limit positional arg — caused arg leakage onto the CLI (`--limit 5 5`). The cleanup hint also referenced a stale agent_id; now reads from $ARTENSCHUTZ_AGENT_ID. Smoke is now green end-to-end: 18 records ingested across bnatschg / mhbasp / biotopwertliste(BY) / state-berlin under agent_id=gruenstifter_smoketest, hybrid + payload-filter searches all return hits. 24/24 unit tests pass. Companion fixes pushed to github.com/machine-machine/agent.memory.system: d0e9529 fix: /memory/store passed wrong kwarg name to AgentMemory ebc896c fix: don't close caller-supplied clients in AgentMemory.close()	2026-05-14 23:30:57 +02:00
m2 (AI Agent)	6da874b2e6	Initial commit — phase 1 + 2 of artenschutz-digest concept Phase 1: router skeleton + Qdrant read path - FastAPI app with /health, /search/generic, /ingest endpoints - Qdrant reader (hybrid dense+sparse w/ RRF, plus payload scroll) - BGE-M3 TEI client for query-side embedding - memory-api client proxying writes to /memory/store (m2-memory untouched) - Pydantic Fact + Exemplar payloads, discriminated by `kind` - Dockerfile + docker-compose joining the coolify Docker network Phase 2: authoritative-source ingest pipeline - PDF text extraction + paragraph-aware (§ N) and size chunkers - Loaders for bnatschg, mhbasp, biotopwertliste, state-<slug> - CLI: artenschutz-ingest --source <name> [--states ...] [--dry-run] - Helper script to fetch BNatSchG from gesetze-im-internet.de - End-to-end smoke script (scripts/smoke.sh) for Coolify validation 22/22 unit tests pass. Real-world dry-runs verified against mhbasp Anhang 4, biotopwertlisteNEU and Berlin Kartierstandards PDFs from GST-DATA. See COOLIFY-DEPLOY.md for staging deploy + smoke procedure.	2026-05-14 16:36:00 +02:00

Author

SHA1

Message

Date

m2 (AI Agent)

04d2f5b9e4

fix: align router with memory-api's actual payload shape (smoke findings)

End-to-end smoke against the live coolify memory stack surfaced three
gaps between the spec and what memory-api actually writes:

1. memory-api wraps the /ingest payload under `metadata.*` in Qdrant —
   our domain fields (fact_type, scope.states, …) don't live at the top
   of the payload, they're nested. `build_qdrant_filter` now auto-prefixes
   any non-top-level key with `metadata.`. Callers can write
   `{"fact_type": "legal"}` and it just works; top-level fields
   (`agent_id`, `memory_type`, `source`, …) pass through unprefixed.

2. The deployed BGE-M3 TEI image doesn't serve /embed_sparse (returns
   424 Failed Dependency). Stored points have `has_sparse=false`.
   `BGEClient.embed_sparse` now returns `{}` on 4xx and logs a warning;
   `QdrantReader.hybrid_query` falls back to a dense-only query when
   sparse is empty. RRF fusion is skipped in that path.

3. `scripts/smoke.sh` had a missing `shift` after reading the limit
   positional arg — caused arg leakage onto the CLI (`--limit 5 5`).
   The cleanup hint also referenced a stale agent_id; now reads from
   $ARTENSCHUTZ_AGENT_ID.

Smoke is now green end-to-end: 18 records ingested across bnatschg /
mhbasp / biotopwertliste(BY) / state-berlin under
agent_id=gruenstifter_smoketest, hybrid + payload-filter searches all
return hits. 24/24 unit tests pass.

Companion fixes pushed to github.com/machine-machine/agent.memory.system:
  d0e9529  fix: /memory/store passed wrong kwarg name to AgentMemory
  ebc896c  fix: don't close caller-supplied clients in AgentMemory.close()

2026-05-14 23:30:57 +02:00

m2 (AI Agent)

6da874b2e6

Initial commit — phase 1 + 2 of artenschutz-digest concept

Phase 1: router skeleton + Qdrant read path
- FastAPI app with /health, /search/generic, /ingest endpoints
- Qdrant reader (hybrid dense+sparse w/ RRF, plus payload scroll)
- BGE-M3 TEI client for query-side embedding
- memory-api client proxying writes to /memory/store (m2-memory untouched)
- Pydantic Fact + Exemplar payloads, discriminated by `kind`
- Dockerfile + docker-compose joining the coolify Docker network

Phase 2: authoritative-source ingest pipeline
- PDF text extraction + paragraph-aware (§ N) and size chunkers
- Loaders for bnatschg, mhbasp, biotopwertliste, state-<slug>
- CLI: artenschutz-ingest --source <name> [--states ...] [--dry-run]
- Helper script to fetch BNatSchG from gesetze-im-internet.de
- End-to-end smoke script (scripts/smoke.sh) for Coolify validation

22/22 unit tests pass. Real-world dry-runs verified against mhbasp Anhang 4,
biotopwertlisteNEU and Berlin Kartierstandards PDFs from GST-DATA.

See COOLIFY-DEPLOY.md for staging deploy + smoke procedure.

2026-05-14 16:36:00 +02:00