m2 (AI Agent) 6da874b2e6 Initial commit — phase 1 + 2 of artenschutz-digest concept

Phase 1: router skeleton + Qdrant read path
- FastAPI app with /health, /search/generic, /ingest endpoints
- Qdrant reader (hybrid dense+sparse w/ RRF, plus payload scroll)
- BGE-M3 TEI client for query-side embedding
- memory-api client proxying writes to /memory/store (m2-memory untouched)
- Pydantic Fact + Exemplar payloads, discriminated by `kind`
- Dockerfile + docker-compose joining the coolify Docker network

Phase 2: authoritative-source ingest pipeline
- PDF text extraction + paragraph-aware (§ N) and size chunkers
- Loaders for bnatschg, mhbasp, biotopwertliste, state-<slug>
- CLI: artenschutz-ingest --source <name> [--states ...] [--dry-run]
- Helper script to fetch BNatSchG from gesetze-im-internet.de
- End-to-end smoke script (scripts/smoke.sh) for Coolify validation

22/22 unit tests pass. Real-world dry-runs verified against mhbasp Anhang 4,
biotopwertlisteNEU and Berlin Kartierstandards PDFs from GST-DATA.

See COOLIFY-DEPLOY.md for staging deploy + smoke procedure.

2026-05-14 16:36:00 +02:00

2.1 KiB

Raw Permalink Blame History

artenschutz-router

Sidecar service for Artenschutzgutachten domain queries against agent.memory.system. m2-memory stays untouched.

Writes proxy memory-api /memory/store — the existing endpoint accepts arbitrary metadata: dict, so our Fact / Exemplar payloads slot in.
Reads go directly to Qdrant for rich payload-filtered hybrid queries (dense + sparse via the existing BGE-M3 TEI service).

See .planning/notes/artenschutz-digest-concept.md in GST-DATA for the draft concept that birthed this.

Endpoints

Method	Path	Purpose
`GET`	`/health`	Live-probe Qdrant, BGE-M3, memory-api
`POST`	`/search/generic`	Hybrid search (or scroll if no `query`); arbitrary payload filters
`POST`	`/ingest`	Push a Fact or Exemplar record

Domain-specific endpoints (/facts/search, /exemplar/search, /mitigation/search) are deferred to a later phase — they will be thin wrappers over /search/generic.

Quick start (local dev)

cp .env.example .env
# Edit .env to point QDRANT_URL / BGE_TEI_URL / MEMORY_API_URL at your stack
docker compose up --build
curl http://localhost:8080/health

If you're targeting the production Coolify memory stack, leave the default container-name URLs and ensure the coolify Docker network is joined.

Running ingest

# After you've placed authoritative source files into sources/...
artenschutz-ingest --source bnatschg --router-url http://localhost:8080
artenschutz-ingest --source mhbasp --router-url http://localhost:8080
artenschutz-ingest --source biotopwertliste --router-url http://localhost:8080
artenschutz-ingest --source state-berlin --router-url http://localhost:8080

See sources/README.md for which files each source expects.

Tests

pip install -e .[dev]
pytest

Architecture

m2-gpt agents
   ↓
artenschutz-router (this repo)
   ├─ /ingest        ─→ memory-api /memory/store (unchanged)
   └─ /search/generic ─→ Qdrant /points/query + BGE-M3 TEI /embed[_sparse]
                            ↓
                       agent_memory collection (Qdrant)

2.1 KiB Raw Permalink Blame History