m2 (AI Agent)
6da874b2e6
Phase 1: router skeleton + Qdrant read path - FastAPI app with /health, /search/generic, /ingest endpoints - Qdrant reader (hybrid dense+sparse w/ RRF, plus payload scroll) - BGE-M3 TEI client for query-side embedding - memory-api client proxying writes to /memory/store (m2-memory untouched) - Pydantic Fact + Exemplar payloads, discriminated by `kind` - Dockerfile + docker-compose joining the coolify Docker network Phase 2: authoritative-source ingest pipeline - PDF text extraction + paragraph-aware (§ N) and size chunkers - Loaders for bnatschg, mhbasp, biotopwertliste, state-<slug> - CLI: artenschutz-ingest --source <name> [--states ...] [--dry-run] - Helper script to fetch BNatSchG from gesetze-im-internet.de - End-to-end smoke script (scripts/smoke.sh) for Coolify validation 22/22 unit tests pass. Real-world dry-runs verified against mhbasp Anhang 4, biotopwertlisteNEU and Berlin Kartierstandards PDFs from GST-DATA. See COOLIFY-DEPLOY.md for staging deploy + smoke procedure.
67 lines
2.1 KiB
Markdown
67 lines
2.1 KiB
Markdown
# artenschutz-router
|
|
|
|
Sidecar service for Artenschutzgutachten domain queries against
|
|
`agent.memory.system`. **m2-memory stays untouched.**
|
|
|
|
- **Writes** proxy memory-api `/memory/store` — the existing endpoint accepts
|
|
arbitrary `metadata: dict`, so our Fact / Exemplar payloads slot in.
|
|
- **Reads** go directly to Qdrant for rich payload-filtered hybrid queries
|
|
(dense + sparse via the existing BGE-M3 TEI service).
|
|
|
|
See `.planning/notes/artenschutz-digest-concept.md` in `GST-DATA` for the
|
|
draft concept that birthed this.
|
|
|
|
## Endpoints
|
|
|
|
| Method | Path | Purpose |
|
|
|---|---|---|
|
|
| `GET` | `/health` | Live-probe Qdrant, BGE-M3, memory-api |
|
|
| `POST` | `/search/generic` | Hybrid search (or scroll if no `query`); arbitrary payload filters |
|
|
| `POST` | `/ingest` | Push a Fact or Exemplar record |
|
|
|
|
Domain-specific endpoints (`/facts/search`, `/exemplar/search`,
|
|
`/mitigation/search`) are deferred to a later phase — they will be thin
|
|
wrappers over `/search/generic`.
|
|
|
|
## Quick start (local dev)
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env to point QDRANT_URL / BGE_TEI_URL / MEMORY_API_URL at your stack
|
|
docker compose up --build
|
|
curl http://localhost:8080/health
|
|
```
|
|
|
|
If you're targeting the production Coolify memory stack, leave the default
|
|
container-name URLs and ensure the `coolify` Docker network is joined.
|
|
|
|
## Running ingest
|
|
|
|
```bash
|
|
# After you've placed authoritative source files into sources/...
|
|
artenschutz-ingest --source bnatschg --router-url http://localhost:8080
|
|
artenschutz-ingest --source mhbasp --router-url http://localhost:8080
|
|
artenschutz-ingest --source biotopwertliste --router-url http://localhost:8080
|
|
artenschutz-ingest --source state-berlin --router-url http://localhost:8080
|
|
```
|
|
|
|
See `sources/README.md` for which files each source expects.
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
pip install -e .[dev]
|
|
pytest
|
|
```
|
|
|
|
## Architecture
|
|
|
|
```
|
|
m2-gpt agents
|
|
↓
|
|
artenschutz-router (this repo)
|
|
├─ /ingest ─→ memory-api /memory/store (unchanged)
|
|
└─ /search/generic ─→ Qdrant /points/query + BGE-M3 TEI /embed[_sparse]
|
|
↓
|
|
agent_memory collection (Qdrant)
|
|
```
|