Phase 1: router skeleton + Qdrant read path - FastAPI app with /health, /search/generic, /ingest endpoints - Qdrant reader (hybrid dense+sparse w/ RRF, plus payload scroll) - BGE-M3 TEI client for query-side embedding - memory-api client proxying writes to /memory/store (m2-memory untouched) - Pydantic Fact + Exemplar payloads, discriminated by `kind` - Dockerfile + docker-compose joining the coolify Docker network Phase 2: authoritative-source ingest pipeline - PDF text extraction + paragraph-aware (§ N) and size chunkers - Loaders for bnatschg, mhbasp, biotopwertliste, state-<slug> - CLI: artenschutz-ingest --source <name> [--states ...] [--dry-run] - Helper script to fetch BNatSchG from gesetze-im-internet.de - End-to-end smoke script (scripts/smoke.sh) for Coolify validation 22/22 unit tests pass. Real-world dry-runs verified against mhbasp Anhang 4, biotopwertlisteNEU and Berlin Kartierstandards PDFs from GST-DATA. See COOLIFY-DEPLOY.md for staging deploy + smoke procedure.
4.3 KiB
Coolify staging deploy — artenschutz-router
Goal: stand up the router on Coolify staging, in the same Docker network as
agent.memory.system, so it can reach memory-qdrant, memory-embeddings,
and memory-api by container name. Then run an end-to-end smoke ingest +
search against real data.
Pre-flight
You should already have on staging Coolify:
- A running
agent.memory.system(memory-api + memory-qdrant + memory-embeddings) - The Docker network name those services live on (probably
coolify— checkdocker network lson the staging host)
Note down:
| Setting | Value (example) |
|---|---|
| Coolify project | gruenstifter-staging |
| Docker network | coolify |
| memory-api container name | memory-api-<id> |
| memory-qdrant container name | memory-qdrant-<id> |
| memory-embeddings container name | memory-embeddings-<id> |
The full container names include Coolify's stack hash suffix. The compose file uses the short service names (
memory-api, etc.) which work when the router is on the same Coolify-managed network — Coolify creates service-name aliases. If they don't resolve, fall back to container IPs viadocker inspect.
Path A — Push to forgejo, let Coolify auto-deploy (preferred)
cd /home/m2/artenschutz-router
git init -b main
git add .
git commit -m "Initial commit — phase 1 + 2"
git remote add origin <forgejo-url>:gruenstifter/artenschutz-router.git
git push -u origin main
In Coolify staging:
- New Resource → Application → Public Git Repository (or Private, add the SSH key)
- Source: the forgejo URL above
- Build Pack: Dockerfile
- Set environment variables:
QDRANT_URL=http://memory-qdrant:6333 QDRANT_COLLECTION=agent_memory BGE_TEI_URL=http://memory-embeddings:8000 MEMORY_API_URL=http://memory-api:8000 ARTENSCHUTZ_AGENT_ID=gruenstifter_staging ROUTER_PORT=8080 LOG_LEVEL=INFOgruenstifter_stagingkeeps smoke data cleanly separable from any future production write. - Network: join the same network as
agent.memory.system(usuallycoolify). - Port: 8080 (no public exposure needed for smoke — internal-only is fine).
- Deploy.
Path B — Docker Compose on the Coolify host
If you'd rather not push to forgejo yet:
# On the Coolify host (or via SSH):
scp -r /home/m2/artenschutz-router coolify-host:/opt/coolify-apps/
ssh coolify-host
cd /opt/coolify-apps/artenschutz-router
cp .env.example .env
# Edit .env to set ARTENSCHUTZ_AGENT_ID=gruenstifter_staging
docker compose build
docker compose up -d
docker compose logs -f artenschutz-router
This bypasses Coolify's UI but uses the same coolify external network the
memory stack joins.
Verify it's up
From any shell on the Coolify host:
docker exec artenschutz-router curl -sS http://localhost:8080/health
# expect: {"status":"ok","qdrant":true,"bge":true,"memory_api":true}
status: "degraded" with one of the three false means that service isn't
reachable from inside the router's network namespace — fix container names /
network attachment before continuing.
End-to-end smoke (uses the test data already symlinked in sources/)
# 1. Fetch BNatSchG into the container's sources/bnatschg/ if not already.
docker exec artenschutz-router python scripts/fetch_bnatschg.py
# 2. Run the smoke (ingests small batches + queries them back).
docker exec artenschutz-router bash scripts/smoke.sh
The smoke script ingests ~5 records per source, then runs three searches:
- Hybrid query
"Fledermäuse Kartierung"filtered tofact_type=method - Hybrid query
"besonders geschützte Arten"filtered tofact_type=legal - Payload-only scroll on
scope.states=BEto verify state filtering
Expected: each search returns ≥1 hit. If hybrid returns 0 hits but scroll works, BGE-M3 embedding is failing — check the logs.
Cleanup (after smoke)
Drop the staging agent_id from Qdrant:
docker exec memory-qdrant-<id> curl -X POST \
http://localhost:6333/collections/agent_memory/points/delete \
-H 'Content-Type: application/json' \
-d '{"filter":{"must":[{"key":"agent_id","match":{"value":"gruenstifter_staging"}}]}}'
This wipes every point we wrote under the staging agent_id. Nothing else is
affected because partition isolation is by agent_id.