2.0 KiB
ADR-002: Docs → Qdrant sync strategy
Status: Accepted (default recommendation — flag in review if you want polling instead) Date: 2026-08-02
Context
The Knowledge Ingestion Service (founding strategy, Section 3) needs to keep Qdrant's embeddings in sync with source documents (Docker Compose files, Markdown docs, configs) living in Gitea. Two options:
- Polling: the Ingestion Service periodically checks Gitea for changes and re-syncs.
- Webhook: Gitea pushes a notification to the Ingestion Service on every relevant commit, triggering an immediate re-sync.
Polling is simpler to build first (no inbound endpoint, no webhook secret to manage) but introduces staleness by design — the exact problem flagged in the founding strategy, Section 1.5 ("knowledge freshness" as a first-class requirement). Webhook-based sync is barely more work at this scale (Gitea supports repository webhooks natively) and directly serves the freshness requirement.
Decision
Use a Gitea webhook on relevant repositories, calling a Backend API endpoint that triggers re-ingestion of the changed files. Every ingested chunk stores a last_synced_at timestamp sourced from the commit, surfaced in every Knowledge Agent answer (per Section 1.5).
A lightweight periodic reconciliation job (e.g. once daily) remains as a safety net in case a webhook delivery is missed — this is not the primary mechanism, just a backstop.
Consequences
- Requires a webhook secret to be configured and kept out of version control (
.env, never committed). - Ingestion Service needs an inbound endpoint reachable from Gitea — trivial on the same homelab network, worth remembering if Gitea and the platform ever live on separate networks.
- Freshness timestamps become a real, enforceable product feature from day one rather than an afterthought.
- If webhook reliability becomes a problem in practice, the daily reconciliation job already covers the failure mode — no redesign needed, just tune its frequency.