homelab-copilot/docs/architecture/00-founding-strategy.md

# Homelab Copilot — Founding Strategy Document

**Status:** Draft v0.2 — Day 1, decisions locked
**Owner:** You (Product Owner / Sole Operator)
**Role of this document:** Source-of-truth for direction before a single line of code is written. Lives at `docs/architecture/00-founding-strategy.md` in the repo.

---

## Decisions Log

| # | Decision | Answer | Recorded as |
|---|---|---|---|
| 1 | Reordering in Section 1 (thin Q&A slice before automation) | **Accepted** | This document, Sections 1 & 5 |
| 2 | Backend language/framework | **Python + FastAPI** | ADR-001 |
| 3 | CI for M0 | **Deferred.** Manual pre-merge checklist for now. Long-term deployment via **Coolify**, connected to Gitea by webhook, auto-pulling/building/deploying on merge to the appropriate branch. Gitea = Git source of truth; Coolify = build & deploy. | ADR-003 |

Decision 3 has real design implications beyond "skip CI for now" — it means the repo and branching strategy need to be Coolify-compatible from the start, even though we're not wiring Coolify up yet. That's threaded through Sections 4, 7, and 8 below.

---

## 0. How to read this document

This is long on purpose — you asked for a real Day 1, and Day 1 at a real company is mostly disagreement, scoping, and paperwork, not code. Ten sections follow, in the order you asked for them. Section 1 is the one I want you to actually argue with me about before we move on — everything downstream depends on it.

---

## 1. Critical Review of the Vision

I like this project. I also think it will fail in a specific, predictable way if we don't correct for it now, so I'm going to be blunt.

### 1.1 The core risk: this is scoped like a 5-person startup's 3-year plan

Twelve agents, an infrastructure-aware LLM, self-improving knowledge, automated deployment, safe execution — each of these is a legitimate multi-month project on its own. Listed together as "the plan," they create a classic failure mode: **infinite runway to the first real user-facing value.** You are the founder, the entire engineering team, and the only user. If Milestone 1 requires the knowledge layer, the agent framework, the execution guard, and three integrations to all be done before you get a useful answer to a single question, you will lose motivation in about six weeks — I've watched this exact pattern play out in homelab and side projects a hundred times, and it's the single most common cause of death for infra-tooling side projects.

**My pushback:** we will not build "the platform" first and features second. We will build the thinnest possible vertical slice that answers *one* real question you actually have this week, ship it, use it, and let the platform grow around real usage. Everything in your brief remains the destination — I'm only fighting the *ordering*.

### 1.2 "n8n is only one component" — good instinct, needs a firm boundary

I agree n8n shouldn't be the brain. But n8n is seductive precisely because it's fast to build in, and fast-to-build-in tools accumulate business logic silently until nobody can tell you where a decision actually gets made. **Rule I want us to adopt from day one:** n8n only ever orchestrates calls to a backend API and external systems. It contains zero domain logic, zero prompts with actual reasoning in them, and zero data transformation beyond wiring inputs to outputs. If a workflow needs an `if` statement more complex than "did the API return an error," that logic belongs in the backend, not in an n8n node. This is the difference between n8n as glue vs. n8n as undocumented shadow-codebase.

### 1.3 LLM-driven infrastructure automation is a real safety problem, not a checkbox

"Read-only by default, destructive requires confirmation" is the right instinct but it's not a design yet — it's a slogan. A model that can see "restart the unhealthy container" and a model that can *decide* to restart a container are different systems with different failure modes (hallucinated container names, prompt injection via docker logs or file content the model reads, cascading restarts). I want us to design a real **policy/execution-guard layer** as its own component from the start (Section 3), not bolt safety on later. This is non-negotiable — I'll refuse to help implement any execution path that skips it.

### 1.4 The local-first claim needs an honest boundary, now

You're using Claude Desktop, which means part of this system talks to Anthropic's cloud API. "Local-first whenever possible" is a good principle, but right now it's ambiguous about what data crosses that boundary. Compose files, container names, and architecture docs are relatively low-sensitivity; secrets, tokens, and credentials are not. I want an explicit, written rule (Section 3.6) about what is allowed to be sent to a cloud model and what must stay local (Ollama-only), rather than discovering the boundary by accident when something sensitive ends up in a prompt.

### 1.5 Improvements I'm proposing to the concept

- **Add a "knowledge freshness" requirement from day one.** A RAG system over stale docs is worse than no RAG system, because it answers confidently and wrong. Every ingested source needs a last-synced timestamp surfaced in every answer.
- **Add an audit log as a first-class product feature, not an implementation detail.** "What did Homelab Copilot do to my infrastructure this week" should be answerable as easily as "why is Gitea unhealthy." This becomes trust infrastructure for the whole project.
- **Treat the MVP as a Q&A system, not an action system.** Actions come after trust is earned by weeks of correct, cited answers.
- **Explicitly deprioritize the 12-agent framework** until we have one agent (Knowledge Agent) proven useful in daily use. Multi-agent orchestration is a solution to a coordination problem you don't have yet.

If you disagree with any of the above, say so before we lock the roadmap — this is the one section worth spending real time on.

---

## 2. System Architecture

Layered, replaceable, boring on purpose. Nothing exotic — this is infrastructure software, not a research project.

```mermaid
flowchart TB
    subgraph Interfaces["Interface Layer"]
        OWU[Open WebUI]
        CD[Claude Desktop / MCP Client]
        FUTURE_UI[Future: Dedicated Web Dashboard]
    end

    subgraph Core["Core Backend (single source of truth)"]
        API[Backend API<br/>domain logic, orchestration]
        GUARD[Execution Guard<br/>policy engine + confirmation]
        AUDIT[Audit & Logging Service]
    end

    subgraph Knowledge["Knowledge Layer"]
        INGEST[Ingestion Service<br/>docs, compose, configs]
        VDB[(Qdrant<br/>vector store)]
        RDB[(PostgreSQL<br/>state, audit, metadata)]
        CACHE[(Redis<br/>cache/queue)]
    end

    subgraph Agents["Agent Layer (grows over time)"]
        KA[Knowledge Agent]
        MA[Monitoring Agent]
        DA[Docker Agent]
        MORE[...future agents]
    end

    subgraph Integrations["Integration Connectors"]
        DOCKER[Docker / Portainer API]
        PROX[Proxmox API]
        PROM[Prometheus / Grafana]
        GITEA[Gitea API]
        LLM_LOCAL[Ollama - local models]
        LLM_CLOUD[Claude API - cloud, non-sensitive only]
    end

    Interfaces --> API
    API --> GUARD
    GUARD --> AUDIT
    API --> Agents
    Agents --> Integrations
    API --> Knowledge
    INGEST --> VDB
    INGEST --> RDB
    API --> RDB
    API --> CACHE
```

### 2.1 Design principles applied

- **API-first:** every capability (query knowledge, get container status, request a restart) exists as a documented backend endpoint before any UI or n8n workflow calls it. n8n and Open WebUI are *clients* of the backend, never the source of logic.
- **Everything replaceable:** Qdrant could become another vector DB, Ollama could become vLLM, Portainer could be dropped — none of that should require touching the agent or backend logic, only the connector.
- **Event-driven where it matters, not everywhere:** container health changes, Grafana alerts, and Gitea commits are naturally event-driven and should flow through a lightweight event bus (Redis streams is enough at this scale — no Kafka). Request/response (asking a question) stays synchronous.
- **Config as code / infra as code:** every environment variable, every agent's allowed actions, every policy rule lives in versioned YAML in the repo, never in a UI-only setting.

---

## 3. Major Components & Responsibilities

| Component | Responsibility | Owns | Does NOT own |
|---|---|---|---|
| **Backend API** | Single source of domain logic; exposes REST/GraphQL endpoints for every capability | Business rules, request validation, orchestration | UI rendering, raw automation execution |
| **Knowledge Ingestion Service** | Pulls docs, compose files, configs, README's, network diagrams into a normalized, chunked, embedded form | Parsing, chunking, embedding, freshness tracking | Answering questions (that's the Knowledge Agent) |
| **Knowledge Agent** | Answers infrastructure questions using retrieved context + cites sources and freshness | RAG query logic, prompt construction | Taking any action |
| **Execution Guard (Policy Engine)** | Every action request — read or write — passes through here first | Allow/deny rules, confirmation workflow, rate limits, blast-radius checks | Deciding *what* the user wants (that's the agent) |
| **Audit & Logging Service** | Immutable record of every question asked, every action proposed, approved, executed, or denied | Structured logs, queryable history | Alerting (separate, later) |
| **Monitoring Agent** | Interprets Prometheus/Grafana signals into plain-language explanations | Correlating alerts to likely causes | Auto-remediation (that's Execution Guard + a future Remediation Agent, v2.0+) |
| **Docker Agent** | Container status, health, logs summarization | Read-only Docker/Portainer queries | Restart/deploy actions directly — must request via Execution Guard |
| **Integration Connectors** | One per external system (Docker, Proxmox, Gitea, Grafana, Prometheus) | Auth, API calls, response normalization | Business logic or interpretation |
| **Documentation Generator** | Produces Markdown + Mermaid docs from the actual current state of the system | Doc generation, keeping docs in sync with reality | Writing product roadmap or ADRs (human-owned) |
| **MCP Server(s)** | Exposes backend capabilities to Claude Desktop / other MCP clients | Tool schemas, request translation | Independent logic — thin passthrough only |

Everything below the double line in "Does NOT own" is deliberate — it's the part of the design that prevents the shadow-logic problem from Section 1.2.

---

## 4. Repository Structure

Design only — nothing is generated yet.

```
homelab-copilot/
├── README.md
├── LICENSE
├── CHANGELOG.md
├── .gitea/
│   ├── ISSUE_TEMPLATE/
│   └── PULL_REQUEST_TEMPLATE.md
├── docs/
│   ├── architecture/
│   │   ├── 00-founding-strategy.md      <- this document
│   │   ├── adr/                          <- Architecture Decision Records
│   │   └── diagrams/
│   ├── roadmap/
│   │   ├── roadmap.md
│   │   └── milestones/
│   ├── api/                              <- generated + hand-written API docs
│   ├── agents/                           <- one doc per agent, responsibility + interface
│   ├── deployment/
│   │   ├── deployment-topology.md        <- which branch deploys where, via what
│   │   └── coolify-setup.md              <- one-time manual Coolify config (webhook, env vars) — written when we wire it up, not in M0
│   └── changelog/
├── backend/
│   ├── src/
│   ├── tests/
│   └── README.md
├── agents/
│   ├── knowledge-agent/
│   ├── monitoring-agent/
│   ├── docker-agent/
│   └── shared/                           <- shared agent contracts/interfaces
├── integrations/
│   ├── docker/
│   ├── proxmox/
│   ├── prometheus-grafana/
│   ├── gitea/
│   └── llm-providers/                    <- ollama + claude connectors
├── workflows/                            <- n8n workflow exports (thin glue only)
├── infrastructure/
│   ├── docker/
│   │   ├── docker-compose.yml            <- base: shared service definitions
│   │   ├── docker-compose.dev.yml        <- local dev overrides (your laptop)
│   │   └── docker-compose.prod.yml       <- overrides Coolify will build/deploy from
│   ├── terraform/                        <- future IaC, if/when needed
│   └── policies/                         <- YAML: execution guard rules
├── examples/
│   └── sample-queries.md
├── scripts/
└── tests/
    ├── integration/
    └── e2e/
```

A few deliberate choices worth flagging:
- **`docs/architecture/adr/`** — I want every non-trivial technical decision (language choice, DB choice, why n8n is glue-only) recorded as a numbered ADR. Cheap now, invaluable in 18 months when you've forgotten why you rejected an alternative.
- **`infrastructure/policies/`** as its own top-level concern, not buried in the backend — the Execution Guard's rules should be reviewable and diff-able independent of code changes.
- **`docker-compose.prod.yml` as its own file, not env-var branching inside one compose file.** Coolify builds/deploys from a specific compose file per app — keeping prod config physically separate means a Coolify webhook can point at exactly one file with no ambiguity about what gets deployed. `docker-compose.dev.yml` stays yours-only, never touched by Coolify.
- **`docs/deployment/` exists now, gets written later.** We're designing the *shape* of Coolify readiness in M0 (Section 8) without spending time actually connecting Coolify, since there's nothing deployable yet. `coolify-setup.md` is a placeholder until there's a real app behind it.
- **`workflows/`** holds only n8n JSON exports, treated as build artifacts, not hand-edited source of truth.

---

## 5. Product Roadmap

Every version below is scoped so that skipping straight to it would be a mistake — each one earns the next.

### MVP — "It can answer one true thing"
- Ingest: Docker Compose files + Markdown docs from one Gitea repo.
- Ask a question in Open WebUI or Claude Desktop, get an answer with citations + freshness timestamp.
- No automation. No agents beyond Knowledge Agent. No write access to anything.
- **Success criterion:** you trust the answer enough to stop opening the compose file yourself.

### v0.1 — "It knows what's running"
- Docker Agent (read-only): container status, health, uptime.
- "What changed yesterday" via Docker events + Gitea commit log correlation.
- Audit log exists and is queryable (even though nothing destructive has happened yet).

### v0.5 — "It can safely touch things"
- Execution Guard is fully designed and implemented (Section 1.3/3).
- First write action enabled end-to-end: restart a named container, always with explicit confirmation.
- Full audit trail of every proposed/approved/denied action.
- **Gate:** we do not add a second write action until this one has run correctly for at least 2 weeks in real use.

### v1.0 — "It's a real platform"
- Monitoring Agent (Grafana/Prometheus interpretation, read-only).
- Documentation Generator producing live architecture docs from actual state.
- Deployment Agent: deploy a new stack from a template, confirmation required.
- MCP server matured enough to be the primary interface from Claude Desktop.

### v2.0 — "It's proactive"
- Alert correlation across Grafana + Docker + Gitea to explain incidents, not just report them.
- Backup Agent, Networking Agent added.
- Suggests (never auto-executes) remediation for known-safe failure patterns.

### v3.0 — "It's an operating system for the homelab"
- Multi-agent coordination for complex requests ("deploy Immich and wire it into the reverse proxy and backup schedule").
- Self-updating knowledge base with confidence scoring.
- Full self-service: propose infra changes as PRs against your IaC, human merges.

### Stretch / Future / Wishlist / Technical Debt
Tracked live in `docs/roadmap/roadmap.md`, not fixed here — this is a living backlog, not a promise.

---

## 6. Development Methodology

- **Milestone-driven, not sprint-driven.** You're a team of one; artificial two-week sprints add ceremony without benefit. Each milestone has explicit acceptance criteria (Section 9) and we do not start the next one until the current one is *done*, per your instruction.
- **Definition of Done** for every milestone: code merged to `main`, tests passing, docs updated in the same PR, ADR written if a non-trivial decision was made, CHANGELOG updated.
- **No feature work without a doc update in the same commit/PR.** This is how documentation stays non-optional rather than a follow-up task that never happens.
- **Weekly written retro (even solo).** Two paragraphs in `docs/changelog/` — what shipped, what you learned, what you're deprioritizing. This is what lets the project survive gaps of weeks or months between working sessions, which will happen.

---

## 7. Git Workflow

- **Branching:** trunk-based with short-lived feature branches. `main` is always deployable. No long-running `develop` branch — with a single contributor, git-flow's ceremony creates merge pain without the coordination benefit it exists for.
  - Branch naming: `feat/knowledge-agent-ingest`, `fix/qdrant-timeout`, `docs/adr-004-vector-db`.
  - **Coolify implication:** once Coolify is wired up, merging to `main` *is* a production deploy — there's no CI gate in between yet, only the manual pre-merge checklist (Section 8). That raises the bar on what "ready to merge" means: the checklist has to actually catch problems, not just be a formality. I'd rather we feel that pressure honestly than fake safety with a checklist nobody reads.
  - **Staging branch — deliberately not adding one yet.** A `staging` branch mapped to a second Coolify app would give you a pre-prod environment, but it's another homelab service to run, another webhook to maintain, and another thing that can drift from `main`. For a Q&A-only MVP with no destructive actions, the blast radius of a bad deploy is low (worst case: wrong answers, not broken infra). **Revisit this at v0.5**, when the Execution Guard starts letting the system touch real containers — that's when a staging environment starts paying for itself.
- **Commits:** Conventional Commits (`feat:`, `fix:`, `docs:`, `chore:`, `refactor:`, `test:`) — this also gives us free changelog generation later.
- **Pull Requests:** required even solo, as a forcing function for the "docs updated in the same PR" rule and to keep a reviewable history. PR template asks: what changed, why, docs updated?, tested how?
- **Issue templates:** `bug_report.md`, `feature_request.md`, `epic.md`, `agent_proposal.md` (new agent = its own template, since it needs responsibility + interface defined before code).
- **Labels:** `type:feature` / `type:bug` / `type:docs` / `type:debt`; `area:backend` / `area:agent` / `area:integration` / `area:infra`; `priority:p0-p3`; `status:blocked`.
- **Milestones in Gitea:** one per roadmap version (MVP, v0.1, v0.5, v1.0...), issues assigned accordingly.
- **Releases & SemVer:** tags follow `v0.1.0` etc. Pre-1.0, minor bumps can include breaking changes (standard SemVer allowance); post-1.0, strict SemVer.

---

## 8. First Milestone: M0 — Foundation

We are not writing feature code yet. M0 is scaffolding and decisions, because every milestone after this depends on choices made here.

**Goals**
- Repository created in Gitea with the structure in Section 4.
- Core technology decisions made and recorded as ADRs:
  - Backend language/framework (recommend: Python + FastAPI, given your AI-stack is already Python-centric via Ollama/Qdrant tooling — but this is your call, and worth its own ADR debate).
  - Sync strategy for docs → Qdrant (polling vs. Gitea webhook — I'd push for webhook from day one, since it's not more work now and saves a rewrite later).
- `docker-compose.dev.yml` for the platform's own local dev environment (Postgres, Qdrant, Redis) — infra for the tool, not the homelab automation itself.
- **No CI pipeline in M0** — deferred by decision. Instead: a written **manual pre-merge checklist** (`docs/architecture/pre-merge-checklist.md`) that you actually run through before every merge to `main`. This matters more than usual here, since merges to `main` will eventually trigger a real Coolify deploy with no automated gate in front of them.
- `docker-compose.prod.yml` scaffolded (per Section 4) even though Coolify isn't connected yet — the file should exist and be valid, just not wired to anything.
- `docs/deployment/deployment-topology.md` drafted: states plainly that `main` → Coolify → production, no staging environment yet, and why (Section 7). This is a design artifact, not a Coolify setup task.
- `docs/architecture/adr/003-deployment-strategy.md` recorded: Coolify + Gitea webhook for build/deploy, CI intentionally deferred, rationale.
- `docs/roadmap/roadmap.md` and `docs/architecture/adr/` populated with this document's decisions (ADR-001 backend framework, ADR-002 sync strategy, ADR-003 deployment strategy).

**Required technologies:** Gitea repo, Python/FastAPI, Docker Compose, Postgres, Qdrant, Redis. (Coolify itself is *not* required for M0 — there's nothing deployable yet.)

**Implementation:** none — this milestone produces scaffolding, config, and decisions, not features.

**Testing:** none automated. The manual pre-merge checklist is exercised once, on this milestone's own PR, to prove it's usable and not just theoretical.

**Documentation:** README with project overview and "how to run this locally"; 3 ADRs written (backend framework, sync strategy, deployment strategy); deployment topology doc drafted.

**Acceptance criteria**
- [ ] Repo exists with the agreed structure
- [ ] `docker-compose -f infrastructure/docker/docker-compose.yml -f infrastructure/docker/docker-compose.dev.yml up` brings up Postgres + Qdrant + Redis locally with no errors
- [ ] `docker-compose.prod.yml` exists and is valid (`docker-compose config` succeeds), even though nothing deploys from it yet
- [ ] 3 ADRs merged: ADR-001 (backend framework), ADR-002 (sync strategy), ADR-003 (deployment strategy)
- [ ] Manual pre-merge checklist doc exists and was actually used for this milestone's PR
- [ ] `deployment-topology.md` drafted, stating current state (no staging, no live Coolify connection yet)
- [ ] Roadmap doc committed

We do not start M1 (MVP Knowledge Agent) until every box above is checked.

---

## 9. Initial Gitea Epics & Issues (pre-code)

### Epics

- **EPIC-1: Foundation & Tooling** (M0)
- **EPIC-2: Knowledge Layer** (MVP)
- **EPIC-3: Execution Guard & Safety** (v0.5 — designed early, built when needed)
- **EPIC-4: Docker Observability** (v0.1)
- **EPIC-5: Agent Framework** (ongoing, starts thin in MVP)

### Issues to create before any code (all under EPIC-1 / Milestone M0)

1. **[docs] Write ADR-001: Backend language & framework choice (Python + FastAPI — decided, needs write-up)**
   `area:backend` `type:docs` `priority:p0`
2. **[docs] Write ADR-002: Docs→Qdrant sync strategy (webhook vs polling)**
   `area:backend` `type:docs` `priority:p0`
3. **[docs] Write ADR-003: Deployment strategy — Coolify + Gitea webhook, CI deferred (decided, needs write-up)**
   `area:infra` `type:docs` `priority:p0`
4. **[infra] Scaffold repository structure per Section 4**
   `area:infra` `type:feature` `priority:p0`
5. **[infra] docker-compose.yml + docker-compose.dev.yml for platform dev environment (Postgres, Qdrant, Redis)**
   `area:infra` `type:feature` `priority:p0`
6. **[infra] Scaffold docker-compose.prod.yml (valid but not yet connected to Coolify)**
   `area:infra` `type:feature` `priority:p1`
7. **[docs] Write manual pre-merge checklist (`docs/architecture/pre-merge-checklist.md`)** — replaces CI for M0/M1
   `area:infra` `type:docs` `priority:p0`
8. **[docs] Draft `docs/deployment/deployment-topology.md`** — states current state: `main` → Coolify → production, no staging yet, why
   `area:infra` `type:docs` `priority:p1`
9. **[docs] Write initial roadmap.md from Section 5**
   `area:backend` `type:docs` `priority:p0`
10. **[epic-proposal] Define Knowledge Agent interface & responsibility doc** (prep for MVP, no code)
    `area:agent` `type:docs` `priority:p1`
11. **[epic-proposal] Define Execution Guard policy schema (YAML shape) — design only**
    `area:infra` `type:docs` `priority:p1`

Nothing here writes application code. That's deliberate — M0's entire job is to make M1 unambiguous.

---

## 10. Status: M0 is unblocked

All three founding decisions are locked (Decisions Log, top of document). M0 is now fully specified in Section 8 and ready to start:

- Repo scaffold (Section 4, including the Coolify-aware `docker-compose.prod.yml` and `docs/deployment/` layout)
- ADR-001, ADR-002, ADR-003 written
- Manual pre-merge checklist drafted and used once, for real
- `deployment-topology.md` drafted (no staging yet — revisit at v0.5)

One thing I want flagged explicitly rather than quietly assumed: **Coolify is not being connected in M0.** We're shaping the repo so it's ready for that wiring later, but actually standing up the webhook and first deploy happens once there's something real to deploy — most naturally at the end of M1 (MVP Knowledge Agent), when `main` first contains a working service. I'll call that out again when we get there so it doesn't slip through as "already done."