V0.1 · Architecture & service documentation

IMPLEMENTED, LEARNED, STILL TO TEST

The lab keeps changing.
The evidence should keep up.

This page separates existing practices from planned experiments and captures lessons that explain the current choices.

DOCUMENTED IMPLEMENTATION

What the current design includes.

A repeatable Linux baseline

Version-controlled Ansible, executed through Semaphore with baseline and task operations separated.

Production monitoring and central logs

Zabbix as the primary monitoring layer; Elastic/Fleet promoted after UAT with the documented rollout completed.

Dynamic dashboards

Groups, tags and discovery replace fixed lists where the view can support dynamic membership.

Service-oriented documentation

Application state, databases, storage and recovery dependencies are documented together.

A successful response can still be the wrong service.

Endpoint reconciliation found an intended application name serving another interface. The lesson is concrete: status-only checks can miss a configuration error. Probe the intended URL and application identity, not merely whether a web server answers.

HA depends on capacity and complete configuration.

A guest can fail to start because one configured device is missing even when its main data disk exists. Recovery needs to identify the failed reference precisely instead of assuming data loss or making broad changes. Eligible hosts and available resources also constrain failover.

Collecting every historical log can be the wrong default.

Initial Docker collection imported substantial old history and created an ingestion burst. The resulting design uses an explicit onboarding cutoff and a separate retention policy. Collection scope, resource limits and the investigation question should be decided before importing a backlog.

A dashboard tool is not always the right reader.

A dashboard experiment was rejected in favour of a dedicated FreshRSS reader with a compact Homepage summary. Separating those responsibilities made the intended workflow clearer than asking one dashboard to do everything.

Previous Kubernetes experience is context, not a diagnosis.

The lab previously used k3s and Argo CD before returning to Docker. Reported storage performance concerns are a reason to design a better experiment; they are not proof that NFS was the root cause. Talos would change operating-system management, but would not by itself solve storage performance.

PLANNED OR UNDER EVALUATION

What is not implemented by this roadmap.

PLANNED / EVALUATION

Regular restore testing

Requirement confirmed; define the calendar, cover each layer, execute tests and retain functional evidence.

PLANNED / EVALUATION

Kubernetes / Talos evaluation

An isolated pilot is under consideration. No migration or deployment is claimed.

PLANNED / EVALUATION

Storage for a Kubernetes pilot

Evaluate local storage with replication rather than assuming the NAS will hold active workload volumes. Longhorn is a candidate; alternatives and compatibility require testing.

PLANNED / EVALUATION

Public evidence and case studies

Add reviewed screenshots and deeper examples while keeping the operational inventory private.

How I would judge a Kubernetes pilot.

Start isolated and with appropriate test workloads. Exercise deployment and revert from Git, node failure, upgrades, storage consistency and recovery. Distinguish active application replicas from restarting an instance elsewhere, and distinguish volume replicas from application replicas.

Measure the behaviour of the complete workload and check where copies physically live. A one-VM-per-hypervisor idea is a hypothesis to evaluate, not an approved or dimensioned implementation.

What stays true as the tools change.

Configuration should be reviewable, current state should be observed, recovery should be demonstrated and public content should be selected for its audience. New tooling is useful when it improves those behaviours rather than merely adding another product to the diagram.