Practice04
Servers and infrastructure.
TwoRing's voice engine, the Knee Ledger's training loops, the quantum instrument, AllSpark's dashboards — all of it runs on servers we own, rack, and operate ourselves. The infrastructure is not a footnote to the projects; it is one of them.
Why it matters
Production discipline, at personal scale.
It is easy to preach runbooks, monitoring, and tested restores when someone else carries the pager. We run our own environment to the standard we recommend, at the scale of a small company: real products with real customers depend on this hardware, so “it is just the homelab” is never an excuse.
When we design an integration's error handling or insist on a rollback plan before a cutover, we are describing how we run our own systems on an ordinary Tuesday. Operating this environment is the difference between recommending practices and living with them.
How it is run
Six habits, no exceptions.
Virtualized, not improvised
A Proxmox VE environment carves the hardware into isolated containers and virtual machines — one workload per guest, so a misbehaving experiment cannot take a product down with it.
AI inference stays home
Local LLM inference via Ollama and local Whisper speech-to-text run alongside the cloud APIs they back up. Every AI-dependent system in the lab has an answer to "what if the provider is down?" — because the fallback is in the rack.
Services, not sessions
Long-running systems are managed as proper system services with restart policies and health checks, provisioned by scripted, repeatable automation rather than by hand-typed history.
Secrets live in one place
Credentials are never committed, never pasted into configs: they are fetched at runtime from a dedicated secrets manager under least-privilege, read-only automation tokens. Repositories hold the map, never the keys.
Restores, not just backups
Backup jobs are boring; restore documents are the asset. Every important system has a written path back from bare metal — the same runbook standard we hand clients at go-live.
Watched while we sleep
Dashboards, reconciliation jobs, and push alerts to a phone mean the lab reports its own failures. A system that requires someone to notice something is wrong is a system that is already down.
Next step
The pager is ours, not yours.
If you are weighing whether to self-host, migrate, or hand a system back to your own team, we have made those calls with our own money and our own sleep on the line.
We reply within one business day