dev-llc

#The problem

Binary Prosperity LLC is my web design and hosting business. My home-lab provisioner, dlt, could stand up a site on my own servers, but client sites need something different. A client shouldn't share a box with my experiments, or with other clients. Setting up a server by hand for each new client means a dozen manual steps across three providers, and each one is a chance to forget something.

I wanted one button that takes a client from "domain purchased" to "running on its own server behind Cloudflare". It also had to be safe to press again when something fails halfway.

#Architecture

dev-llc is a Laravel app with a queue worker, running in Docker Compose on my home dev host. It talks to three outside systems: Vultr for servers, Cloudflare for DNS and certificates, and GitHub's container registry (GHCR) for images. It reaches each client's VPS over SSH and SFTP, using that client's own key.

The dev preview runs first on purpose: the client can see their site at dev.{domain} before any production infrastructure exists. Deploying is a separate, repeatable action:

Each step's status and log is stored, so the provisioning page shows exactly where a run stopped and why.

#Key decisions

One VPS per client. If one client's site is compromised or crashes, the damage stays on that box. Billing is clean, and offboarding means destroying one server. For fewer than ten low-traffic sites, Kubernetes would be pure overhead.

One SSH keypair per client. Revoking access for one client never touches the others. Private keys are stored encrypted, and SSH/SFTP run entirely in memory through phpseclib, so a key never lands on disk.

Every step guarded by a stored ID. Each cloud resource's ID is saved the moment it's created, and checked before trying again. DNS writes are upserts. The Origin CA step skips if a certificate already exists, because re-issuing would mint a duplicate. The pipeline is deliberately sequential: it runs a few times a year, so one readable log beats saving a minute.

Least-privilege tokens. The DNS token and the Origin CA token are separate. The GHCR push token never leaves the dev host, and client servers only get a pull token. Account-wide tokens live in .env. Per-client secrets (SSH keys, TLS keys, database passwords, client-owned Cloudflare tokens) are encrypted at rest. The design doc points out that this makes APP_KEY a master key with its own backup procedure.

Three ways a client can own their domain. I can be the registrar. The client can be the registrar with the zone in my Cloudflare account. Or the zone can stay in the client's own account, used through a scoped, encrypted per-client token. Each model maps to a different credential path.

Cloudflare Origin CA instead of Let's Encrypt on the VPS. Traffic always goes through Cloudflare, so the origin certificate only needs Cloudflare to trust it. A 15-year certificate means no renewal job on every box.

Teardown stays manual, for now. Destroying a client's infrastructure is rare and can't be undone. The offboarding runbook covers it step by step instead of a button.

#What broke and how I fixed it

All of these happened against live infrastructure, and each fix is commented in the code.

The certificate nobody could read. I passed SFTP's chmod the string '0600'. It was treated as decimal 600 (octal 1130), so the TLS cert and key landed on a live server unreadable by anyone. I found it with ls -la over SSH, and the method now requires an integer literal.

Deploys cut off mid-pull. phpseclib's default 10-second inactivity timeout killed docker compose pull as soon as image layers started streaming, and the exit status came back empty. The fix was explicit timeouts sized to the actual work.

I had the Origin CA backwards. The first version expected Cloudflare to send back a private key. The live API answered "CSR parsed as empty", because a CA signs your certificate request. I wrote a local key and CSR generator (verified with openssl req -verify), including a workaround to get phpseclib to attach the SAN extension. Partway through, Cloudflare deprecated the old Origin CA key header, so the code moved to a scoped bearer token.

A hostname too deep for the edge certificate. dev.llc.binaryprosperity.com is two levels deep, Cloudflare's Universal SSL doesn't cover it, and the TLS handshake failed. Turning off Cloudflare's proxy would have "fixed" it, and I made a rule never to do that. Dev hostnames now use a hyphen at the same depth (dev-llc.), chosen automatically before any real client could hit the problem.

"DNS resolves but nothing serves." While renaming the app, I pointed DNS at the dev host, but public traffic actually lands on a separate reverse-proxy VM. That's why the first pipeline step now configures DNS and the proxy vhost and TLS together, and why the two-tier layout is written down.

#Results

• My company's own site, binaryprosperity.com, runs on a Vultr VPS built to the same spec.
• ~90 Pest tests fake every external API with Http::fake() and Process::fake(), including failure paths. One test asserts that a retry never duplicates a Vultr resource.
• Three GitHub Actions workflows: tests, Pint lint, and composer audit on every push plus a weekly scheduled audit. The scheduled run catches newly disclosed CVEs in a lockfile that hasn't changed.
• A 315-line design doc and onboarding/offboarding runbooks. The offboarding runbook goes: final snapshot with 30-day retention → revoke the SSH key → DNS cleanup per ownership model → revoke the Origin CA cert → destroy the VPS → purge secrets but keep the record for invoicing. Writing them made me think through the irreversible steps before I need them.

Around it are a handful of in-house Laravel packages with their own CI, and a combined nginx + PHP-FPM production image for my company's site. Both are still work in progress.

I'm now building deprovisioning with an approval gate: each offboarding step automated in reverse, with a manual confirmation before anything destructive runs, so test sites can be spun up and removed without a trace.