CASE STUDY / 02
Find the limit before the cutover.
Local load tests exposed connection-budget and job-capacity limits. Transaction pooling reduced database connection demand; a separate restore drill verified point-in-time recovery by checking the data.
Local infrastructure rehearsal
Rails / PostgreSQL / PgBouncer / Ansible · infrastructure rehearsal
The problem
More worker processes could improve job throughput while exhausting a database role’s connection budget. Backups also needed a demonstrated recovery path before an infrastructure transition.
The implementation
I implemented local infrastructure and test tooling across Ansible provisioning, k6 scenarios, a mock LLM, infrastructure monitoring, and PgBouncer transaction pooling. The tests examined job concurrency, slow and failing providers, backups under load, and a pooler restart.
What the measurements showed
At 100 simulated users, a direct-connection run peaked at 70 application-role database connections; a pooled run peaked at 18. Both recorded zero HTTP request failures. These were separate lab runs, not a controlled proof of a production latency improvement.
At higher load, connections stayed bounded while CPU load and job queues grew. A pooler restart caused 7 failed HTTP requests out of 5,756 and one reply timeout. At 150 users, zero HTTP failures still accompanied two reply timeouts: successful HTTP responses did not establish successful completion.
Proving recovery
An encrypted pgBackRest backup and WAL replay restored an earlier 1,000-row batch and excluded a later batch on a fresh VM. The original repository stayed separate from the restored timeline. A 22 MB database proves the procedure, not a production recovery-time objective.
The boundaries
Local VMs, synthetic admin users, a mock model, and polling instead of the real streaming client. Cloud rollout, learner paths, production capacity, production-size recovery, and automatic failover are not established by these tests.