Context
A decision-serving system used PostgreSQL as both the system of record and the compute engine: scoring logic ran as SQL, and the application tier mainly shuttled requests to and from the database.
Problem
Core computations were pushed into PostgreSQL while the application tier used only a small fraction of available CPU. Scaling the existing design increased infrastructure cost without fixing the execution model.
Constraints
The database had to remain the durable, authoritative store — this was not a rewrite that could relax consistency guarantees. The system was live, so the migration had to preserve correctness throughout, not just at a finish line.
Approach
- moved the critical computation into an in-memory execution layer,
- used parallel CPU-bound processing for scoring workloads,
- kept the database for durable state rather than treating it as a compute engine,
- measured the machine as a whole rather than tuning a single query in isolation.
Trade-offs
Moving computation off the database traded SQL's transactional simplicity for application-level correctness responsibility — consistency between the in-memory execution layer and the durable store had to be maintained explicitly rather than left to the database engine.
Measured outcome
Processing dropped from minutes to roughly one or two seconds, CPU resources were used effectively and infrastructure expenditure fell from approximately $70k to $7k.