Institutional Memory
Patterns learned across all debates
Synchronous writes on hot database paths
Adding synchronous write operations to high-throughput database paths without async decoupling. This pattern has caused latency spikes in 14 prior decisions when write volume exceeded initial projections.
Avoidance Hint
Always decouple new writes from hot paths using async queues or WAL-based approaches. Validate with load tests before production.
Mitigation
Implement write-behind cache or async event-driven write path. Add circuit breaker with fallback to async batch mode.
Unbounded cache growth in long-running services
Services that cache data without TTL or max-size bounds. Memory usage grows monotonically until OOM. Observed in 8 prior decisions, typically surfacing 2-4 weeks after deployment.
Avoidance Hint
Every cache must have both a TTL and a max entry count. No exceptions, even for 'small' caches.
Mitigation
Add LRU eviction policy, configurable TTL, and memory usage alerting. Use bounded data structures.
Third-party API calls inside database transactions
Making external API calls while holding a database transaction lock. Connection pool exhaustion occurs when the third party has latency spikes. Seen in 21 prior decisions.
Avoidance Hint
Never call external services inside a transaction boundary. Use saga pattern or separate the external call from the DB write.
Mitigation
Refactor to saga/outbox pattern. Add transaction timeout as safety net. Monitor connection pool utilization.
Schema migration without backward compatibility
Deploying schema changes that break the currently running application version. Causes errors during rolling deployments when old code encounters new schema.
Avoidance Hint
All schema migrations must be backward-compatible with the previous application version. Use expand-contract pattern.
Mitigation
Deploy schema change first (expand), deploy new code, then clean up old columns (contract) in a separate release.