From Logs To Signal
Databases can log queries that exceed a slow query threshold, say 500 milliseconds. Raw, this is a noisy stream. Useful alerting turns it into action by answering two questions: which queries are slow, and are they getting worse.
Normalize Then Rank
The key trick is to normalize queries by stripping literal values, so that the same statement with different parameters collapses into one query shape or fingerprint. Then you can:
- Rank shapes by total time, frequency times average duration, to find what actually loads the database.
- Track each shape's latency over time to catch regressions after a deploy.
- Surface newly slow shapes that crossed the budget recently.
A single rare slow query matters less than a fast query run a million times that just got slower.
Alert On Impact, Not Noise
Avoid alerting on every slow line. Instead alert when a shape's total time or p95 rises past a budget or jumps sharply versus baseline. That points an on call engineer at one fingerprint to fix, often a missing index or a plan change, rather than a wall of logs.
Key idea
Slow query alerting normalizes queries into shapes, ranks them by total time, and alerts when a shape exceeds its latency budget or regresses, turning noisy logs into a focused fix list.