Add CloudWatch alarms to RDS instances

Amazon RDS writes metrics — CPU, storage, connections, latency — to CloudWatch automatically and at no incremental cost. A CloudWatch alarm on one of those metrics costs roughly $0.10 per alarm per month. The entire standard set of five alarms on a production database costs less than a dollar a month. Despite that, RDS creates zero alarms by default — the platform gives you the data, the alerting is entirely opt-in.

The finance relevance is what 'unalarmed' costs when something goes wrong. Without alarms the first signal of a database problem is a customer complaint or application error spike — both of which arrive after the incident has already started, when it is too late for preventive action. Detection latency has a direct dollar cost: every minute of customer-visible downtime that a timely alarm would have prevented is revenue and reputation lost for roughly $0.10/month of avoided spend.

The cloudwatchOps check COV-002 flags this gap at CRITICAL severity. From a finance perspective the finding is almost never a cost reduction opportunity — CloudWatch alarms are too cheap to optimise. It is a risk-and-efficiency opportunity: alarms shorten mean time to detect, shorten mean time to respond, and reduce the probability of a costly extended outage. The cost of not having them is paid in incident hours, not cloud spend.

Caitlin is the FinOps partner reviewing the monthly operations scorecard. The cloudwatchOps section shows 47 CRITICAL COV-002 findings — 47 RDS instances with zero CloudWatch alarms. Her first question is not 'how much does it cost to fix?' — CloudWatch alarms are less than a dollar each per month, so the remediation cost is negligible. Her question is 'what is the expected cost of not fixing this?'

She works with the platform team to split the 47 into tiers: 31 are production or production-adjacent databases behind customer-facing services; 16 are development and test instances. For the 31 production databases, she prices the gap. The team's last major RDS incident — a disk-full event six months ago — caused four hours of partial outage; a FreeStorageSpace alarm would have given two days of lead time. At roughly $30k/hour in lost transaction revenue, that single preventable incident cost more than the total annual alarm spend on all 47 databases combined.

Caitlin's recommendation: approve the full alarm rollout immediately, budget the alarm spend as a fixed reliability line item under the operations budget, and flag unalarmed production RDS instances on the monthly risk register until coverage is complete. The math is clear enough that she doesn't need a second meeting to get the decision.

The cost of not alarming a production RDS database has two components: the cost of the incidents that happen and the cost of the incidents that could have been prevented. Both are substantially larger than the cost of the alarms themselves. A full standard alarm set for one database costs roughly $0.50–$1.00 per month. A single major incident attributable to late detection — a disk-full that flipped the database read-only while engineers chased the application layer for an hour before diagnosing the real cause — easily runs to tens of thousands of dollars in incident labour, lost revenue, and customer remediation.

For finance, the disk-space alarm is the clearest case to model. FreeStorageSpace declines predictably over days or weeks, so an alarm at 20% typically gives 48–72 hours of lead time. The cost of the alarm firing and someone extending storage is engineer-hours. The cost of missing it is a STORAGE_FULL transition that makes the database read-only, creates a partial outage that looks like an application bug, generates an all-hands incident, and — depending on how the application handles write failures — may require manual data reconciliation afterward. The mean time to recovery on the prevented path is minutes; on the undetected path it is hours.

Alarm coverage also has a direct compliance cost. SOC 2 Type II, ISO 27001, and PCI DSS all expect organisations to demonstrate a documented monitoring posture — alarms wired to a paging destination with a response SLA — for systems holding regulated data. An unalarmed production database that holds payment or customer records is a control gap that an auditor will flag and that will require remediation evidence, which is additional cost on top of the original exposure. Addressing COV-002 now removes this as a future audit finding.

The right budget framing: alarm coverage is not an optional operational nicety — it is a component of the cost of running a production database responsibly. Add the monthly alarm spend to the database line item as a standard overhead, track the percentage of production RDS instances with full coverage on the operations scorecard, and treat any unalarmed production database as an open risk item with a dollar estimate attached until it is remediated.

AWS provides detailed database health data but creates zero automatic alerts on it. Without explicit configuration, the first sign of a database problem is typically a customer complaint or a monitoring alert from the application layer — by which point the incident has already started. CloudWatch alarms are the mechanism that makes the database report its own problems first.

The check COV-002 flags this at CRITICAL severity because silent database failures are the hardest kind to respond to: there is no early-warning signal, responders arrive late, and the window for a simple preventive action has already closed. This is a monitoring posture question, not a cost question — setting up the standard alarm set costs under a dollar a month per database. The leadership issue is whether the organisation has decided that production databases must alert on their own problems, and enforces that as policy rather than leaving it to individual engineers.

Before the CTO at one company set alarm coverage as a tracked metric, the answer to 'how do we find out about database problems?' was effectively 'a customer tells us or the app goes red.' The mean time to detect a database incident was measured in minutes to hours, and the first responders spent most of the early recovery window figuring out what layer the problem was even at.

After defining a policy — every production RDS instance must carry the standard five alarms, with any exception documented — the character of database incidents changed. The on-call engineer now arrives to a page that names the problem: 'FreeStorageSpace LOW on db-payments-1' is a different runbook from 'CPU HIGH', and both are faster to resolve than 'the app is slow, good luck.' Mean time to detect dropped from tens of minutes to under five.

The CTO's one-line question at each review became: 'What percentage of our production RDS instances have full alarm coverage, and what are the documented exceptions?' A consistent answer above 95% means the monitoring posture is governed. That's the signal that matters — not whether a specific incident happened, but whether the infrastructure is set up to tell the team before a customer does.

Running a production database without alarms is equivalent to deciding that the organisation will hear about database problems from customers before it hears about them internally. That is not a technical default — it is an implicit risk decision that most organisations would not make deliberately. COV-002 makes it explicit: these are the systems currently configured to fail silently.

The detection-latency gap has a compounding effect on incident cost. Early-warning alarms don't just shorten time to detect — they change the character of the response. An engineer who gets a 'FreeStorageSpace LOW' alert two days in advance has a scheduled task; an engineer who gets a 'the API is timing out' call at 2am has an investigation. The latter produces more total downtime, more engineer hours, more customer communication, and a much higher probability of a compliance or contractual SLA breach.

On the audit and accountability side, regulators and enterprise customers increasingly ask for documented evidence of monitoring controls — not dashboards, but alarms with defined thresholds, named destinations, and response SLAs. The absence of alarms on a production database holding regulated data is an audit finding regardless of whether an incident has occurred. Leadership owns the answer to 'are our production databases configured to tell us before customers notice?', and right now the honest answer, for every instance flagged by COV-002, is no.