Set up an AWS Cost Optimization data export
Create an AWS Cost Optimization recommendations data export step by step in the Billing console, so your Emnode dashboard can surface idle resources, right-sizing and cheaper equivalents.
Last updated
Your Emnode dashboard finds savings, idle resources, over-provisioned instances and cheaper equivalents, partly from AWS's own Cost Optimization recommendations. This guide creates the data export that delivers those recommendations to an S3 bucket the dashboard can read.
It is the same Data Exports flow as the FOCUS export, with one difference: you pick the Cost optimization recommendations data table instead of FOCUS. If you have already created your FOCUS export, this will feel familiar, and you can even deliver both to the same bucket under different prefixes.
You need access to the AWS account you want optimised, with permission to use Billing and Cost Management and to create an S3 bucket. The whole process takes about ten minutes and changes none of your workloads. For the richest recommendations, make sure Cost Optimization Hub is enabled on the account (it is free to turn on).
Create the export
Work through the seven steps below. As you scroll each console screen into view it comes into focus; the cyan highlights mark the control to use, and the amber highlights mark a value to write down.
Step 1 of 7
Open Data Exports and choose Create
Sign in to the AWS account you want optimised. In the top search bar, type Billing, open Billing and Cost Management, and under Cost and Usage Analysis choose Data Exports.
The page lists any exports the account already has. Choose Create to start a new one.
Data Exports
A data export tool that enables you to create customised exports from multiple AWS cost management and usage datasets.
Exports and dashboards
Find exports ActionsChoose Create to begin
Step 2 of 7
Name it and keep Standard data export
Under Export type, leave Standard data export selected. Give the export a name you will recognise later, using lower-case letters, numbers and hyphens.
Write this name down: you will confirm it when you connect the dashboard.
Create export
Export details
Choose an export type and provide a name for your export.
Export type
✎ Record the export name
Step 3 of 7
Choose the Cost optimization recommendations table
Under Data table content settings, choose Cost optimization recommendations. This is the table the dashboard reads to find savings, like idle resources and cheaper equivalents.
Leave Include all recommendations on and the Column selection at its default (all columns). You do not need to touch the SQL or filters.
Data table content settings
Start creating your export by choosing a data table. We then set the columns to export and the SQL query.
Select Cost optimization recommendations
Data table configurations
Step 4 of 7
Set the delivery options
Under Data export delivery options, choose Parquet - Parquet for the compression and file format. The refresh cadence is Daily.
For File versioning, choose Overwrite existing data export file so each refresh keeps one current dataset rather than piling up new files.
Data export delivery options
Define your delivery options. Select the file format and overwrite rules.
Compression type and file format
Choose Parquet - Parquet
File versioning Overwrite keeps a single, always-current dataset. The dashboard re-reads it each day.
Step 5 of 7
Create the S3 bucket
Choose Configure to set where the export is delivered. In Configure S3 bucket, choose Create a bucket, enter a globally unique bucket name, and select the Region.
Record the bucket name and the Region. AWS fills in the bucket policy that lets the export service write to it, so you can leave it as-is and choose Create bucket.
Configure S3 bucket
Create a bucket or select an existing bucket
Data Exports delivers your export to the S3 bucket that you specify.
✎ Record the bucket name and Region
Bucket policy (added automatically)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EnableAWSDataExportsToWriteToS3",
"Effect": "Allow",
"Principal": {
"Service": [
"billingreports.amazonaws.com",
"bcm-data-exports.amazonaws.com"
]
},
"Action": ["s3:PutObject", "s3:GetBucketPolicy"],
"Resource": [
"arn:aws:s3:::acme-cost-optimization-exports",
"arn:aws:s3:::acme-cost-optimization-exports/*"
]
}
]
}Step 6 of 7
Set the storage path prefix
Back in Data export storage settings, keep This account selected and confirm the bucket. Set an S3 path prefix such as recommendations: this is the folder inside the bucket the export writes into.
Record the prefix exactly as you type it, then choose Create.
Data export storage settings
Define where you want your export to be stored. Amazon S3 is object storage built to store and retrieve data.
Storage account
✎ Record the path prefix
Step 7 of 7
Submit and confirm
AWS validates and submits the export. You return to the Data Exports list with a confirmation, and your new export shows a Healthy status.
The first recommendation files land under your bucket and prefix within about 24 hours, then refresh every day. Keep the details you recorded for when you connect the dashboard.
Data Exports
What to record
You will hand these to the dashboard when you connect the account. Note the values you actually used, not the examples. You can deliver this export to the same bucket as your FOCUS export under a different prefix, if you prefer.
| Detail | Example | Why it matters |
|---|---|---|
| AWS account ID | 123456789012 | Identifies the account the export belongs to. |
| Export name | acme-cost-recommendations | Confirms which export feeds your recommendations. |
| S3 bucket name | acme-cost-optimization-exports | The bucket the dashboard reads your exports from. |
| Region | US East (N. Virginia) | The region your bucket is in. |
| S3 path prefix | recommendations | The path prefix within that bucket. |
What happens next
Once the export shows a Healthy status and the first files have landed (within about a day), you are ready to give the dashboard read-only access to the bucket. That is the next guide, Grant the dashboard access to your data. Keep the details you recorded to hand: you will enter them when you connect the account.
Store the bucket name, Region, path prefix and export name somewhere you can find them again. If you ever recreate the export, make sure these still match what the dashboard expects, or it will not find your recommendations.