AWS RDS/Aurora: Evaluation setup—Local agent installation

This guide helps you install the dbtune-agent on your laptop or desktop for a quick evaluation.

Prerequisite

An IAM user authorized to configure policies and manage access keys.
You must have an existing RDS or Aurora for PostgreSQL instance.
The rds_superuser role is required to create an extension pg_stat_statements.
Configure the Access key ID and Secret access key on your local machine using aws configure. Follow these instructions to learn how to configure them. If you have already set this up, you can find these values in ~/.aws/credentials.
Make sure that you have a custom parameter group attached to your RDS/Aurora instance. DBtune will modify the parameter group that is assigned to the instance during tuning. We advise a separate parameter group per database instance so as to not impact the server parameters on other database instances that are not the target for the tuning. You must manually update this setting by selecting modify your database instance in the AWS console when assigning a new parameter group.

Create a new parameter group if one doesn't exist as mentioned here.

Note:
Attaching the custom parameter group requires a manual reboot of the database.

Apply that parameter group to the database as mentioned here.
Ensure that port 5432 is allowed in your RDS/Aurora instance security group's inbound rules so you can connect to the RDS/Aurora instance from your local IP. Refer to this guide.
Ensure Docker desktop is installed if you want to run the agent through Docker.
Ensure you have outbound access to whitelist the following URL for external access: https://app.dbtune.com

Step 1: Database user & extension setup

psql "host=$RDSHOST port=5432 dbname=<your_db_name> user=<username> password=<your_master_password>"

Create user dbtune_agent and grant pg_monitor role.

postgres=> create user dbtune_agent with password 'password';
CREATE ROLE
postgres=> grant pg_monitor to dbtune_agent;
GRANT ROLE

Enable pg_stat_statements on your database.

postgres=> create extension pg_stat_statements;
CREATE EXTENSION

Step 2: Configure IAM permissions

Create a new IAM policy with the following permissions.

Navigate to IAM > Policies > Create policy > Policy editor > select JSON and copy the below permissions.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DescribeRDSInstanceTypes",
      "Effect": "Allow",
      "Action": "ec2:DescribeInstanceTypes",
      "Resource": "*"
    },
    {
      "Sid": "DescribeAndModifyDBInstance",
      "Effect": "Allow",
      "Action": [
        "rds:DescribeDBInstances",
        "rds:ModifyDBInstance"
      ],
      "Resource": "*"
    },
    {
      "Sid": "GetDBMetrics",
      "Effect": "Allow",
      "Action": [
        "cloudwatch:GetMetricStatistics",
        "pi:GetResourceMetrics"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ModifyDBParameterGroup",
      "Effect": "Allow",
      "Action": "rds:ModifyDBParameterGroup",
      "Resource": "*"
    },
    {
      "Sid": "RebootDBInstance",
      "Effect": "Allow",
      "Action": "rds:RebootDBInstance",
      "Resource": "*"
    }
  ]
}

Click Next, provide a unique policy name, and click Create policy.

Attach the policy to your existing IAM user or create a new IAM user.

Find at: IAM > Users > IAM user > Add permissions.

Select Attach policies directly and choose the policy you created in the previous step, then click Next and Add permissions.

Step 3: Deployment options

Option A: Using environment variables

Docker installation: This installation works on Linux, MacOS, any operating system that supports Docker.

To run the dbtune-agent using docker, make sure you have gathered the below parameters mentioned below:

Variable	Description	Find at
`DBT_RDS_DATABASE_IDENTIFIER`	Identifier of the RDS instance you want to optimize	RDS/Aurora Console > Databases > Instance > Configuration (DB instance ID)
`DBT_RDS_PARAMETER_GROUP_NAME`	Name of the parameter group for the instance	RDS/Aurora Console > Parameter Groups
`DBT_AWS_ACCESS_KEY_ID`	AWS access key ID that DBtune will use to authenticate with AWS services	Find at: cat `~/.aws/credentials`
`DBT_AWS_SECRET_ACCESS_KEY`	AWS secret access key that DBtune will use to authenticate with AWS services	Find at cat `~/.aws/credentials`
`DBT_AWS_REGION`	AWS region where your database is located	Find at: RDS/Aurora Console > Databases > Summary
`DBT_DBTUNE_API_KEY`	Unique API key is assigned to your DBtune account	Find at: DBtune app >Agent tab > Docker > Environment variables
`DBT_DBTUNE_DATABASE_ID`	Unique Database ID that is assigned to your database instance	Find at: DBtune app >Agent tab > Docker > Environment variables

After filling in the parameters in the command below, run it to start your dbtune-agent.

docker run docker run --restart always \
-e DBT_POSTPOSGRESQL_CONNECTION_URL="postgresql://<User>:<password>@<rds_endpoint>:5432/<dbname>" \
-e DBT_RDS_DATABASE_IDENTIFIER=<rds identifier> \
-e DBT_RDS_PARAMETER_GROUP_NAME=<custom parameter group/default> \
-e DBT_AWS_REGION=<RDS region> \
-e DBT_DBTUNE_SERVER_URL=https://app.dbtune.com \
-e DBT_DBTUNE_API_KEY=<copy-from-dbtune-agent-page> \
-e DBT_DBTUNE_DATABASE_ID=<copy-from-dbtune-agent-page> \
-e DBT_POSTGRESQL_INCLUDE_QUERIES=true \
-e DBT_AWS_ACCESS_KEY_ID=<your-aws-access-key> \
-e DBT_AWS_SECRET_ACCESS_KEY=<your-aws-secret-key> \
public.ecr.aws/dbtune/dbtune/agent:latest --rds

Binary installation (MacOS/Linux only)

Download the binary agent by running the below command.

curl https://raw.githubusercontent.com/dbtuneai/dbtune-agent/refs/heads/main/setup.sh > /tmp/dbtune-agent.sh && sh /tmp/dbtune-agent.sh

Run with environment variables:

export DBT_RDS_DATABASE_IDENTIFIER=your-database-identifier
export DBT_RDS_PARAMETER_GROUP_NAME=your-parameter-group
export DBT_AWS_ACCESS_KEY_ID=your-aws-access-key
export DBT_AWS_SECRET_ACCESS_KEY=your-aws-secret-key
export DBT_AWS_REGION=your-aws-region
export DBT_DBTUNE_SERVER_URL=https://app.dbtune.com
export DBT_DBTUNE_API_KEY= <copy-from-dbtune-agent-page>
export DBT_DBTUNE_DATABASE_ID= <copy-from-dbtune-agent-page>
export DBT_POSTGRESQL_CONNECTION_URL="postgresql://<User>:<password>@<rds_endpoint>:5432/<dbname>"
export DBT_POSTGRESQL_INCLUDE_QUERIES=true

Finally run the DBtune agent:

./dbtune-agent --rds

./dbtune-agent --aurora

Option B: Using a YAML configuration file

Alternatively you can create dbtune.yaml to run the agent

To run the dbtune-agent using dbtune.yaml, make sure you have gathered the below parameters mentioned below.

connection_url: Username, password, rds endpoint(Find at: RDS Console > Your DB > Connectivity & Security > Endpoint) and database name.
RDS_DATABASE_IDENTIFIER: Find at: RDS/Aurora Console > Databases > Instance > Configuration (DB instance ID).
AWS_REGION: Find at: RDS/Aurora Console > Databases > Summary.
RDS_PARAMETER_GROUP_NAME:Find at: RDS/Aurora Console > Parameter Groups.
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY : Find at: cat ~/.aws/credentials.

Create a dbtune.yaml file and copy the contents from the Agent page > Binary > Config file.

# Example config file for RDS PostgreSQL
postgresql:
  connection_url: postgresql://<User>:<password>@<rds_endpoint>:5432/<dbname>" 
  include_queries: true
  # Set to true to allow the agent to restart the database during tuning

rds:
  AWS_REGION: "<your-aws-region>" 
  AWS_ACCESS_KEY_ID: "<your-aws-access-key-id>"
  AWS_SECRET_ACCESS_KEY: "<your-aws-secret-access-key>"
  RDS_DATABASE_IDENTIFIER: "<your-database-identifier>" # The writer instance of the cluster
  RDS_PARAMETER_GROUP_NAME: "<your-parameter-group-name>" # Be sure to define a custom one and not to use the default.postgresXX one

dbtune:
  server_url: https://app.dbtune.com
  api_key: <copy-from-dbtune-agent-page>
  database_id: <copy-from-dbtune-agent-page>

debug: false

Finally run the DBtune agent:

./dbtune-agent --rds

./dbtune-agent --aurora