- By Admin
- March 28, 2015
- 11 minute read
Being part of the fintech space means compliance and that entails having a good number of years of data backup. We previously hosted our postgres instance manually on an ec2 machine with master-slave setup with daily backups using scripts. But that was when RDS didn’t support postgres.
There are a lot of benefits of moving to RDS but our only concern was how to retain data backups longer than the maximum 30 days that amazon currently allows you. It turned out there is a pretty easy way to do that.
Create an IAM user (DatabaseBackups)
Give it following permissions
| { | |
| "Version": "2012-10-17", | |
| "Statement": [ | |
| { | |
| "Sid": "Stmt1427229307000", | |
| "Effect": "Allow", | |
| "Action": [ | |
| "rds:CreateDBSnapshot", | |
| "rds:CopyDBSnapshot", | |
| "rds:DeleteDBSnapshot", | |
| "rds:DescribeDBInstances", | |
| "rds:DescribeDBSnapshots", | |
| "rds:DescribeReservedDBInstances" | |
| ], | |
| "Resource": [ | |
| "arn:aws:rds" | |
| ] | |
| } | |
| ] | |
| } |
| #!/bin/bash - | |
| export AWS_ACCESS_KEY=<your aws access key> | |
| export AWS_SECRET_KEY=<your aws secret> | |
| date_current=`date -u +%Y-%m-%d` | |
| aws rds describe-db-snapshots --snapshot-type "automated" --db-instance-identifier <db_instance_name> | grep `date +%Y-%m-%d` | grep rds | tr -d '",' | awk '{ print $2 }' > /tmp/sandbox-snapshot.txt | |
| snapshot_name=`cat /tmp/<db_instance_name>-snapshot.txt` | |
| target_snapshot_name=`cat /tmp/<db_instance_name>-snapshot.txt | sed 's/rds://'` | |
| aws rds copy-db-snapshot --source-db-snapshot-identifier $snapshot_name --target-db-snapshot-identifier $target_snapshot_name-copy > /home/ubuntu/rds-snapshot-$date_current.log 2>&1 | |
| echo "-------------" >> /home/ubuntu/$date_current-results.txt | |
| cat /home/ubuntu/rds-snapshot-$date_current.log >> /home/ubuntu/$date_current-results.txt | |
| cat /home/ubuntu/$date_current-results.txt | mail -s "[Daily RDS Snapshot Backup] $date_current" <email@foo.com> | |
| rm /home/ubuntu/$date_current-results.txt | |
| rm /home/ubuntu/rds-snapshot-$date_current.log |
What the script essentially does is finds the latest automated snapshot of your database instance and creates a manual copy of it for the given day. In order for this script to actually work you need to:
1. Setup automated backups on your database instance (why would you not have already done this!!)
2. Setup the cron job time so that it runs after your automated snapshot occurs.
And that’s all thats required.
