Amazon Aurora Migration

The Challenge

Remitly is a digital money transfer platform that makes the process of sending money faster, easier, more transparent and less costly. As a rapidly growing company, it is critical that their Amazon Web Services (AWS) infrastructure effectively handles their fast pace and increase in capacity demands. Changes to their AWS infrastructure required making improvements and upgrades to their existing databases and database infrastructure to increase their performance, scalability and reliability.

In their search for a solution and through recommendations from AWS, Remitly turned to 1Strategy, an AWS Premier Consulting Partner, to plan and execute their database migration from Amazon RDS MySQL to Amazon Aurora. Aurora is a relational database engine compatible with MySQL and PostgreSQL and was a progressive solution that could be applied to each of Remitly's five Amazon RDS MySQL production databases. They sought guidance and recommendations from 1Strategy to complete the following tasks:

• Design and organize the migration of five mission-critical RDS MySQL production transactional databases, migrating one-by-one to Aurora using Aurora Read Replicas.
• Execute the migration with Remitly's developers being trained and observing simultaneously.
• Ensure data integrity and database backups throughout the migration process.

Why Amazon Web Services

As a startup in 2011, Remitly had strong in-house expertise with AWS. Launching their company in AWS allowed them to take advantage of cloud-native architecture. Although Remitly had years of experience working in AWS, they sought the help of 1Strategy’s deep expertise in cloud architecture and database migration strategies on AWS, providing both the assurance of the migration’s success and expert knowledge to their development team.

“We already had deep knowledge and experience with AWS; however, it’s part of our culture to continuously improve. We turned to 1Strategy for their expertise, assistance and to grow our proficiency during this migration,” said Mike Foster, Director of Engineering for Platform and Internal Products at Remitly.

The Benefits

For Remitly, the storage and performance of Amazon RDS MySQL was hitting the limits and did not effectively handle the volume of data flooding into their expanding databases. They instead chose to begin using Amazon Aurora. Aurora is a fully managed, enterprise-grade relational database engine that delivers the speed and reliability in a cost-effective and simple way.

By migrating to Aurora, Remitly is now able to leverage Aurora’s capabilities to achieve their database infrastructure goals including:

• Increased scalability
• Higher performance
• Faster replication

“Aurora was the logical next step in our AWS persistence evolution. Long term, we want to be fully on DynamoDB. We needed a scalable, high-performance database to meet the needs of our growing customer base, who need to send money quickly, safely and securely. Amazon Aurora gives us better performance and scalability than MySQL, allowing us to provide better services to our customers,” Foster said.

Scalability and Automatic Failover

Amazon Aurora provides a low-cost, stable, scalable and highly performant relational database solution for Remitly. Aurora offers cluster volumes that can be divided into segments and can automatically grow to a size that fits the need of the user or to a maximum size of 64 terabytes (TB). Aurora automatically maintains six copies of your data across three Availability Zones (AZ) and will automatically attempt to recover your database from a healthy AZ with no data loss.

Remitly was able to take advantage of these benefits in two ways: First, they do not need to provision storage in advance. This saves the company the effort of predicting and provisioning storage required during busy times of the year when demand is high, allowing for uninterrupted business. Second, as Aurora replicates data to storage nodes in multiple AZs, it allows the company to withstand catastrophic losses if disaster were to strike in one AZ, without any impact to Remitly’s applications.

Additionally, Aurora has a very simple pay-as-you-go pricing model. Remitly pays for only database reads and writes (I/O) and utilized storage. There are no upfront costs or long-term commitments.

“While MySQL instances on Amazon RDS are slightly less expensive than equivalently sized Amazon Aurora instances, Aurora’s superior performance translates into a need for fewer and smaller instances. Our dollars go farther with Aurora than they did with MySQL,” Foster said.

High Performance

Utilizing Aurora allows Remitly to improve their database performance. Aurora delivers up to five times the performance of MySQL running on the same hardware as a result of the reduced write load to the filesystem and minimization of resource contention due to locking operations. With Aurora, Remitly is able to use smaller EC2 instances than with MySQL.

At the same time, Remitly saw improved latency for APIs that use the migrated databases. All metrics above the 50th percentile were improved, and at the higher percentiles, they saw much more consistency in response time. Below are two metrics graphs of p90 API latency during the migration:

With Aurora, Remitly can now horizontally scale their databases to up to 15 read replicas, whereas with MySQL, they were limited to five read replicas. Aurora’s higher performance implies that Remitly can now often use fewer or smaller instances than with MySQL.

Replication, Automatic Failover and Backups

Remitly’s approach in migrating to Aurora included assistance from 1Strategy to establish Aurora’s Read Replicas. These replicas are well-suited for horizontally scaling and can increase the write capacity of the cluster. Aurora supports up to 15 replicas, the creation of which has minimal impact on the performance of write operations. Remitly’s developers can scale up resources allocated to a database instance and improve availability through the use of Amazon Aurora Replicas.

A crucial benefit is that one of Remitly’s Aurora Replicas can be promoted to a primary instance without any data loss, which helps the company with fault tolerance if their primary instance fails. Given that these were mission-critical production databases being migrated to Aurora, this was a big win for Remitly.

Amazon Aurora provides continuous backups to Amazon S3 in real time, with no impact on database performance. This eliminates the time and need for backup windows and automated backup scripts. Remitly can also configure the length and time Aurora maintains these backups, as well as allow the company to restore from a database cluster snapshot.

Migrating to Amazon Aurora on Amazon RDS from Amazon RDS MySQL database engine proved valuable for Remitly by helping to boost the performance, reliability, elasticity and scalability of their databases, allowing them to satisfy their growth while substantially reducing the total cost of ownership. Additionally, Remitly’s administrative overhead is now decreased as Amazon Aurora is a managed service that incorporates operational support. Now Remitly doesn’t have to worry about installing software or dealing with any hardware failures.

“I was extremely impressed with 1Strategy as they collaborated with our team in the migration and execution of our databases to Amazon Aurora,” Foster said. “1Strategy’s efforts and the transition to Amazon Aurora resulted in saving an estimated 10 minutes of database-related downtime per month, while training and educating our developers on performing production migrations without negatively impacting customers.”