Migrating to Amazon RDS with Database Migration Service

Editor's Notes

1. Intro myself. What I do. Time with Amazon/AWS
2. Start with some audience interaction… How many of you have been to the AWS Database Migration website? Has anyone tried using DMS yet?
3. When migrating your database to the cloud you do more than just change location. You leverage the power of the cloud. Customers appreciate the flexibility that platforms like EC2 offer, and many do run database engines there, but the RDS engines take things to the next level.
4. Provision a database in 6 minutes Provision an MAZ database with a few mouse clicks Scale a database up/down with 60–90 seconds of downtime Apply patches with 60–90 seconds of downtime Add Read Replicas with a few mouse clicks Protect your backups and logs with 11 9s of durability Recover to any point in time from nightly backups + logs Detailed metrics, down to 1-second intervals Secure your data with single-click encryption at rest >> Don’t have to maintain shards Leaves time for DBA’s and DBEng’s to optimise and improve instead of doing maintenance
5. Responsive to customers Migrations easier and less intrusive Migrate will low downtime to any RDS engine type   Post migration, customers asked for ongoing sync Sync between on prem and also between engines This mobility addresses one of the big requests: The ability to migrate off of commercial, license-intensive database engines, onto cloud-native, non-commercial engines.
6. - Pre DMS, costly, license-intensive replication software to move data - Were not simple to setup or use required expert resources - Long downtimes the other option - Customers who wanted to switch engines had to deal with differences in datatypes, table syntax, sp’s, functions, triggers
7. So it was with all these factors in mind that we developed the database migration service. We designed it to be simple - you can get started in less than ten minutes. We designed it to enable near-zero-downtime migration. And we designed it to be a kind of replication Swiss army knife. To replicate data between on-premises systems, RDS, EC2, and across database engine type
8. There are other ways of migrating too. Often the native option is the best option. They come with drawbacks though: downtime and the inability to move between engines
9. DMS and SCT can help you modernize, migrate, and replicate Modernise: Convert from one database engine to another, and update the associated application code using the code conversion features of SCT Migrate: You’re looking to move data from A to B. Whether that is on premise to the cloud, between EC2 and RDS, or between database engines/warehouse Replicate: Leverage the power of CDC and keep changes flowing from source to target on an ongoing basis
10. The real power of the solution becomes apparent when you realise that you can move between database engines or between data warehouses, enabling you to move away from expensive, commercial databases, to cloud-native, open source solutions. And don’t worry, at AWS we don’t believe in vendor lock-in. You can use DMS just as easily to move data out of the cloud as into it.
11. Data warehouses generally have very large data sets than cannot be moved over the wire in a reasonable amount of time SCT can now convert the warehouse schema to Redshift and extract the data from the source warehouse in a format Redshift can ingest Redshift is a fully managed, petabyte-scale data warehouse that makes it simple and cost-effective to analyze all your data using your existing business intelligence tools.
12. The net is that DMS and SCT bring a lot of advantages to the market and can make it easier to move to the cloud.
13. Fix step 2. Depends.
14. The first step of any migration is to move your schema. If you are doing a homogeneous migration, there are a range of tools available to help with this (including SCT). If you’re doing a heterogeneous migration, SCT can tell you how much work it will take to convert your schema and, if you don’t have a set preference, recommend the best target database engine. Cloud optimization Code conversion Etc.
15. Can also extract warehouse data
16. Let’s explore how it is that DMS can enable near zero downtime migration. You use DMS for relational migrations and migrating relational data into Redshift. The customer starts by launching a DMS instance in their AWS account. (click) Then the customer provides database connection information to connect out to their on-premises database, and in to their AWS database. (click) Next the customer will select which tables, schemas or databases they want to migrate, (click) and DMS will load the data, and keep it in sync on an ongoing basis. (click) Finally, at the time of the customer’s choosing, they can simply change the application to point to the new AWS database, instead of their old on-premises database. (click)
17. Creates tables at target database Sets up metadata required at target Populates data from source Each process loads one entire table Can use multiple processes Can be paused When restarted will continue from where it was stopped Will reload any tables that were currently in progress
18. Oracle supplemental logging MySQL row-level bin logging SQL Server bulk logged/full recovery Postgres WAL No agent Uses recovery log Native change data capture API
19. When you either migrate or replicate, you can choose the Multi-AZ option. This option provides DMS in a high availability mode–in case the primary DMS server fails, the secondary one kicks in.
20. So by now it should be pretty obvious that data can move from A to B, and many have used DMS to do just that. Some customers have been pretty ingenious with DMS and have fanned in, or consolidated systems. DMS will put data where you ask it to, so if you want to merge data from a number of legacy systems into a single repository, DMS can help with this. And remember, DMS can just as easily move data between like databases and unlike databases, so you can take information from a number of different platforms and consolidate it in a single, cloud database.
21. In addition to “fanning-in” or consolidating, we’ve had a customer who has taken their database and “fanned-out” different portions to different targets. For example, from their ERP system they replicate they customer list out to the CRM system and also take some of the general ledger transactions and funnel them over to a BI system. We’ve also had a customer who, for legacy reasons, used a single database for three different applications. With DMS you can split this information out to different databases. One could be PostgreSQL, one SQL Server, and one Aurora
22. Some organisations have many years of historical data, and for those that are willing to make the leap, filtering can be done during a migration so that only newer information is kept. Filtering can be done on any field in the source database–not just dates.
23. Just in case it wasn’t abundantly clear–DMS can move your data from pretty much anywhere to pretty much anywhere. Sort of like a courier company for the cloud
24. Mention why no older engines as sources
25. Where can you learn more? (CLICK) The User Guide and download option are available from the DMS section of the AWS Console (CLICK) Yeah, there’s the Download button. Click it and you’ll see something like this. (CLICK) Information on installation and use (CLICK) The Schema Conversion Tool has its own forum to ask questions and get help
26. And that’s it. I really appreciate your attendance at this session today and I hope you got a lot of useful information out of this and that it helps you move forward in better utilizing DMS and SCT.