SlideShare a Scribd company logo

MongoDB Backup & Disaster Recovery

•Download as PPTX, PDF•
•
E
Elankumaran Srinivasan

Presentation on disaster recover for MongoDB Atlas. Different approaches to recover and restore MongoDb Data. https://elang2.github.io/myblog/

Related slideshows

ProxySQL High Avalability and Configuration Management Overview
ProxySQL High Avalability and Configuration Management OverviewProxySQL High Avalability and Configuration Management Overview
ProxySQL High Avalability and Configuration Management Overview
 
Modeling Data and Queries for Wide Column NoSQL
Modeling Data and Queries for Wide Column NoSQLModeling Data and Queries for Wide Column NoSQL
Modeling Data and Queries for Wide Column NoSQL
 
Galera Cluster Best Practices for DBA's and DevOps Part 1
Galera Cluster Best Practices for DBA's and DevOps Part 1Galera Cluster Best Practices for DBA's and DevOps Part 1
Galera Cluster Best Practices for DBA's and DevOps Part 1
 
1 of 18
MongDB Backup & Disaster Recover Elankumaran Srinivasan
Why Backup & Disaster Recovery ? Application MongoDB Driver Company’s Data Centre 1. Possible network issue between the data center & other external cloud services. 2. Cloud cluster down or unreachable or terminated accidentally or data deleted accidentally Application MongoDB Driver Company’s Data Centre Disaster recovery strategy is essential to quickly get access to the minimum required data and keep the application depending on the critical data running with smallest down time.
Key Measurement Parameters For DR 1. Minimum time required to spin up a new cluster with snapshot data. a) Account for snapshot data download/transfer time. b) Cluster setup, restore and configure time. 2. Time lag between live mirrored secondary cluster and the primary cluster. 3. Can the data be restored into clusters of any topology?
Production Cluster On MongoDB Atlas • Sharded Cluster. • Two replica sets. • Each replica set has 3 nodes. One primary and two secondary. • 6 mongos running on the same hosts as 6 mongod. • Config servers running as a 3 node replica set. • Snapshots taken twice a day. • Automatic, periodic snapshots. Useful for point in time recovery. • Snapshots stored on Amazon S3. • All mongos are part of a load balancer. Applications connect to these mongos through load balancer.
Production Cluster On MongoDB Atlas Shard#1 Shard#2 Instance Port mongod 27017 mongos 37017 Primary Secondary Secondary mongos mongos mongos Load Balancer For Mongos Nodes Application MongoDB Driver Mongocfg (Primary) Mongocfg (Sec) Mongocfg (Sec) Primary Secondary Secondary mongos mongos mongos
Disaster Recovery (Different Approaches) Create new cluster in company’s data center & restore snapshot MongoDB Atlas MongoDB Atlas (Create a new cluster & restore snapshot) Spin up a new cluster on AWS and restore
Data Restore Considerations Different Topology Mongo Cluster • Restore into a sharded cluster. • Restore into a stand alone node or a replica set. • Restore into a cluster with similar topology as Production. • Restore into a cluster with with a different topology.
Restoring Snapshot to a new Cluster on MongoDB Atlas After selecting restore my snapshot, select the new cluster that you provisioned on MongoDB Atlas
Downloading the Snapshot to restore into non-Atlas Environment
Steps to restore a sharded cluster ( Same Topology) Restoring data into a new sharded cluster involves the following steps: 1. Setting up and restoring data into mongod sharded servers. 2. Setting up config servers as a replica set. 3. Setting up mongos (routers) and hooking them up to the config servers. 4. Adding sharded nodes(configured in step #1) via mongos
Steps to restore a sharded cluster ( Same Topology) 1. a. Set up a new cluster with the same topology as Production. Don’t start any of the mongo services. 1. b. Restore the snapshot into one of the mongod nodes. Lets assume in a path /data/db 1. c. Start the mongod on the node while pointing to the restored data path. mongod --dbpath /data/db --replSet <replName> --shardsvr 1.d. Connect to mongo shell on the node and initiate a new replica set. mongo rs.initate() Example : mongod --dbpath /data/db --replSet rs0 --shardsvr
Steps to restore a sharded cluster ( Same Topology) 1. e. While connected to the mongo shell , configure the replica set. rs.initiate( { _id : "rs1", members: [ { _id : 0, host : ”s1-mongo1:27017" } ] }) 1.g. Repeat steps 1.b through 1.f for replica set ”r1 ” and hosts s2-mongo1, s2-mongo2 and s2-mongo3 1. f. Add the other members of the replica set. rs.add(”s2-mongo1”) rs.add(”s3-mongo1”)
Steps to restore a sharded cluster ( Same Topology) - Contd… 2.a. Setup the 3 config servers as replica set. Start mongod as config servers for each node. mongod --configsvr --replSet configReplSet mongo rs.initiate( { _id: "configReplSet", configsvr: true, members: [ { _id: 0, host: ”config01:27019" }, { _id: 1, host: ”config02:27019" }, { _id: 2, host: ”config03:27019" } ] } ) 2.b. Log into one of the config server to set up the replica set and add other members to the replicaset.
Steps to restore a sharded cluster ( Same Topology) - Contd… 3.a. Setup 6 mongos by executing the following command on each of the mongos nodes. mongos --configdb configReplSet/config01:27019,config02:27019,config03:27019
Steps to restore a sharded cluster ( Same Topology) - Contd… 4.a. Add shard#1 into the cluster. In order to do this, connect to one of the mongos shells and execute the following command. mongo mong01-router:27017/admin sh.addShard( "rs0/s1-mongo1:27017,s1-mongo2:27017,s1-mongo3:27017" ) 4.b. Add the nodes for shard#2 to the cluster. sh.addShard( "rs1/s2mongo1:27017,s2-mongo2:27017,s2-mongo3:27017" )
Steps to restore to a single replica set. Restoring data into a single replica set. 1. Setting up and restoring data into mongod servers. Follow the same instructions as in Slide #10 and Slide #11 . 2. Non-Sharded cluster does not have mongos & mongod config servers.
Live Mirrored Stand By Cluster Application MongoDB Driver Company’s Data Centre Mongo-mirror application Replicate into The Mongo connector utility can be used to migrate data from one cluster to another. This can be used to setup a live secondary stand by cluster. Reference : https://github.com/mongodb-labs/mongo-connector
Setting Up Mongo Mirror Shard#2 Shard#2 Primary Secondary Secondary mongos mongos mongos Primary Secondary Secondary mongos mongos mongos Shard#1 Shard#2 Primary Secondary Secondary mongos mongos mongos Primary Secondary Secondary mongos mongos mongos Mongo connector Mongo connector

More Related Content

MongoDB Backup & Disaster Recovery

  • 1. MongDB Backup & Disaster Recover Elankumaran Srinivasan
  • 2. Why Backup & Disaster Recovery ? Application MongoDB Driver Company’s Data Centre 1. Possible network issue between the data center & other external cloud services. 2. Cloud cluster down or unreachable or terminated accidentally or data deleted accidentally Application MongoDB Driver Company’s Data Centre Disaster recovery strategy is essential to quickly get access to the minimum required data and keep the application depending on the critical data running with smallest down time.
  • 3. Key Measurement Parameters For DR 1. Minimum time required to spin up a new cluster with snapshot data. a) Account for snapshot data download/transfer time. b) Cluster setup, restore and configure time. 2. Time lag between live mirrored secondary cluster and the primary cluster. 3. Can the data be restored into clusters of any topology?
  • 4. Production Cluster On MongoDB Atlas • Sharded Cluster. • Two replica sets. • Each replica set has 3 nodes. One primary and two secondary. • 6 mongos running on the same hosts as 6 mongod. • Config servers running as a 3 node replica set. • Snapshots taken twice a day. • Automatic, periodic snapshots. Useful for point in time recovery. • Snapshots stored on Amazon S3. • All mongos are part of a load balancer. Applications connect to these mongos through load balancer.
  • 5. Production Cluster On MongoDB Atlas Shard#1 Shard#2 Instance Port mongod 27017 mongos 37017 Primary Secondary Secondary mongos mongos mongos Load Balancer For Mongos Nodes Application MongoDB Driver Mongocfg (Primary) Mongocfg (Sec) Mongocfg (Sec) Primary Secondary Secondary mongos mongos mongos
  • 6. Disaster Recovery (Different Approaches) Create new cluster in company’s data center & restore snapshot MongoDB Atlas MongoDB Atlas (Create a new cluster & restore snapshot) Spin up a new cluster on AWS and restore
  • 7. Data Restore Considerations Different Topology Mongo Cluster • Restore into a sharded cluster. • Restore into a stand alone node or a replica set. • Restore into a cluster with similar topology as Production. • Restore into a cluster with with a different topology.
  • 8. Restoring Snapshot to a new Cluster on MongoDB Atlas After selecting restore my snapshot, select the new cluster that you provisioned on MongoDB Atlas
  • 9. Downloading the Snapshot to restore into non-Atlas Environment
  • 10. Steps to restore a sharded cluster ( Same Topology) Restoring data into a new sharded cluster involves the following steps: 1. Setting up and restoring data into mongod sharded servers. 2. Setting up config servers as a replica set. 3. Setting up mongos (routers) and hooking them up to the config servers. 4. Adding sharded nodes(configured in step #1) via mongos
  • 11. Steps to restore a sharded cluster ( Same Topology) 1. a. Set up a new cluster with the same topology as Production. Don’t start any of the mongo services. 1. b. Restore the snapshot into one of the mongod nodes. Lets assume in a path /data/db 1. c. Start the mongod on the node while pointing to the restored data path. mongod --dbpath /data/db --replSet <replName> --shardsvr 1.d. Connect to mongo shell on the node and initiate a new replica set. mongo rs.initate() Example : mongod --dbpath /data/db --replSet rs0 --shardsvr
  • 12. Steps to restore a sharded cluster ( Same Topology) 1. e. While connected to the mongo shell , configure the replica set. rs.initiate( { _id : "rs1", members: [ { _id : 0, host : ”s1-mongo1:27017" } ] }) 1.g. Repeat steps 1.b through 1.f for replica set ”r1 ” and hosts s2-mongo1, s2-mongo2 and s2-mongo3 1. f. Add the other members of the replica set. rs.add(”s2-mongo1”) rs.add(”s3-mongo1”)
  • 13. Steps to restore a sharded cluster ( Same Topology) - Contd… 2.a. Setup the 3 config servers as replica set. Start mongod as config servers for each node. mongod --configsvr --replSet configReplSet mongo rs.initiate( { _id: "configReplSet", configsvr: true, members: [ { _id: 0, host: ”config01:27019" }, { _id: 1, host: ”config02:27019" }, { _id: 2, host: ”config03:27019" } ] } ) 2.b. Log into one of the config server to set up the replica set and add other members to the replicaset.
  • 14. Steps to restore a sharded cluster ( Same Topology) - Contd… 3.a. Setup 6 mongos by executing the following command on each of the mongos nodes. mongos --configdb configReplSet/config01:27019,config02:27019,config03:27019
  • 15. Steps to restore a sharded cluster ( Same Topology) - Contd… 4.a. Add shard#1 into the cluster. In order to do this, connect to one of the mongos shells and execute the following command. mongo mong01-router:27017/admin sh.addShard( "rs0/s1-mongo1:27017,s1-mongo2:27017,s1-mongo3:27017" ) 4.b. Add the nodes for shard#2 to the cluster. sh.addShard( "rs1/s2mongo1:27017,s2-mongo2:27017,s2-mongo3:27017" )
  • 16. Steps to restore to a single replica set. Restoring data into a single replica set. 1. Setting up and restoring data into mongod servers. Follow the same instructions as in Slide #10 and Slide #11 . 2. Non-Sharded cluster does not have mongos & mongod config servers.
  • 17. Live Mirrored Stand By Cluster Application MongoDB Driver Company’s Data Centre Mongo-mirror application Replicate into The Mongo connector utility can be used to migrate data from one cluster to another. This can be used to setup a live secondary stand by cluster. Reference : https://github.com/mongodb-labs/mongo-connector
  • 18. Setting Up Mongo Mirror Shard#2 Shard#2 Primary Secondary Secondary mongos mongos mongos Primary Secondary Secondary mongos mongos mongos Shard#1 Shard#2 Primary Secondary Secondary mongos mongos mongos Primary Secondary Secondary mongos mongos mongos Mongo connector Mongo connector