SlideShare a Scribd company logo

MongoDB Server Provisioning - From 2 Months to 2 Minutes

•Download as PPTX, PDF•
•
MongoDB
MongoDB

Anthem is one of the largest healthcare companies in the USA. Transforming such a large enterprise and enabling true agile development depends on being able to get resources fast when you need them. How hard can it be to get a single server set up? The answer is - it can take weeks or even months! In this presentation, we will talk about how we shortened the delivery of MongoDB deployments to our developers to minutes using MongoDB OpsManager and, of course, some magic. We will cover the details of the approach and the challenges we had to face so that you can do it too.

1 of 20
MongoDB server provisioning: From 2 months… to 2 minutes Chetan Gavankar, DB Executive Advisor@ Anthem September 2018
22 Topics to Cover • MongoDB in Anthem • Demand for quick deployment • Road Map to achieve our goal • How we did it • Lesson learned • Future Ideas • Q/A
MongoDB in Anthem • Anthem as an organization – 2nd largest HealthCare company – Lots of development using SQL and NoSQL • Why MongoDB a) Flexible schema design b) High availability • From few serves to 200 servers in short time 3
4 Demand for quick deployment Current Process (1-2 months) Mongodb Server App team request server Work in Take Process to acquire HW Request to create all IDs Verify HW configurati on OS setup and verificatio n Firewall /Port setup Centrify setup Install MongoDB Setup Ops manager DBA checklist
5 Demand for quick deployment Proposed Process (2 minutes) Mongodb Server App team request server MongoDB server delivered from Ops manager server pool Work in Take Process acquired new servers Servers are added back to server pool
6 What’s the advantage of Automation • Faster delivery of servers to customers i.e. happy customers • Consistent setup when delivering these servers • DBA efficiency Time and effort saved to build these servers can be utilized to work closely with application team on design review , performance testing and more interesting things for DBA to work on .
7 How We did it • Automation of Server Build process • Ops Manager’s Server Pool feature • Automation Software using Ops Manager API and Python
8 Automation of Server Build process • Standard offering – We developed SPO (server image) which allows to build servers with specific T-shirt sizes i.e. predefined CPU/MEMORY/STORAGE (mount points)/OS • Scripted OS configuration – The script performs OS setup as recommended in MongoDB Production notes https://docs.mongodb.com/manual/administration/production-notes • No more individual firewall rules! – We created a Firewall group for the given VLAN so required ports are open to allow communications between DB servers and Ops Managers and within DB server itself • Minimizing human intervention – By the time servers are delivered to DBA team only manual process is to install ops manager automation agent and edit the server pool configuration file (this also we will automate soon)
9 Ops Manager Server Pool Feature
10 Ops Manager Server Pool Feature Custom user defined properties (stored in the pool properties file on each host)
11 Automation Software • Ops Manager REST API – Gives incredible flexibility in building scripts/software to manage MongoDB deployments • Python – Easy to learn programming language – OOO development – Package to handle REST API calls
12 Single Command to Deploy A single command will create new Group in Ops manager , bind servers to new group depending on the deployment topology and characteristics selected , install MongoDB version selected , setup monitoring , setup alerts , setup backup and put DB servers under Ops manager Automation …and bring you coffee and fresh milk python maintenance/creategroupanddeploy.py --opsmgrUri http://***.**.com:80 --opsmgrapiuser <email user id> --opsmgrapikey *** --organizationId 5af1482f99c8472f18541146 --groupName "FAMRTEST01" --environments DEV --deploymentTopology 'single-node' --tShirtSize small --chipset intel --location dc1 --loglevel debug --mongoDbVersion "3.6.5“ --deploymentName "farmDV01" --disableAlerts --disableBackup
13 Originally just a small single script Automation SW Architecture Diagram
14 Ops manager REST API - Example
15 Calling Ops Manager API from Python - Example import requests Connector = OpsMgrConnector(args.opsMgrUri, args.opsMgrApiUser, args.opsMgrApiKey) return creategroup(args) _________________________________________________________________________________ def creategroup(args): return addGroup(args.opsMgrUri, args.groupName, environmentType, organizationId) def addGroup(opsMgrUri, groupName, groupType, organizationId): addGroupResultJson = opsMgrConnector.addGroup(newGroupName, organizationId) def addGroup(self, newGroupName, organizationId): payload = { "name" : newGroupName, "orgId" : organizationId } return self.post("{}/groups".format(self.apiURL), payload) def post(self, url, payload): result = requests.post(url, json=payload, auth=self.auth).json() return result
16 MongoDB Server Deployed in Ops manager
17 Lesson Learned • DNS Caching – Nontrivial issues when DNS queries are taking long time • Lots of testing – Different test cases – Different HW such as Intel , Power , Private Cloud, etc. • Non-technical – Project planning and roadmap is a must – Everyone involved should understand project goals and individual responsibilities
18 Future Ideas • More automation – Alert templates • Explore integration with other automation/orchestration tools – Chef, Puppet, Ansible – Kubernetes
19 Yes it is possible , we did it and All Clicked Faster delivery of servers to customers i.e. happy customers  Consistent setup when delivering these servers  Time saved to build these servers can be utilized to work closely with application team on design review , performance testing and more interesting things for DBA to work on . Automation is flexible enough to be deployed on other platforms
20 Q/A Chetan Gavankar, 2018

More Related Content

MongoDB Server Provisioning - From 2 Months to 2 Minutes

  • 1. MongoDB server provisioning: From 2 months… to 2 minutes Chetan Gavankar, DB Executive Advisor@ Anthem September 2018
  • 2. 22 Topics to Cover • MongoDB in Anthem • Demand for quick deployment • Road Map to achieve our goal • How we did it • Lesson learned • Future Ideas • Q/A
  • 3. MongoDB in Anthem • Anthem as an organization – 2nd largest HealthCare company – Lots of development using SQL and NoSQL • Why MongoDB a) Flexible schema design b) High availability • From few serves to 200 servers in short time 3
  • 4. 4 Demand for quick deployment Current Process (1-2 months) Mongodb Server App team request server Work in Take Process to acquire HW Request to create all IDs Verify HW configurati on OS setup and verificatio n Firewall /Port setup Centrify setup Install MongoDB Setup Ops manager DBA checklist
  • 5. 5 Demand for quick deployment Proposed Process (2 minutes) Mongodb Server App team request server MongoDB server delivered from Ops manager server pool Work in Take Process acquired new servers Servers are added back to server pool
  • 6. 6 What’s the advantage of Automation • Faster delivery of servers to customers i.e. happy customers • Consistent setup when delivering these servers • DBA efficiency Time and effort saved to build these servers can be utilized to work closely with application team on design review , performance testing and more interesting things for DBA to work on .
  • 7. 7 How We did it • Automation of Server Build process • Ops Manager’s Server Pool feature • Automation Software using Ops Manager API and Python
  • 8. 8 Automation of Server Build process • Standard offering – We developed SPO (server image) which allows to build servers with specific T-shirt sizes i.e. predefined CPU/MEMORY/STORAGE (mount points)/OS • Scripted OS configuration – The script performs OS setup as recommended in MongoDB Production notes https://docs.mongodb.com/manual/administration/production-notes • No more individual firewall rules! – We created a Firewall group for the given VLAN so required ports are open to allow communications between DB servers and Ops Managers and within DB server itself • Minimizing human intervention – By the time servers are delivered to DBA team only manual process is to install ops manager automation agent and edit the server pool configuration file (this also we will automate soon)
  • 9. 9 Ops Manager Server Pool Feature
  • 10. 10 Ops Manager Server Pool Feature Custom user defined properties (stored in the pool properties file on each host)
  • 11. 11 Automation Software • Ops Manager REST API – Gives incredible flexibility in building scripts/software to manage MongoDB deployments • Python – Easy to learn programming language – OOO development – Package to handle REST API calls
  • 12. 12 Single Command to Deploy A single command will create new Group in Ops manager , bind servers to new group depending on the deployment topology and characteristics selected , install MongoDB version selected , setup monitoring , setup alerts , setup backup and put DB servers under Ops manager Automation …and bring you coffee and fresh milk python maintenance/creategroupanddeploy.py --opsmgrUri http://***.**.com:80 --opsmgrapiuser <email user id> --opsmgrapikey *** --organizationId 5af1482f99c8472f18541146 --groupName "FAMRTEST01" --environments DEV --deploymentTopology 'single-node' --tShirtSize small --chipset intel --location dc1 --loglevel debug --mongoDbVersion "3.6.5“ --deploymentName "farmDV01" --disableAlerts --disableBackup
  • 13. 13 Originally just a small single script Automation SW Architecture Diagram
  • 14. 14 Ops manager REST API - Example
  • 15. 15 Calling Ops Manager API from Python - Example import requests Connector = OpsMgrConnector(args.opsMgrUri, args.opsMgrApiUser, args.opsMgrApiKey) return creategroup(args) _________________________________________________________________________________ def creategroup(args): return addGroup(args.opsMgrUri, args.groupName, environmentType, organizationId) def addGroup(opsMgrUri, groupName, groupType, organizationId): addGroupResultJson = opsMgrConnector.addGroup(newGroupName, organizationId) def addGroup(self, newGroupName, organizationId): payload = { "name" : newGroupName, "orgId" : organizationId } return self.post("{}/groups".format(self.apiURL), payload) def post(self, url, payload): result = requests.post(url, json=payload, auth=self.auth).json() return result
  • 16. 16 MongoDB Server Deployed in Ops manager
  • 17. 17 Lesson Learned • DNS Caching – Nontrivial issues when DNS queries are taking long time • Lots of testing – Different test cases – Different HW such as Intel , Power , Private Cloud, etc. • Non-technical – Project planning and roadmap is a must – Everyone involved should understand project goals and individual responsibilities
  • 18. 18 Future Ideas • More automation – Alert templates • Explore integration with other automation/orchestration tools – Chef, Puppet, Ansible – Kubernetes
  • 19. 19 Yes it is possible , we did it and All Clicked Faster delivery of servers to customers i.e. happy customers  Consistent setup when delivering these servers  Time saved to build these servers can be utilized to work closely with application team on design review , performance testing and more interesting things for DBA to work on . Automation is flexible enough to be deployed on other platforms