7-Step Recipe For Continuous Integration Using OpenStack - Part 2
•
1 like•747 views
Looking to automate your CI/CD workload and wondering how OpenStack can help? Then look no further. In this webinar, we captured best practices for running a smoothly integrated CI/CD environment using OpenStack. These are lessons from running a large scale private cloud internally at Platform9. Topics covered in this webinar: 1. The Ingredients: Basic requirements for setting up your CI/CD workflow with OpenStack, including recommended hardware specification. 2. The Recipe: - Getting your OpenStack environment up and running - Choosing the appropriate Continuous Integration Tool (TeamCity, Jenkins, etc) - End to end 7-step workflow with examples, starting with Developers pushing changesets source control, to those changes being picked up and tested via a 100% automated workflow using OpenStack.
Report
Share
7-Step Recipe For Continuous Integration Using OpenStack - Part 2
- 1. A Recipe for CI using OpenStack Private Cloud Part II: The 7-Step Recipe Bich Le & Madhura Maskasky
- 2. Agenda • The product being integrated and tested • CI Architecture • Flow • Exploiting OpenStack features
- 3. OpenStack Controller Platform9 Architecture nova-scheduler nova-api nova-conductor cinder- api cinder- scheduler glance- registry glance-api neutron-server Compute Nodes Platform9 agent Block Storage Node Platform9 agent Network Node Platform9 agent Internet Customer intranet nova-compute cinder-volume neutron agents Platform9 serviceskeystone
- 4. Software Organization • Component – Single purpose, self-contained – Stored in a single git repo – Repo contains build script and unit tests – Build produces artifacts: RPM or Tarball • Product – Grouping of all components that make up the product – Central repo (pf9-main) contains build script and integration tests – Build produces a Controller “Deployment Unit” Image
- 5. Steps 1 and 2: Build Components CI Engine: Teamcity Git Repos 2a. Check out Scripts 2b. Build & run unit tests Artifacts 2c. Collect artifacts 2d. Merge 1. Developer pushes change into automerge branch of a component
- 6. Flow summary for components 1. Developer pushes change to “automerge” branch of a component’s git repo 2. Teamcity detects change, checks out code, builds , runs unit tests. If unit tests pass, archive artifacts, mark build as successful, and merge changes to master branch
- 8. Steps 3-5: build image, deploy pod, test CI Engine: Teamcity Git Repos OpenStack Private Cloud OS Images 5. ITG Tests 3a. Check out integration repo and all dependent artifacts Scripts Artifacts 3b. Run integration scripts Temp Instance Create Populate using AnsibleDU Image 3c. Snapshot Instances Pods 4. Create & configure
- 9. OpenStack Controller A pod represents a full deployment nova-scheduler nova-api nova-conductor cinder- api cinder- scheduler glance- registry glance-api neutron-server Compute Nodes Platform9 agent Block Storage Node Platform9 agent Network Node Platform9 agent Internet Customer intranet nova-compute cinder-volume neutron agents Platform9 serviceskeystone
- 10. Integration tests • Exercise end user product features, for e.g: – Upload images to cloud – Spawn instances – Connect instances over virtual network • Automation – REST APIs for interacting with the cloud controller – Fabric for running ssh commands inside instances
- 11. Flow summary for product build & test - Step 3 Build cloud image if >=1 components changed - Step 4 Create and configure one pod per integration test - Step 5 Run integration tests
- 12. Tagging and cleanup Step 6 – Tag image with PASS/FAIL flag Step 7 – Cleanup resources – Instances are deleted on successful run, preserved on failure to allow troubleshooting – Images kept for a while for reuse. All objects subject to pruning (next slide)
- 13. Resource pruning job • Important! Prevents your private cloud from filling up • Prunes instances, images, temporary git branches • Grace period: 4 Hours for instances, 12 hours for images • Keep “last known good image” • Objects with “dont_delete” tag are spared
- 14. Useful Openstack Cloud Features • Instance flavors – Different tests require different CPU, disk and memory – Optimize cloud utilization • Host aggregates – Guide instance placement. Example: nested virtualization requirements • Tagging – Build metadata for images (e.g. success, branch) – Protection from the pruning job
- 16. Backup Slides
- 17. Pod creation details • Spawn controller instance from DU image • Spawn M additional instances from plain OS images • Simulate customer installation of PF9 host agent • Automate deployment of openstack agents