CloudStack, jclouds, Jenkins and CloudCat
- 2. What is jclouds?
• Portable Java API for many cloud APIs and
providers
• Compute and Blobstore
• Lower-level Java API for cloud-specific APIs
as well
• Usable with any JVM language
Saturday, December 1, 2012
- 3. CloudStack and jclouds
• CloudStack is supported by jclouds
• 2.x and 3.x tested regularly
• 4.x hasn’t been tested much yet, but
minimal API changes from 3.x
Saturday, December 1, 2012
- 4. Standard jclouds API
and CloudStack
• CloudStack support for jcloud standard
API:
• Instance creation/management
• Listing available “hardware” (service
offerings), “images” (templates)
• Security groups
Saturday, December 1, 2012
- 5. Lower-level CloudStack
API with jclouds
• Pretty much everything the CloudStack API
can do is implemented in jclouds
• Administration tasks
• Template creation
• User management
• Much, much more
Saturday, December 1, 2012
- 6. What can you do with
jclouds and
CloudStack?
Saturday, December 1, 2012
- 7. A lot.
Here are some
examples.
Saturday, December 1, 2012
- 8. jclouds Jenkins plugin
• Dynamically provision Jenkins build slaves
from CloudStack (or any other supported
cloud)
• Much like the Jenkins ec2 plugin, but more
features and obviously more clouds
Saturday, December 1, 2012
- 9. How the Jenkins plugin
works
• Uses standard jclouds API for instance
creation and destruction
• Allows you to choose “image” or OS
family/version, and “hardware” (from list of
available) or minimum CPU/RAM
• Uses underlying clouds’ APIs to find the
right image/hardware for the given input
Saturday, December 1, 2012
- 15. Single use slave
• The single use slave option lets you use a
slave once and only once
• Slave is marked offline after job completes,
guaranteeing the next time that job runs, it
gets a fresh slave
Saturday, December 1, 2012
- 16. Jenkins Instance
Creation
• Create instances at beginning of build, tear
down instances at end of build
• Use clouds and templates available as slaves
• Exposes IPs of instances to build steps
• Useful for testing clusters, etc
Saturday, December 1, 2012
- 17. Building an app using
CloudStack and jclouds
Saturday, December 1, 2012
- 18. CloudCat
• Open source, Grails-based reporting and
provisioning application for CloudStack
• github.com/abayer/cloudcat
Saturday, December 1, 2012
- 19. Using Lower-Level API
• CloudStack admin information (accounts,
hosts, etc) not available through jclouds
standard API
• So we’ve got to use the CloudStack API
itself - which we can do through jclouds!
• Same jclouds connection whether using
standard API or CloudStack-specific
Saturday, December 1, 2012
- 21. Mapping CloudStack
Data
• jclouds has objects for data from
CloudStack API
• VirtualMachine, Template, ServiceOffering,
etc
• In CloudCat, we map those objects to
Grails domain classes
Saturday, December 1, 2012
- 23. Importing CloudStack
data into Grails
• Run API commands as admin
• Persist results into database once an hour,
rather than querying API live
• Pull credentials and endpoint from config
domain object
• Use jclouds’ *Clients to get lists of hosts,
templates, service offerings, instances
Saturday, December 1, 2012
- 25. Associating templates,
instances, etc
• We want to know how instances,
templates, service offerings and hosts are
related
• We want to keep historical information on
instances, and know what instances were
still existent the last time we got reported
data
Saturday, December 1, 2012
- 27. Alert Notification
• Email notification when CloudStack has
problems - not available in CloudStack
itself, so far as I can tell.
• Events and alerts are available from the API
- check periodically for new alerts, send
email when new ones are seen.
Saturday, December 1, 2012
- 30. Provisioning through
CloudCat
• CloudStack UI for instance creation doesn’t
make it easy to create a bunch of
otherwise identical instances
• Hence, CloudCat provisioned instance
groups
Saturday, December 1, 2012
- 31. jclouds standard API or
CloudStack-specific?
• jclouds provides instance creation in a
standard way across all clouds, including
CloudStack
• But the standard instance creation makes a
lot of largely redundant API calls, especially
to guarantee instance name uniqueness
Saturday, December 1, 2012
- 32. jclouds standard API or
CloudStack-specific?
• We can guarantee uniqueness within
CloudCat
• Using lower level CloudStack API (still
through jclouds) improves performance
considerably
• Especially since we’re doing API calls as
admin user
Saturday, December 1, 2012
- 33. Speaking of accounts...
• How do we create instances for a logged-in
user while making API calls as admin? Two
options:
• Use account and domainid options for
deployVirtualMachine
• Fetch account’s keys and make API call
using those keys
Saturday, December 1, 2012
- 34. Speaking of accounts...
• After experimentation, I went with using
the logged-in user’s account’s keys
• Less API traffic when run as logged-in user
Saturday, December 1, 2012
- 35. Grails
• Groovy
• Rails-like
• I’m lazy and bad with UIs, so I like getting
sortable tables, MVC, and much more for
free, with even more available from plugins!
Saturday, December 1, 2012
- 36. So how does this all
look?
Saturday, December 1, 2012
- 49. Known Issues
• I’m not a very good programmer!
• Only really tested with basic networking
• Init script requires hardcoded credentials
on instances
• Needs work on packaging/installation
• Only one user so far!
Saturday, December 1, 2012
- 50. More Information
• jclouds: http://jclouds.org
• jclouds Jenkins plugin: https://wiki.jenkins-
ci.org/display/JENKINS/JClouds+Plugin
• CloudCat
• source: http://github.com/abayer/cloudcat
• demo: http://cloudcat.andrewbayer.com
Saturday, December 1, 2012