SlideShare a Scribd company logo

A Blueprint for Scala Microservices

•
•
Federico Feroldi
Federico Feroldi

Building a microservices architecture means making a lot of decisions, about tools, about frameworks. In this talk I share the decisions that we made at Measurence during our journey for building a microservices architecture based on Scala technologies. We're going to talk about Spray, Akka, Swagger, Sbt, Docker, Jenkins, Mesos and Marathon.

Related slideshows

Testing Java Code Effectively - BaselOne17
Testing Java Code Effectively - BaselOne17Testing Java Code Effectively - BaselOne17
Testing Java Code Effectively - BaselOne17
 
PHP 8.1 - What's new and changed
PHP 8.1 - What's new and changedPHP 8.1 - What's new and changed
PHP 8.1 - What's new and changed
 
RESTful API using scalaz (3)
RESTful API using scalaz (3)RESTful API using scalaz (3)
RESTful API using scalaz (3)
 
1 of 46
Download to read offline
A BLUEPRINT FOR SCALA MICROSERVICES FEDERICO FEROLDI CTO / MEASURENCE.COM @CLOUDIFY
OUR GOAL Empower developers with tools and process to own a service from development to production.
Monolithic (1990s) SOA (2000s) Microservices (2010s)
RATIONALE Small team: time is scarce Heavy use of open source Automate al the things Sane standards, easy override
A Blueprint for Scala Microservices
MAIN TOPICS Minimum Viable Service Deployment & Discovery Best practices
MINIMUM VIABLE SERVICE Business Logic REST API Configuration Health / Monitoring Releasing / Versioning Packaging
BUSINESS LOGIC = AKKA ACTORS Sum
 Service HTTP API SumOp SumRes “ask” pattern
BUSINESS LOGIC = AKKA ACTORS object SumService { def props = Props(classOf[SumService]) ! case class SumOp(a: Int, b: Int) case class SumRes(result: Int) } ! class SumService extends Actor { import SumService._ ! def receive: Receive = { case SumOp(a, b) => sender() ! SumRes(a + b) } }
REST API = SPRAY.IO Sum
 Service SPRAY API SumOp SumRes GET /v1/sum JSON
REST API = SPRAY.IO case class SumApiResponse(result: Int) ! object SumProtocol extends DefaultJsonProtocol { implicit val sumApiResponseFormat = jsonFormat1(SumApiResponse) } ! class SumHttpServiceV1(services: Services) { import SumProtocol._ ! def route = path("v1" / "sum") { get { parameters('a.as[Int], 'b.as[Int]) { (a, b) => def future = (services.sumService ? SumOp(a, b)).mapTo[SumRes] onSuccess(future) { response => complete(SumApiResponse(response.result)) } } } } }
API DOCS = SWAGGER SPRAY- SWAGGER GET / HTML/JSON docs
API DOCS = SWAGGER @Api(value = “/v1/sum", description = "Sum numbers.") class SumHttpServiceV1(services: Services) { … }
API DOCS = SWAGGER ! @ApiOperation(value = "Returns the sum”, httpMethod = “GET") ! @ApiImplicitParams(Array( new ApiImplicitParam( name="a", required=true, dataType="integer", paramType=“query" ), new ApiImplicitParam( name="b", required=true, dataType="integer", paramType=“query" ) )) ! @ApiResponses(Array( new ApiResponse( code=200, message="The sum", response=classOf[SumApiResponse] ) )) ! def route = path("v1" / "sum") { … }
API DOCS = SWAGGER @ApiModel(description = "Result of the sum") case class SumApiResponse( @(ApiModelProperty @field)(value = "The sum of a and b") result: Int )
REST API = SPRAY.IO + SWAGGER
CONFIG = TYPESAFE CONFIG application.conf development.conf testing.conf staging.conf production.conf
CONFIG = TYPESAFE CONFIG akka.loglevel = "INFO" ! metrics.carbon = "graphite.local:2003" ! acme { environment = "production" ! svc-calculator { hostname = “0.0.0.0” port = "9999" } } application.conf
CONFIG = TYPESAFE CONFIG include "application" production.conf
CONFIG = TYPESAFE CONFIG include "application"
 ! akka.loglevel = "DEBUG" metrics.carbon = "localhost:2003" ! acme { environment = "development" ! svc-calculator { hostname = "127.0.0.1" } } development.conf
CONFIG = TYPESAFE CONFIG val config = ConfigFactory.defaultApplication() $ sbt -Dconfig.resource=development.conf run Loads application.conf by default Loads development.conf
CONFIG = TYPESAFE CONFIG { mysql_username=${MYSQL_USERNAME} mysql_password=${MYSQL_PASSWORD} } PRO-TIP
 Don’t store passwords with source code! Instead make use ENV vars substitution and keep passwords in a safe place.
HEALTH = SPRAY + HAPROXY SPRAY GET /ping Load Balancer Are you ok?
HEALTH = SPRAY + HAPROXY def route = get { path("ping") { complete("pong") } }
PERFMON = METRICS + GRAPHITE CODAHALE METRICS data points GRAPHITE CARBON
PERF.MON. = METRICS + GRAPHITE object MetricsInstrumentation { val metricRegistry = new MetricRegistry() } ! trait Instrumented extends InstrumentedBuilder { val metricRegistry = MetricsInstrumentation.metricRegistry } ! class Actor extends Actor with Instrumented { val meter = metrics.meter("requests") ! def receive: Receive = { ! case Request => meter.mark() ! } }
PERFMON = METRICS + GRAPHITE
RELEASING = SBT-RELEASE SNAPSHOT/RELEASE artifacts Semantic versioning version.sbt + Git tags publishing of artifacts 100% customizable process interactive OR automated
PACKAGING = ASSEMBLY + DOCKER JVM SERVICE (fat jar) HTTP Requests METRICS “Immutable” container Other Svc HTTP Reqs
PACKAGING = SBT-ASSEMBLY + SBT-DOCKER docker <<= (docker dependsOn assembly) ! dockerfile in docker := { val artifact = (outputPath in assembly).value val artifactTargetPath = s"/app/${artifact.name}" new Dockerfile { from(“acme/javabase") expose(9999) add(artifact, artifactTargetPath) entryPoint("java", "-jar", artifactTargetPath) } } ! imageNames in docker := Seq( ImageName( namespace = Some(organization.value), repository = name.value, tag = Some("v" + version.value) ) )
MINIMUM VIABLE SERVICE Business Logic = Akka REST API = Spray + Swagger Configuration = Typesafe Config Health / Monitoring = Metrics Packaging = Assembly & Docker
DEPLOYMENT & DISCOVERY Continuous Integration Deployment Service Discovery
CI/DEPLOY = JENKINS + SBT + ANSIBLE Git repo Jenkins Jobs + sbt Test Release Dockerize Deploy Commit Staging / Production Ansible
HOSTING = MESOS/MARATHON
{ "id": "/svc-calculator", "container": { "type": "DOCKER", "docker": { "image": "docker.com/svc-calculator:1.0.0", "network": "BRIDGE", "portMappings": [{ "containerPort": 9999, "protocol": "tcp" }] } }, "env": {"JAVA_ARGS": “-Dconfig.resource=production"}, "cpus": 1, "mem": 1024, "instances": 2 } HOSTING = MESOS/MARATHON
HOSTING = MESOS/MARATHON Dynamic port mapping
DISCOVERY = ? mesos1 /svc-calculator /svc-calculator mesos2 /web-app mesos3 HOST:PORT?
DISCOVERY = BAMBOO + HAPROXY mesos1 /svc-calculator /svc-calculator mesos2 /web-app mesos3 BAMBOO + HAPROXY BAMBOO + HAPROXY BAMBOO + HAPROXY MARATHON http://localhost/svc-calculator :80 :30001:30002
DEPLOYMENT & DISCOVERY CI = Jenkins + sbt Deploy = Ansible + Marathon Discovery = Bamboo + HAProxy
FINAL TIPS DRY + keep team aligned Zero friction on new services
DRY: CUSTOM SBT PLUGIN package com.acme.sbt.plugin import sbt._ import Keys._ object AcmePlugin extends AutoPlugin { lazy val acmeBuildSettings = Seq( scalacOptions ++= Seq( "-unchecked", "-deprecation", "-feature", "-Xlint", "-Ywarn-dead-code", "-target:jvm-1.7" ) ) } AcmePlugin.scala
FINAL TIP: CUSTOM SBT PLUGIN import com.acme.sbt.plugin.AcmePlugin._ ! Seq(acmeBuildSettings:_*) ! // custom project settings build.sbt
ZERO FRICTION: GITER8 TEMPLATES % g8 file://g8-svc-spray/ --name=svc-calculator -—servicePort=9999 ! Template applied in ./svc-calculator ! % ls svc-calculator build.sbt project src version.sbt
Recap
A Blueprint for Scala Microservices
Thank you!

More Related Content

A Blueprint for Scala Microservices

  • 1. A BLUEPRINT FOR SCALA MICROSERVICES FEDERICO FEROLDI CTO / MEASURENCE.COM @CLOUDIFY
  • 2. OUR GOAL Empower developers with tools and process to own a service from development to production.
  • 3. Monolithic (1990s) SOA (2000s) Microservices (2010s)
  • 4. RATIONALE Small team: time is scarce Heavy use of open source Automate al the things Sane standards, easy override
  • 6. MAIN TOPICS Minimum Viable Service Deployment & Discovery Best practices
  • 7. MINIMUM VIABLE SERVICE Business Logic REST API Configuration Health / Monitoring Releasing / Versioning Packaging
  • 8. BUSINESS LOGIC = AKKA ACTORS Sum
 Service HTTP API SumOp SumRes “ask” pattern
  • 9. BUSINESS LOGIC = AKKA ACTORS object SumService { def props = Props(classOf[SumService]) ! case class SumOp(a: Int, b: Int) case class SumRes(result: Int) } ! class SumService extends Actor { import SumService._ ! def receive: Receive = { case SumOp(a, b) => sender() ! SumRes(a + b) } }
  • 10. REST API = SPRAY.IO Sum
 Service SPRAY API SumOp SumRes GET /v1/sum JSON
  • 11. REST API = SPRAY.IO case class SumApiResponse(result: Int) ! object SumProtocol extends DefaultJsonProtocol { implicit val sumApiResponseFormat = jsonFormat1(SumApiResponse) } ! class SumHttpServiceV1(services: Services) { import SumProtocol._ ! def route = path("v1" / "sum") { get { parameters('a.as[Int], 'b.as[Int]) { (a, b) => def future = (services.sumService ? SumOp(a, b)).mapTo[SumRes] onSuccess(future) { response => complete(SumApiResponse(response.result)) } } } } }
  • 12. API DOCS = SWAGGER SPRAY- SWAGGER GET / HTML/JSON docs
  • 13. API DOCS = SWAGGER @Api(value = “/v1/sum", description = "Sum numbers.") class SumHttpServiceV1(services: Services) { … }
  • 14. API DOCS = SWAGGER ! @ApiOperation(value = "Returns the sum”, httpMethod = “GET") ! @ApiImplicitParams(Array( new ApiImplicitParam( name="a", required=true, dataType="integer", paramType=“query" ), new ApiImplicitParam( name="b", required=true, dataType="integer", paramType=“query" ) )) ! @ApiResponses(Array( new ApiResponse( code=200, message="The sum", response=classOf[SumApiResponse] ) )) ! def route = path("v1" / "sum") { … }
  • 15. API DOCS = SWAGGER @ApiModel(description = "Result of the sum") case class SumApiResponse( @(ApiModelProperty @field)(value = "The sum of a and b") result: Int )
  • 16. REST API = SPRAY.IO + SWAGGER
  • 17. CONFIG = TYPESAFE CONFIG application.conf development.conf testing.conf staging.conf production.conf
  • 18. CONFIG = TYPESAFE CONFIG akka.loglevel = "INFO" ! metrics.carbon = "graphite.local:2003" ! acme { environment = "production" ! svc-calculator { hostname = “0.0.0.0” port = "9999" } } application.conf
  • 19. CONFIG = TYPESAFE CONFIG include "application" production.conf
  • 20. CONFIG = TYPESAFE CONFIG include "application"
 ! akka.loglevel = "DEBUG" metrics.carbon = "localhost:2003" ! acme { environment = "development" ! svc-calculator { hostname = "127.0.0.1" } } development.conf
  • 21. CONFIG = TYPESAFE CONFIG val config = ConfigFactory.defaultApplication() $ sbt -Dconfig.resource=development.conf run Loads application.conf by default Loads development.conf
  • 22. CONFIG = TYPESAFE CONFIG { mysql_username=${MYSQL_USERNAME} mysql_password=${MYSQL_PASSWORD} } PRO-TIP
 Don’t store passwords with source code! Instead make use ENV vars substitution and keep passwords in a safe place.
  • 23. HEALTH = SPRAY + HAPROXY SPRAY GET /ping Load Balancer Are you ok?
  • 24. HEALTH = SPRAY + HAPROXY def route = get { path("ping") { complete("pong") } }
  • 25. PERFMON = METRICS + GRAPHITE CODAHALE METRICS data points GRAPHITE CARBON
  • 26. PERF.MON. = METRICS + GRAPHITE object MetricsInstrumentation { val metricRegistry = new MetricRegistry() } ! trait Instrumented extends InstrumentedBuilder { val metricRegistry = MetricsInstrumentation.metricRegistry } ! class Actor extends Actor with Instrumented { val meter = metrics.meter("requests") ! def receive: Receive = { ! case Request => meter.mark() ! } }
  • 27. PERFMON = METRICS + GRAPHITE
  • 28. RELEASING = SBT-RELEASE SNAPSHOT/RELEASE artifacts Semantic versioning version.sbt + Git tags publishing of artifacts 100% customizable process interactive OR automated
  • 29. PACKAGING = ASSEMBLY + DOCKER JVM SERVICE (fat jar) HTTP Requests METRICS “Immutable” container Other Svc HTTP Reqs
  • 30. PACKAGING = SBT-ASSEMBLY + SBT-DOCKER docker <<= (docker dependsOn assembly) ! dockerfile in docker := { val artifact = (outputPath in assembly).value val artifactTargetPath = s"/app/${artifact.name}" new Dockerfile { from(“acme/javabase") expose(9999) add(artifact, artifactTargetPath) entryPoint("java", "-jar", artifactTargetPath) } } ! imageNames in docker := Seq( ImageName( namespace = Some(organization.value), repository = name.value, tag = Some("v" + version.value) ) )
  • 31. MINIMUM VIABLE SERVICE Business Logic = Akka REST API = Spray + Swagger Configuration = Typesafe Config Health / Monitoring = Metrics Packaging = Assembly & Docker
  • 32. DEPLOYMENT & DISCOVERY Continuous Integration Deployment Service Discovery
  • 33. CI/DEPLOY = JENKINS + SBT + ANSIBLE Git repo Jenkins Jobs + sbt Test Release Dockerize Deploy Commit Staging / Production Ansible
  • 35. { "id": "/svc-calculator", "container": { "type": "DOCKER", "docker": { "image": "docker.com/svc-calculator:1.0.0", "network": "BRIDGE", "portMappings": [{ "containerPort": 9999, "protocol": "tcp" }] } }, "env": {"JAVA_ARGS": “-Dconfig.resource=production"}, "cpus": 1, "mem": 1024, "instances": 2 } HOSTING = MESOS/MARATHON
  • 37. DISCOVERY = ? mesos1 /svc-calculator /svc-calculator mesos2 /web-app mesos3 HOST:PORT?
  • 38. DISCOVERY = BAMBOO + HAPROXY mesos1 /svc-calculator /svc-calculator mesos2 /web-app mesos3 BAMBOO + HAPROXY BAMBOO + HAPROXY BAMBOO + HAPROXY MARATHON http://localhost/svc-calculator :80 :30001:30002
  • 39. DEPLOYMENT & DISCOVERY CI = Jenkins + sbt Deploy = Ansible + Marathon Discovery = Bamboo + HAProxy
  • 40. FINAL TIPS DRY + keep team aligned Zero friction on new services
  • 41. DRY: CUSTOM SBT PLUGIN package com.acme.sbt.plugin import sbt._ import Keys._ object AcmePlugin extends AutoPlugin { lazy val acmeBuildSettings = Seq( scalacOptions ++= Seq( "-unchecked", "-deprecation", "-feature", "-Xlint", "-Ywarn-dead-code", "-target:jvm-1.7" ) ) } AcmePlugin.scala
  • 42. FINAL TIP: CUSTOM SBT PLUGIN import com.acme.sbt.plugin.AcmePlugin._ ! Seq(acmeBuildSettings:_*) ! // custom project settings build.sbt
  • 43. ZERO FRICTION: GITER8 TEMPLATES % g8 file://g8-svc-spray/ --name=svc-calculator -—servicePort=9999 ! Template applied in ./svc-calculator ! % ls svc-calculator build.sbt project src version.sbt
  • 44. Recap