SlideShare a Scribd company logo

Kubernetes and Gitops at Quicksign - Koncrete Kube Oct 2018

•
•
Cedric Vidal
Cedric Vidal

Quicksign embraced GitOps with Kubernetes early on. In this talk, I explained how we approached it and the challenges we faced.

1 of 22
Download to read offline
2018-10-16 Cédric Vidal, Quicksign CTO @cedricvidal Kubernetes and GitOps at Quicksign O
Who are we ? ● QuickSign is the European leader in digital onboarding for financial services ● White label ● Now handling millions of digital subscriptions per year 10 years experience 15 countries
70% annual growth in traffic The fo S l !
Traceability ● Regulated business ● Qualified signature ETSI 319 411 ● Be able to prove conformance ● Non-repudiation
Multi tenancy ● Different workloads ● SLAs ● Shared cluster for small customers ● Dedicated cluster for big customers
Kubernetes trend po M s ad K8S ob v K8S
GitOps ?? “Put simply, GitOps is the art and science of using Git pull requests to manage infrastructure provisioning and software deployment.” https://www.twistlock.com/2018/08/06/gitops-101-gitops-use/
GitOps benefits • One tool to rule them all • Version control for all changes • Easy diff between two moments • Covers also supporting infrastructure: monitoring and logging https://www.twistlock.com/2018/08/06/gitops-101-gitops-use/
GitOps security benefits • Many security workflows possible • Branch level ACLs • PR checks
GitOps overview Ex e t s a 1 Dif Com di 4 Pus d Sub P 1b Rev P 1c Sync 2 Pok Ob e v s a 3 Fet 5 Al e 4 ha s
Automatic deployment
Kubernetes is sweet pie for GitOps ;) • Everything is described ... as YAML, easy to version • Most state, can be dumped as YAML and “imported” / applied • Diff engine built-in (kubectl apply)
Custom Resource Definitions (since 1.7) • Your own Kubernetes resources ! yeah • Ex: a product configuration per customer (POCO) 1. public static resources -> GCS bucket / CDN 2. BPMN models -> deployed in our BPMN engine • “POCO deployer”: CRD Controller
POCO deployment workflow
Service labels, annotations and queries • Extended service registry • Overlay your own metadata • Query services by its metadata • We use it to specify the transport, data formats and protocol schemes of our BPMN tasks
GPU • GPU on GKE • Attached to VM • nvidia driver installed as daemonset • Only pods asking for GPUs as allocated to GPU nodes • More expensive • one GPU are not shareable to multiple pods :-(
Ingress • Ingress Nginx behind a GCE LB level 4 (TCP) • We don’t use the GCE level 7 load balancer 1. All services are down when a route is created … WTF ? 2. No http to https redirection
Outbound gateway • GCE VM with fixed IP • outbound traffic goes through that VM to have a fix outbound IP • required for external IP filtering Gateway of India, Mumbai, India
HPA (Horizontal Pod Autoscaler) • Sky is the limit in the Cloud • Automatically add replicas when under load • Stateless pods only for now (most are) • PDB (Pod Disruption Budget) : max unavailable = 1 • On GKE: node auto-scaling
Certificates • Let’s encrypt for everything but production • Wildcard certificates for production
Monitoring • Prometheus 1. alert-manager 2. kube-state-metrics 3. node-exporter 4. blackbox-exporter
Your turn! Cédric Vidal CTO @cedricvidal Multiple free photographies from unsplash.com

More Related Content

Kubernetes and Gitops at Quicksign - Koncrete Kube Oct 2018

  • 1. 2018-10-16 Cédric Vidal, Quicksign CTO @cedricvidal Kubernetes and GitOps at Quicksign O
  • 2. Who are we ? ● QuickSign is the European leader in digital onboarding for financial services ● White label ● Now handling millions of digital subscriptions per year 10 years experience 15 countries
  • 3. 70% annual growth in traffic The fo S l !
  • 4. Traceability ● Regulated business ● Qualified signature ETSI 319 411 ● Be able to prove conformance ● Non-repudiation
  • 5. Multi tenancy ● Different workloads ● SLAs ● Shared cluster for small customers ● Dedicated cluster for big customers
  • 6. Kubernetes trend po M s ad K8S ob v K8S
  • 7. GitOps ?? “Put simply, GitOps is the art and science of using Git pull requests to manage infrastructure provisioning and software deployment.” https://www.twistlock.com/2018/08/06/gitops-101-gitops-use/
  • 8. GitOps benefits • One tool to rule them all • Version control for all changes • Easy diff between two moments • Covers also supporting infrastructure: monitoring and logging https://www.twistlock.com/2018/08/06/gitops-101-gitops-use/
  • 9. GitOps security benefits • Many security workflows possible • Branch level ACLs • PR checks
  • 10. GitOps overview Ex e t s a 1 Dif Com di 4 Pus d Sub P 1b Rev P 1c Sync 2 Pok Ob e v s a 3 Fet 5 Al e 4 ha s
  • 12. Kubernetes is sweet pie for GitOps ;) • Everything is described ... as YAML, easy to version • Most state, can be dumped as YAML and “imported” / applied • Diff engine built-in (kubectl apply)
  • 13. Custom Resource Definitions (since 1.7) • Your own Kubernetes resources ! yeah • Ex: a product configuration per customer (POCO) 1. public static resources -> GCS bucket / CDN 2. BPMN models -> deployed in our BPMN engine • “POCO deployer”: CRD Controller
  • 15. Service labels, annotations and queries • Extended service registry • Overlay your own metadata • Query services by its metadata • We use it to specify the transport, data formats and protocol schemes of our BPMN tasks
  • 16. GPU • GPU on GKE • Attached to VM • nvidia driver installed as daemonset • Only pods asking for GPUs as allocated to GPU nodes • More expensive • one GPU are not shareable to multiple pods :-(
  • 17. Ingress • Ingress Nginx behind a GCE LB level 4 (TCP) • We don’t use the GCE level 7 load balancer 1. All services are down when a route is created … WTF ? 2. No http to https redirection
  • 18. Outbound gateway • GCE VM with fixed IP • outbound traffic goes through that VM to have a fix outbound IP • required for external IP filtering Gateway of India, Mumbai, India
  • 19. HPA (Horizontal Pod Autoscaler) • Sky is the limit in the Cloud • Automatically add replicas when under load • Stateless pods only for now (most are) • PDB (Pod Disruption Budget) : max unavailable = 1 • On GKE: node auto-scaling
  • 20. Certificates • Let’s encrypt for everything but production • Wildcard certificates for production
  • 21. Monitoring • Prometheus 1. alert-manager 2. kube-state-metrics 3. node-exporter 4. blackbox-exporter
  • 22. Your turn! Cédric Vidal CTO @cedricvidal Multiple free photographies from unsplash.com