FIWARE Tech Summit - Docker Swarm Secrets for Creating Great FIWARE Platforms

macvlan and ipvlan allow VMs and containers to have direct exposure to the host network by assigning them their own MAC/IP addresses without requiring a bridge. macvlan uses MAC addresses to separate traffic while ipvlan uses layer 3. Both are lighter weight than bridges. macvlan is commonly used in bridge mode to allow communication between VMs/containers on the same host, while ipvlan may be preferred when MAC limits are in place or for untrusted networks.

Docker networking provides a networking fabric for containers called libnetwork that defines the container networking model and provides features like multi-host networking, service discovery, load balancing, and security. New features in Docker 1.12 include networking in swarm mode without an external key-value store, macvlan driver support, a gossip-based secure control plane, optional IPSec for the data plane, built-in DNS for service discovery and load balancing, and a routing mesh for edge routing.

OSv is a new, high-performance OS for virtual machines in the cloud. Designed to run one application per guest with minimal overhead, OSv eliminates important bottlenecks for NoSQL applications through improvements in memory management, network I/O, and scheduling. And many important bottlenecks for NoSQL applications are tunable on a conventional OS, but do not require tuning in the OSv environment. OSv is fully stateless and can be configured at runtime with cloud-init or through a REST API, with zero configuration files. OSv offers unified tracing from the application layer through the JVM and the OS kernel. Attendees will learn how to boot Cassandra in one second, and create a simple cluster in a minute.

The document discusses the evolution of XenServer architecture to address scalability limitations. The current architecture works well now but will hit bottlenecks on larger servers. The new "Windsor" architecture uses domain 0 disaggregation to move virtualization functions out of domain 0 and into separate domains for improved performance, scalability, and isolation. Key benefits include better VM density, use of hardware resources, stability, availability, and extensibility. It provides a flexible platform that can scale-out across servers.

This presentation, DEFEATING THE NETWORK SECURITY INFRASTRUCTURE v1.0.pdf, was made after some brainstorming with some friends. The techniques used are not new and the tools readily available for download. The purpose of the discussion however is to debate how internal enterprise resources might be (in)adversely exposed to the internet by in an insider using a combination of common techniques such as SSH and SSL.

Xen currently has two major mechanisms to maintain security while hosting untrusted VMs without causing disruption to those guests: live patching, and live migration. We introduce a third method: live updating Xen. A live-update operation involves loading of the newly-staged hypervisor into RAM, the currently-running Xen serializing its state, and then transferring control to the newly-staged Xen, all without disrupting running instances, beyond a little downtime when neither hypervisor is running guest vCPUs. We present a proposal on the design of such a feature, and invite comments and feedback.

This document discusses Docker Swarm Mode, which allows managing a cluster of Docker Engines called a swarm. Key features include integrated cluster management, declarative application deployment across nodes, automatic scaling, service discovery, and encryption between nodes. The document demonstrates initializing a swarm on two VMs, adding a worker node, deploying services in replicated and global modes, attaching services to networks, publishing service ports, updating services, and more swarm commands.

Talk from Docker SF Meetup #50 Abstract: Docker swarm mode enables users to manage their applications with service primitives. In this talk we demonstrate how to do service upgrades without impacting your application. The Healthcheck feature provides health indication for a container. Coming up in Docker 1.13 release, Docker Swarm can connect healthcheck result with load balancer to implement no-loss service upgrade. Speaker Biographies: Nishant Totla is a software engineer at Docker, and works on the core open source team. He is currently working on Docker SwarmKit and Docker Swarm. Prior to Docker, he was a PhD student at UC Berkeley, doing research on programming languages. In his spare time, he enjoys long-distance running, biking, and other outdoor activities. Nishant tweets at @nishanttotla. Dongluo Chen is a software engineer at Docker focusing on orchestration and container development. Before Docker he was software engineer manager at Microsoft Azure building and automating global data centers. He worked at France Telecom (Orange) and the Ohio State University as research scientist in networking area.

Heart of the SwarmKit: Store, Topology & Object Model by Aaron, Andrea, Stephen D (Docker) Swarmkit repo - https://github.com/docker/swarmkit Liveblogging: http://canopy.mirage.io/Liveblog/SwarmKitDDS2016

From the Docker London MeetUp, presented on 27th June 2016. A walkthrough of Swarm Mode in Docker 1.12, the presentation introduces demos for creating a Docker Swarm using Azure virtual machines, and running a distributed application with a Node REST API, feeding analytics into Elasticsearch via a Redis queue.

Docker 1.12 introduces several new features for managing containerized applications at scale including Docker Swarm mode for native clustering and orchestration. Key features include services that allow defining and updating distributed applications, a built-in routing mesh for load balancing between nodes, and security improvements like cryptographic node identities and TLS encryption by default. The document also discusses plugins, health checks, and distributed application bundles for declaring stacks of services.

The document discusses Kubernetes networking concepts including pods, services, and ingress. It provides examples of how containers within pods communicate via Docker networking. It also explains how Kubernetes networking solves the problems of pod-to-pod, service-to-pod, and external-to-service communications using services, iptables, and kube-proxy. The document demonstrates creating a deployment, service, and ingress to expose an application externally via a load balancer.

With the announcement of the OCI by Solomon Hykes at last summer's DockerCon, a Docker-contributed reference implementation of the OCI spec, called runC, was born. While some of you may have tried runC or have a history of poking at the OS layer integration library to Linux namespaces, cgroups and the like (known as libcontainer), many of you may not know what runC offers. In this talk Phil Estes, Docker engine maintainer who has also contributed to libcontainer and runC, will show what's possible using runC as a lightweight and fast runtime environment to experiment with lower-level features of the container runtime. Phil will introduce a conversion tool called "riddler", which can inspect and convert container configurations from Docker into the proper OCI configuration bundle for easy conversion between the two environments. He'll also demonstrate how to make custom configurations for trying out security features like user namespaces and seccomp profiles.

This document describes a hands-on CloudStack workshop held on June 24th, 2015 in Amsterdam. It provides instructions on setting up a CloudStack development environment on KVM hypervisors to deploy a CloudStack management server and XenServer hosts. Attendees are guided through compiling CloudStack from source, deploying a sample data center configuration, adding VM templates, launching their first VM, exploring the CloudStack API, and using CloudStackOps for operations tasks. Exercises are suggested for working with tenants, networks, VPC, high availability, and observing hypervisor failures.

Libnetwork provides a native Go implementation for connecting containers The goal of libnetwork is to deliver a robust Container Network Model that provides a consistent programming interface and the required network abstractions for applications.

Docker Engine 1.12 can be rightly called ” A Next Generation Docker Clustering & Distributed System”. Though Docker Engine 1.12 Final Release is around corner but the recent RC3 brings lots of improvements and exciting features. One of the major highlight of this release is Docker Swarm Mode which provides powerful yet optional ability to create coordinated groups of decentralized Docker Engines. Swarm Mode combines your engine in swarms of any scale. It’s self-organizing and self-healing. It enables infrastructure-agnostic topology.The newer version democratizes orchestration with out-of-box capabilities for multi-container on multi-host app deployments.

This document discusses adding context support to the NATS client library. It begins with an overview of NATS and context in Go. It then describes enhancing the Subscription.NextMsg method to support context, avoiding blocking indefinitely. This allows building a RequestWithContext method for cancelling requests. The key steps are selecting on the subscription channel or context being done, and returning the context's error on cancellation. Learning from standard library patterns like validating contexts helps make the API clearer.

This presentation has been given during DevOps Congress in Wrocław. It is about container orchestrations with Docker Swarm and Traefik. It includes the configuration examples of Traefik version 2.0. The source code used in the presentation: https://github.com/jakubhajek/traefik-swarm

Do any VM's contain a particular indicator of compromise? E.g. Run a YARA signature over all executables on my virtual machines and tell me which ones match.

My talk in Bessemer VP R&D / CTO yearly event (Jan 2020). The presentation discusses major concept in resilience testing and MyHeritage's path to Chaos Engineering.

présentation de l'utilisation de Docker, du niveau 0 "je joue avec sur mon poste" au niveau Docker Hero "je tourne en prod". Ce talk fait suite à l'intro de @dgageot et ne comporte donc pas l'intro "c'est quoi Docker ?".

Dr. Piyush Harsh discusses using Docker to streamline the testing environment for Cyclops, an open-source rating, charging, and billing microservices framework. Key benefits of Docker include predictable, standardized environment setup and faster response times compared to virtual machine-based deployments. Some challenges included port conflicts and ensuring service start order, but these were easy to address. A demo of Cyclops integrating with external components for a telecom network virtualization use case is also mentioned.

The document discusses strategies for scaling LAMP applications on cloud computing platforms like AWS. It recommends: 1) Moving static files to scalable services like S3 and using a CDN to distribute load. 2) Using dedicated caching systems like Memcache instead of local caches and storing sessions in Memcache or DynamoDB for scalability. 3) Scaling databases horizontally using master-slave replication or sharding across multiple availability zones for high availability and read scaling. 4) Leveraging auto-scaling and load balancing on AWS with tools like Elastic Load Balancers, CloudWatch, and scaling alarms to dynamically scale application instances based on metrics.

This document discusses microservices architecture compared to a monolithic architecture. A microservices architecture breaks an application into smaller, independent services that each perform discrete functions. This allows for more rapid development and improved scalability. However, a microservices architecture is also more complex to deploy and manage. The document provides an example of how a VoIP application could use a microservices approach by breaking components like billing, fraud detection, and call analytics into separate services. It also discusses using Docker containers and services to deploy and scale the microservices architecture.

This document provides an overview of Container as a Service (CaaS) with Docker. It discusses key concepts like Docker containers, images, and orchestration tools. It also covers DevOps practices like continuous delivery that are enabled by Docker. Specific topics covered include Docker networking, volumes, and orchestration with Docker Swarm and compose files. Examples are provided of building and deploying Java applications with Docker, including Spring Boot apps, Java EE apps, and using Docker for builds. Security features of Docker like content trust and scanning are summarized. The document concludes by discussing Docker use cases across different industries and how Docker enables critical transformations around cloud, DevOps, and application modernization.

This presentation overviews basic principles of high availability architectures and presents how to deploy in high availability FIWARE data management services.

Thomas and James from Demonware discussed their company's evolution to using containers for testing. They started with "fat containers" containing all services, then separated services into containers defined in YAML files. Now they use Docker Swarm to run tests in scalable services across a cluster, addressing issues like limited resources and test parallelization. The talk provided an example of optimizing a testing pipeline through containerization.