HBase and HDFS: Understanding FileSystem Usage in HBaseenissoz
This document discusses file system usage in HBase. It provides an overview of the three main file types in HBase: write-ahead logs (WALs), data files, and reference files. It describes durability semantics, IO fencing techniques for region server recovery, and how HBase leverages data locality through short circuit reads, checksums, and block placement hints. The document is intended help understand HBase's interactions with HDFS for tuning IO performance.
Kvm performance optimization for ubuntuSim Janghoon
This document discusses various techniques for optimizing KVM performance on Linux systems. It covers CPU and memory optimization through techniques like vCPU pinning, NUMA affinity, transparent huge pages, KSM, and virtio_balloon. For networking, it discusses vhost-net, interrupt handling using MSI/MSI-X, and NAPI. It also covers block device optimization through I/O scheduling, cache mode, and asynchronous I/O. The goal is to provide guidance on configuring these techniques for workloads running in KVM virtual machines.
Ceph scale testing with 10 Billion ObjectsKaran Singh
In this performance testing, we ingested 10 Billion objects into the Ceph Object Storage system and measured its performance. We have observed deterministic performance, check out this presentation to know the details.
There are many ways to run high availability with PostgreSQL. Here, we present a template for you to create your own customized, high-availability solution using Python and for maximum accessibility, a distributed configuration store like ZooKeeper or etcd.
New Ways to Find Latency in Linux Using TracingScyllaDB
Ftrace is the official tracer of the Linux kernel. It originated from the real-time patch (now known as PREEMPT_RT), as developing an operating system for real-time use requires deep insight and transparency of the happenings of the kernel. Not only was tracing useful for debugging, but it was critical for finding areas in the kernel that was causing unbounded latency. It's no wonder why the ftrace infrastructure has a lot of tooling for seeking out latency. Ftrace was introduced into mainline Linux in 2008, and several talks have been done on how to utilize its tracing features. But a lot has happened in the past few years that makes the tooling for finding latency much simpler. Other talks at P99 will discuss the new ftrace tracers "osnoise" and "timerlat", but this talk will focus more on the new flexible and dynamic aspects of ftrace that facilitates finding latency issues which are more specific to your needs. Some of this work may still be in a proof of concept stage, but this talk will give you the advantage of knowing what tools will be available to you in the coming year.
HBase and HDFS: Understanding FileSystem Usage in HBaseenissoz
This document discusses file system usage in HBase. It provides an overview of the three main file types in HBase: write-ahead logs (WALs), data files, and reference files. It describes durability semantics, IO fencing techniques for region server recovery, and how HBase leverages data locality through short circuit reads, checksums, and block placement hints. The document is intended help understand HBase's interactions with HDFS for tuning IO performance.
Kvm performance optimization for ubuntuSim Janghoon
This document discusses various techniques for optimizing KVM performance on Linux systems. It covers CPU and memory optimization through techniques like vCPU pinning, NUMA affinity, transparent huge pages, KSM, and virtio_balloon. For networking, it discusses vhost-net, interrupt handling using MSI/MSI-X, and NAPI. It also covers block device optimization through I/O scheduling, cache mode, and asynchronous I/O. The goal is to provide guidance on configuring these techniques for workloads running in KVM virtual machines.
Ceph scale testing with 10 Billion ObjectsKaran Singh
In this performance testing, we ingested 10 Billion objects into the Ceph Object Storage system and measured its performance. We have observed deterministic performance, check out this presentation to know the details.
There are many ways to run high availability with PostgreSQL. Here, we present a template for you to create your own customized, high-availability solution using Python and for maximum accessibility, a distributed configuration store like ZooKeeper or etcd.
New Ways to Find Latency in Linux Using TracingScyllaDB
Ftrace is the official tracer of the Linux kernel. It originated from the real-time patch (now known as PREEMPT_RT), as developing an operating system for real-time use requires deep insight and transparency of the happenings of the kernel. Not only was tracing useful for debugging, but it was critical for finding areas in the kernel that was causing unbounded latency. It's no wonder why the ftrace infrastructure has a lot of tooling for seeking out latency. Ftrace was introduced into mainline Linux in 2008, and several talks have been done on how to utilize its tracing features. But a lot has happened in the past few years that makes the tooling for finding latency much simpler. Other talks at P99 will discuss the new ftrace tracers "osnoise" and "timerlat", but this talk will focus more on the new flexible and dynamic aspects of ftrace that facilitates finding latency issues which are more specific to your needs. Some of this work may still be in a proof of concept stage, but this talk will give you the advantage of knowing what tools will be available to you in the coming year.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
1. DPDK achieves high throughput packet processing on commodity hardware by reducing kernel overhead through techniques like polling, huge pages, and userspace drivers.
2. In Linux, packet processing involves expensive operations like system calls, interrupts, and data copying between kernel and userspace. DPDK avoids these by doing all packet processing in userspace.
3. DPDK uses techniques like isolating cores for packet I/O threads, lockless ring buffers, and NUMA awareness to further optimize performance. It can achieve throughput of over 14 million packets per second on 10GbE interfaces.
This document provides a summary of improvements made to Hive's performance through the use of Apache Tez and other optimizations. Some key points include:
- Hive was improved to use Apache Tez as its execution engine instead of MapReduce, reducing latency for interactive queries and improving throughput for batch queries.
- Statistics collection was optimized to gather column-level statistics from ORC file footers, speeding up statistics gathering.
- The cost-based optimizer Optiq was added to Hive, allowing it to choose better execution plans.
- Vectorized query processing, broadcast joins, dynamic partitioning, and other optimizations improved individual query performance by over 100x in some cases.
Many enterprise are implementing Hadoop projects to manage and process large datasets. Big question is: how to configure Hadoop clusters to connect to enterprise directory containing 100k+ users and groups for access management. Several large enterprises have complex directory servers for managing users and groups. Many advanced features have been recently added to Hadoop user management in order to support various complex directory server structures.
In this session attendees will learn about: setting up Hadoop node with users from Active Directory for executing Hadoop jobs, setting up authentication for enterprise users, and setting up authorization for users and groups using Apache Ranger. Attendees will also learn about the common challenges faced in the enterprise environments while interacting with Active Directory including filtering out users to be brought into Hadoop from Active Directory, restricting access to a set of users from Active Directory, handling users from nested group structures, etc.
Speakers
Sailaja Polavarapu, staff Software Engineer, Hortonworks
Velmurugan Periasamy, Director - Engineering, Hortonworks
Ansible is an open-source configuration management and deployment tool, which can be used to manage servers and software installations. This talk will briefly cover Ansible itself, and then explain how Ansible is used to install and configure PostgreSQL on a server. Examples will round up the talk.
OVN (Open Virtual Network) を用いる事により、OVS (Open vSwitch)が動作する複数のサーバー(Hypervisor/Chassis)を横断する仮想ネットワークを構築する事ができます。
本スライドはOVNを用いた論理ネットワークの構成と設定サンプルのメモとなります。
Using OVN, you can build logical network among multiple servers (Hypervisor/Chassis) running OVS (Open vSwitch).
This slide is describes HOW TO example of OVN configuration to create 2 logical switch connecting 4 VMs running on 2 chassis.
This document discusses deploying IPv6 on OpenStack. It provides an overview of IPv6, including that IPv6 addresses the shortage of IPv4 addresses by providing a vastly larger 128-bit address space. It describes IPv6 address types and allocation methods. It also discusses IPv6 configuration modes in OpenStack, including stateless address autoconfiguration (SLAAC) and DHCPv6 stateless and stateful modes. Additionally, it covers deployment options for IPv6 on OpenStack like dual stack, NAT64/DNS64, and network tunnels. It provides details on IPv6 address and router advertisement configuration in OpenStack.
This presentation provides an overview of the Dell PowerEdge R730xd server performance results with Red Hat Ceph Storage. It covers the advantages of using Red Hat Ceph Storage on Dell servers with their proven hardware components that provide high scalability, enhanced ROI cost benefits, and support of unstructured data.
HAProxy is a free, open-source load balancer and reverse proxy that is fast, reliable and offers high availability. It can be used to load balance HTTP and TCP-based applications. Some key features include out-of-band health checks, hot reconfiguration, and multiple load balancing algorithms. Many large companies use HAProxy to load balance their websites and applications. It runs on Linux, BSD, and Solaris and can be used to load balance applications across servers on-premises or in the cloud.
YOW2018 Cloud Performance Root Cause Analysis at NetflixBrendan Gregg
Keynote by Brendan Gregg for YOW! 2018. Video: https://www.youtube.com/watch?v=03EC8uA30Pw . Description: "At Netflix, improving the performance of our cloud means happier customers and lower costs, and involves root cause
analysis of applications, runtimes, operating systems, and hypervisors, in an environment of 150k cloud instances
that undergo numerous production changes each week. Apart from the developers who regularly optimize their own code
, we also have a dedicated performance team to help with any issue across the cloud, and to build tooling to aid in
this analysis. In this session we will summarize the Netflix environment, procedures, and tools we use and build t
o do root cause analysis on cloud performance issues. The analysis performed may be cloud-wide, using self-service
GUIs such as our open source Atlas tool, or focused on individual instances, and use our open source Vector tool, f
lame graphs, Java debuggers, and tooling that uses Linux perf, ftrace, and bcc/eBPF. You can use these open source
tools in the same way to find performance wins in your own environment."
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
This document provides an overview and comparison of MySQL InnoDB Cluster and MySQL InnoDB ReplicaSet. It discusses the components, goals, and features of each solution. MySQL InnoDB Cluster uses Group Replication to provide high availability, automatic failover, and data consistency. MySQL InnoDB ReplicaSet uses asynchronous replication and provides availability and read scaling through manual primary/secondary configuration and failover. Both solutions integrate MySQL Shell, Router, and automatic member provisioning for easy management.
The document summarizes several industry standard benchmarks for measuring database and application server performance including SPECjAppServer2004, EAStress2004, TPC-E, and TPC-H. It discusses PostgreSQL's performance on these benchmarks and key configuration parameters used. There is room for improvement in PostgreSQL's performance on TPC-E, while SPECjAppServer2004 and EAStress2004 show good performance. TPC-H performance requires further optimization of indexes and query plans.
We will show the advantages of having a geo-distributed database cluster and how to create one using Galera Cluster for MySQL. We will also discuss the configuration and status variables that are involved and how to deal with typical situations on the WAN such as slow, untrusted or unreliable links, latency and packet loss. We will demonstrate a multi-region cluster on Amazon EC2 and perform some throughput and latency measurements in real-time (video http://galeracluster.com/videos/using-galera-replication-to-create-geo-distributed-clusters-on-the-wan-webinar-video-3/)
This document provides information about an upcoming Heat Orchestration Template (HOT) learning session at the OpenStack Summit in Austin, TX on April 27th 2016. It introduces the two presenters, Kanagaraj Manickam and Huang Tianhua, and provides an agenda and overview of the content to be covered, including Heat, HOT schematics, validation and preview, and Heat features like auto-scaling and software deployment.
Migrating Java EE applications to IBM Bluemix Platform-as-a-ServiceDavid Currie
This document discusses migrating Java EE applications from traditional deployment to IBM's Bluemix Platform-as-a-Service (PaaS). It introduces key concepts of cloud computing including IaaS, PaaS and SaaS models. It then focuses on Bluemix, describing it as IBM's cloud platform that is built on Cloud Foundry and provides services across various categories. The document guides developers on migrating an example application called DayTrader to Bluemix, covering steps like using database and other services, scaling the runtime, and adopting additional services to enhance the application.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
1. DPDK achieves high throughput packet processing on commodity hardware by reducing kernel overhead through techniques like polling, huge pages, and userspace drivers.
2. In Linux, packet processing involves expensive operations like system calls, interrupts, and data copying between kernel and userspace. DPDK avoids these by doing all packet processing in userspace.
3. DPDK uses techniques like isolating cores for packet I/O threads, lockless ring buffers, and NUMA awareness to further optimize performance. It can achieve throughput of over 14 million packets per second on 10GbE interfaces.
This document provides a summary of improvements made to Hive's performance through the use of Apache Tez and other optimizations. Some key points include:
- Hive was improved to use Apache Tez as its execution engine instead of MapReduce, reducing latency for interactive queries and improving throughput for batch queries.
- Statistics collection was optimized to gather column-level statistics from ORC file footers, speeding up statistics gathering.
- The cost-based optimizer Optiq was added to Hive, allowing it to choose better execution plans.
- Vectorized query processing, broadcast joins, dynamic partitioning, and other optimizations improved individual query performance by over 100x in some cases.
Many enterprise are implementing Hadoop projects to manage and process large datasets. Big question is: how to configure Hadoop clusters to connect to enterprise directory containing 100k+ users and groups for access management. Several large enterprises have complex directory servers for managing users and groups. Many advanced features have been recently added to Hadoop user management in order to support various complex directory server structures.
In this session attendees will learn about: setting up Hadoop node with users from Active Directory for executing Hadoop jobs, setting up authentication for enterprise users, and setting up authorization for users and groups using Apache Ranger. Attendees will also learn about the common challenges faced in the enterprise environments while interacting with Active Directory including filtering out users to be brought into Hadoop from Active Directory, restricting access to a set of users from Active Directory, handling users from nested group structures, etc.
Speakers
Sailaja Polavarapu, staff Software Engineer, Hortonworks
Velmurugan Periasamy, Director - Engineering, Hortonworks
Ansible is an open-source configuration management and deployment tool, which can be used to manage servers and software installations. This talk will briefly cover Ansible itself, and then explain how Ansible is used to install and configure PostgreSQL on a server. Examples will round up the talk.
OVN (Open Virtual Network) を用いる事により、OVS (Open vSwitch)が動作する複数のサーバー(Hypervisor/Chassis)を横断する仮想ネットワークを構築する事ができます。
本スライドはOVNを用いた論理ネットワークの構成と設定サンプルのメモとなります。
Using OVN, you can build logical network among multiple servers (Hypervisor/Chassis) running OVS (Open vSwitch).
This slide is describes HOW TO example of OVN configuration to create 2 logical switch connecting 4 VMs running on 2 chassis.
This document discusses deploying IPv6 on OpenStack. It provides an overview of IPv6, including that IPv6 addresses the shortage of IPv4 addresses by providing a vastly larger 128-bit address space. It describes IPv6 address types and allocation methods. It also discusses IPv6 configuration modes in OpenStack, including stateless address autoconfiguration (SLAAC) and DHCPv6 stateless and stateful modes. Additionally, it covers deployment options for IPv6 on OpenStack like dual stack, NAT64/DNS64, and network tunnels. It provides details on IPv6 address and router advertisement configuration in OpenStack.
This presentation provides an overview of the Dell PowerEdge R730xd server performance results with Red Hat Ceph Storage. It covers the advantages of using Red Hat Ceph Storage on Dell servers with their proven hardware components that provide high scalability, enhanced ROI cost benefits, and support of unstructured data.
HAProxy is a free, open-source load balancer and reverse proxy that is fast, reliable and offers high availability. It can be used to load balance HTTP and TCP-based applications. Some key features include out-of-band health checks, hot reconfiguration, and multiple load balancing algorithms. Many large companies use HAProxy to load balance their websites and applications. It runs on Linux, BSD, and Solaris and can be used to load balance applications across servers on-premises or in the cloud.
YOW2018 Cloud Performance Root Cause Analysis at NetflixBrendan Gregg
Keynote by Brendan Gregg for YOW! 2018. Video: https://www.youtube.com/watch?v=03EC8uA30Pw . Description: "At Netflix, improving the performance of our cloud means happier customers and lower costs, and involves root cause
analysis of applications, runtimes, operating systems, and hypervisors, in an environment of 150k cloud instances
that undergo numerous production changes each week. Apart from the developers who regularly optimize their own code
, we also have a dedicated performance team to help with any issue across the cloud, and to build tooling to aid in
this analysis. In this session we will summarize the Netflix environment, procedures, and tools we use and build t
o do root cause analysis on cloud performance issues. The analysis performed may be cloud-wide, using self-service
GUIs such as our open source Atlas tool, or focused on individual instances, and use our open source Vector tool, f
lame graphs, Java debuggers, and tooling that uses Linux perf, ftrace, and bcc/eBPF. You can use these open source
tools in the same way to find performance wins in your own environment."
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
This document provides an overview and comparison of MySQL InnoDB Cluster and MySQL InnoDB ReplicaSet. It discusses the components, goals, and features of each solution. MySQL InnoDB Cluster uses Group Replication to provide high availability, automatic failover, and data consistency. MySQL InnoDB ReplicaSet uses asynchronous replication and provides availability and read scaling through manual primary/secondary configuration and failover. Both solutions integrate MySQL Shell, Router, and automatic member provisioning for easy management.
The document summarizes several industry standard benchmarks for measuring database and application server performance including SPECjAppServer2004, EAStress2004, TPC-E, and TPC-H. It discusses PostgreSQL's performance on these benchmarks and key configuration parameters used. There is room for improvement in PostgreSQL's performance on TPC-E, while SPECjAppServer2004 and EAStress2004 show good performance. TPC-H performance requires further optimization of indexes and query plans.
We will show the advantages of having a geo-distributed database cluster and how to create one using Galera Cluster for MySQL. We will also discuss the configuration and status variables that are involved and how to deal with typical situations on the WAN such as slow, untrusted or unreliable links, latency and packet loss. We will demonstrate a multi-region cluster on Amazon EC2 and perform some throughput and latency measurements in real-time (video http://galeracluster.com/videos/using-galera-replication-to-create-geo-distributed-clusters-on-the-wan-webinar-video-3/)
This document provides information about an upcoming Heat Orchestration Template (HOT) learning session at the OpenStack Summit in Austin, TX on April 27th 2016. It introduces the two presenters, Kanagaraj Manickam and Huang Tianhua, and provides an agenda and overview of the content to be covered, including Heat, HOT schematics, validation and preview, and Heat features like auto-scaling and software deployment.
Migrating Java EE applications to IBM Bluemix Platform-as-a-ServiceDavid Currie
This document discusses migrating Java EE applications from traditional deployment to IBM's Bluemix Platform-as-a-Service (PaaS). It introduces key concepts of cloud computing including IaaS, PaaS and SaaS models. It then focuses on Bluemix, describing it as IBM's cloud platform that is built on Cloud Foundry and provides services across various categories. The document guides developers on migrating an example application called DayTrader to Bluemix, covering steps like using database and other services, scaling the runtime, and adopting additional services to enhance the application.
.NET Cloud-Native Bootcamp- Los AngelesVMware Tanzu
This document outlines an agenda for a .NET cloud-native bootcamp. The bootcamp will introduce practices, platforms and tools for building modern .NET applications, including microservices, Cloud Foundry, and cloud-native .NET technologies and patterns. The agenda includes sessions on microservices, Cloud Foundry, hands-on exercises, and a wrap up. Break times are scheduled between sessions.
The document discusses continuous delivery of integration applications using JBoss Fuse and OpenShift. It covers the cost of change in software development, how JBoss Fuse can help with integration challenges, and how OpenShift enables continuous delivery through automation and a developer self-service platform as a service model. The presentation demonstrates how to build a continuous delivery pipeline using tools like Git, Jenkins, Fabric8, and OpenShift to deploy and test applications.
This document provides an overview of the Into the Box (ITB) conference. It includes information about thanking sponsors and speakers, an ITB mobile app, winning an ITB drone, a ColdFusion Alive podcast, happy box activities, the Ortus team members, ColdFusion tooling and modernization, legacy issues, finding developers, education opportunities, and the future of ColdBox, ContentBox, Relax, Elixir, CommandBox, Docker, and Ortus projects.
Slides from Workshop 'Cloud Foundry: Hands-on Deployment Workshop'
http://www.meetup.com/CloudFoundry/events/150601282/
In this workshop you will learn Cloud Foundry fundamental concepts, setup, deployment and operations. We’ll cover a couple of alternatives to deploy CF in a local environment for learning and testing purposes as well as deploying Cloud Foundry atop IaaS production level environment, being able to manage hundreds of components and thousands of applications.
If you did not have a chance to work with Cloud Foundry, it may be useful to test its features locally at first. Deploying this environment on a local machine allows you to get hands-on experience in the solution and, in case you are a contributor, to test some features before you commit them to a production environment.
This document provides an introduction and overview of Docker, including its rapid growth and adoption, key benefits for developers and operations teams, technical underpinnings, ecosystem support, use cases, and future plans. Docker provides a way to package applications into lightweight containers that are portable and can run on any infrastructure. It solves issues around dependency management and consistency across environments.
SOLID Programming with Portable Class LibrariesVagif Abilov
Developers often don't pay attention to code portability until they need to target multiple platforms. However, large amount of non-portable code often hints about violation of clean code principles, so it is worth investigating which part of the source code base are platform-specific and for what reasons.
In this session we will give an overview of portable class libraries, show how to extract PCL components from a real-world application and go through typical challenges that are faced when writing portable code. We will present the original tool that analyzes assemblies for portability compliance and can be used as a guard to prevent mixing business logic with infrastructure-specific functionality. Finally we will demonstrate how PCLs help targeting platforms such as Windows Store, Android and iOS.
Staying on Topic - Invoke OpenFaaS functions with KafkaRichard Gee
This talk introduced both OpenFaaS & OpenFaaS Cloud before demonstrating the Kafka Connector. Based on the Connector SDK, the Kafka Connector enables Kubernetes based functions to be triggered using messages consumed from Kafka topics. The connector easily integrates with your existing Kafka installations enabling you to flexibly complement your existing systems' functionality with serverless functions.
This document discusses identity management solutions for AFS including AFS Manager and PtServer NG. AFS Manager is a graphical user interface for administering AFS that allows monitoring and provisioning users, groups and volumes. PtServer NG allows integrating AFS identity management with external user directories like Active Directory using Winbind for authentication and ID mapping. It aims to provide a pluggable user storage and flexible user and group mapping. The document demonstrates these tools and discusses open points around licensing, performance and storage options.
Wakanda is an open source platform that provides benefits of an open environment including freedom, adaptability, interoperability, portability, reusability, and community. It uses open source libraries and has open source and dual licensing. The Wakanda Studio includes tools like a model designer, GUI designer, and debugger. It supports add-ons, external widgets, and web components. The Wakanda backend integrates technologies like HTTP APIs, modules, and supports accessing external databases and technologies.
The 12 Factor App methodology provides guidelines for building software-as-a-service applications in the cloud. It advocates for codebases that are tracked in revision control, explicit declaration of dependencies, separation of configuration from code, treating backing services as attached resources, and strict separation between build, release, and run stages. The methodology also includes guidelines for processes, port binding, concurrency, disposability, keeping development and production environments similar, and treating logs as event streams. Following the 12 factors can help applications maximize portability, be more robust and agile, and scale smoothly by avoiding reliance on implicit tools or behaviors.
The document discusses best practices for building and deploying Scala applications based on the 12 Factor App methodology. It covers topics like managing dependencies, separating configuration from code, building in a simple and automated way, scaling apps through stateless processes, achieving parity between development and production environments, and running admin tasks isolated from the main app. The presentation provides examples using tools like sbt, Dropwizard, and Heroku to demonstrate how to structure Scala apps according to the 12 factors.
If you're a web developer or a site owner and you've been thinking of breaking out of shared hosting, maybe you've been looking at cloud hosting. This presentation outlines the pros and cons of shared hosting vs cloud hosting, and how to build a roll-your-own cloud host, complete with a clean, fast, free open source control panel. The talk was delivered to the Melbourne Joomla! User Group on 25 March 2015.
DevOps Unleashed: Strategies that Speed DeploymentsForgeRock
Modern identity management platforms must be agile and secure enough to respond to demanding business timelines. As a result, many organizations are seeking cloud-based approaches to digital security and need offerings that are optimized for environments including Cloud Foundry, Azure, GCE, AWS and OpenStack. Your dev-ops strategy could be the difference between hitting or missing business-critical deadlines. In this webinar, learn how we are enhancing the ForgeRock Identity Platform to enable developers to use container-oriented technologies such as Kubernetes and Docker to accelerate deployment.
Dart Past Your Competition by Getting Your Digital Experience into Market Fas...Perficient, Inc.
During the 2015 IBM Digital Experience, Mark Polly, Perficient Director, Strategic Advisors for Portal, Social, Web Content, demonstrated how you can dart past your competition by getting your digital experience into market faster than ever before.
This document discusses different cloud platforms for hosting Grails applications. It provides an overview of infrastructure as a service (IaaS) models like Amazon EC2 and shared/dedicated virtual private servers, as well as platform as a service (PaaS) options including Amazon Beanstalk, Google App Engine, Heroku, Cloud Foundry, and Jelastic. A comparison chart evaluates these platforms based on factors such as pricing, control, reliability, and scalability. The document emphasizes that competition and changes in the cloud space are rapid and recommends keeping applications loosely coupled and testing platforms using free trials.
Phase2 - Large Drupal Multisites (GTA Case Study)Robert Bates
This document discusses Drupal platforms and multi-site configurations. It defines a Drupal platform as a structural form that enables the creation of products and processes without requiring new development. Drupal multi-site allows a single Drupal installation to manage multiple sites, sharing code, modules, and themes. The document then describes how Georgia.gov uses a Drupal platform with a multi-site configuration, including 48 contributed modules, 8 custom modules, and 6 themes to manage the state government website and related sites.
Similar to DevOps with ActiveMQ, Camel, Fabric8, and HawtIO (20)
Move Auth, Policy, and Resilience to the PlatformChristian Posta
Developer's time is the most crucial resource in an enterprise IT organization. Too much time is spent on undifferentiated heavy lifting and in the world of APIs and microservices much of that is spent on non-functional, cross-cutting networking requirements like security, observability, and resilience.
As organizations reconcile their DevOps practices into Platform Engineering, tools like Istio help alleviate developer pain. In this talk we dig into what that pain looks like, how much it costs, and how Istio has solved these concerns by examining three real-life use cases. As this space continues to emerge, and innovation has not slowed, we will also discuss the recently announced Istio sidecar-less mode which significantly reduces the hurdles to adopt Istio within Kubernetes or outside Kubernetes.
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
Service mesh is a powerful pattern for implementing strong zero-trust networking practices, introducing better network observability, and allowing for more fine-grained traffic control. Up until now, the sidecar pattern was used to implement service-mesh capability but as the technology matures, a new pattern has emerged: sidecarless service mesh. Two prominent open-source networking projects, Cilium and Istio, have implemented a sidecar-free approach to service mesh but they both make interesting design decisions and tradeoffs. In this talk we review the architecture of both, focusing on the pros and cons of implementations such as mutual authentication, ingress, and observability.
Understanding Wireguard, TLS and Workload IdentityChristian Posta
Zero Trust Networking has become a standard marketing buzzword but the underlying principles are critical for modern microservice-style architectures. Authentication, authorizations, policy, etc. can be difficult to implement between services and do so in a maintainable way. Google invented their own transparent encryption and authorization protocol called "ALTS" back in 2007 to serve the application layer of Google's Borg workload scheduler, but we don't see others using it outside Google.
In this webinar we look at existing technology like TLS and newcomer Wireguard and see how these technologies come together to provide a secure foundation for workload identity and modern service-to-service networking.
Istio ambient mesh uses a sidecar-less data plane that focuses on ease of operations, incremental adoption, and separation of security boundaries for applications and mesh infrastructure.
In this webinar, we'll explore:
- The forces of modernization and compliance pressures,
- How Zero Trust Architecture (ZTA) can help, and
- How Istio ambient mesh lowers the barrier for establishing the properties necessary to achieve Zero Trust and compliance
The document discusses Cilium and Istio with Gloo Mesh. It provides an overview of Gloo Mesh, an enterprise service mesh for multi-cluster, cross-cluster and hybrid environments based on upstream Istio. Gloo Mesh focuses on ease of use, powerful best practices built in, security, and extensibility. It allows for consistent API for multi-cluster north-south and east-west policy, team tenancy with service mesh as a service, and driving everything through GitOps.
This document discusses service mesh patterns for connecting microservices across multiple clusters. It describes using Envoy proxy to provide service discovery, load balancing, security and resiliency. Patterns are presented for connecting services across clusters with flat, controlled or separate networks. Managing connectivity across clusters can increase operator burden. Gloo Mesh is presented as a way to simplify management across multiple clusters with a centralized control plane.
Multicluster Kubernetes and Service Mesh PatternsChristian Posta
Building applications for cloud-native infrastructure that are resilient, scalable, secure, and meet compliance and IT objectives gets complicated. Another wrinkle for the organizations with which we work is the fact they need to run across a hybrid deployment footprint, not just Kubernetes. At Solo.io, we build application networking technology on Envoy Proxy that helps solve difficult multi-deployment, multi-cluster, and even multi-mesh problems.
In this webinar, we’re going to explore different options and patterns for building secure, scalable, resilient applications using technology like Kubernetes and Service Mesh without leaving behind existing IT investments. We’ll see why and when to use multi-cluster topologies, how to build for high availability and team autonomy, and solve for things like service discovery, identity federation, traffic routing, and access control.
Cloud-Native Application Debugging with Envoy and Service MeshChristian Posta
Microservices have been great for accelerating the software innovation and delivery, but they also present new challenges, especially as abstractions and automated orchestration at every layer make pinpointing the issue seem like walking around a maze with a blindfold. Existing tools weren’t designed for distributed environments, and the new tools need to consider how to leverage these abstraction layers to better observe, test, and troubleshoot issues.
Christian Posta walks you through Envoy Proxy and service mesh architecture for L7 data plane, the key features in Envoy that can help in debugging and troubleshooting, chaos engineering as a testing methodology for microservices, how to approach a testing and debugging framework for microservices, and new open source tools that address these areas. You’ll explore a workflow to discover and resolve microservices issues, including injecting experiments for stress testing the applications, gathering requests in flight, recording and replaying them, and debugging them step by step without affecting production traffic.
Kubernetes Ingress to Service Mesh (and beyond!)Christian Posta
Kubernetes users need to allow traffic to flow into and within the cluster. Treating the application traffic separately from the business logic allows presents new possibilities in how service to service traffic is served, controlled and observed — and provides a transition to intra cluster networking like Service Mesh. With microservices, there is a concept of both North / South traffic (incoming requests from end users to the cluster) and East / West (intra cluster) communication between the services. In this talk we will explain how Envoy Proxy works in Kubernetes as a proxy for both of these traffic directions and how it can be leveraged to do things like traffic shaping, security, and integrate the north/south to east/west behavior.
Christian Posta (@christianposta) is Global Field CTO at Solo.io, former Chief Architect at Red Hat, and well known in the community for being an author (Istio in Action, Manning, Istio Service Mesh, O'Reilly 2018, Microservices for Java Developers, O’Reilly 2016), frequent blogger, speaker, open-source enthusiast and committer on various open-source projects including Istio, Kubernetes, and many others. Christian has spent time at both enterprises as well as web-scale companies and now helps companies create and deploy large-scale, cloud-native resilient, distributed architectures. He enjoys mentoring, training and leading teams to be successful with distributed systems concepts, microservices, devops, and cloud-native application design.
The exploration of service mesh for any organization comes with some serious questions. What data plane should I use? How does this tie in with my existing API infrastructure? What kind of overhead do sidecar proxies demand? As I've seen in my work with various organizations over the years "if you have a successful microservices deployment, then you have a service mesh whether it’s explicitly optimized as one or not."
In this talk, we seek to understand the role of the data plane and how to pick the right component for the problem context. We start off by establishing the spectrum of data-plane components from shared gateways to in-code libraries with service proxies being along that spectrum. We clearly identify which scenarios would benefit from which part of the data-plane spectrum and show how modern service meshes including Istio, Linkerd, and Consul enable these optimizations.
Deep Dive: Building external auth plugins for Gloo EnterpriseChristian Posta
Using the plugin framework for Ext. Auth Service in Gloo Enterprise, we can build any custom AuthN/AuthZ plugins to handle security requirements not provided out of the box.
Role of edge gateways in relation to service mesh adoptionChristian Posta
API Gateways provide functionality like rate limiting, authentication, request routing, reporting, and more. If you’ve been following the rise in service-mesh technologies, you’ll notice there is a lot of overlap with API Gateways when solving some of the challenges of microservices. If service mesh can solve these same problems, you may wonder whether you really need a dedicated API Gateway solution?
The reality is there is some nuance in the problems solved at the edge (API Gateway) compared to service-to-service communication (service mesh) within a cluster. But with the evolution of cluster-deployment patterns, these nuances are becoming less important. What’s more important is that the API Gateway is evolving to live at a layer above service mesh and not directly overlapping with it. In other words, API Gateways are evolving to solve application-level concerns like aggregation, transformation, and deeper context and content-based routing as well as fitting into a more self-service, GitOps style workflow.
In this talk we put aside the “API Gateway” infrastructure as we know it today and go back to first principles with the “API Gateway pattern” and revisit the real problems we’re trying to solve. Then we’ll discuss pros and cons of alternative ways to implement the API Gateway pattern and finally look at open source projects like Envoy, Kubernetes, and GraphQL to see how the “API Gateway pattern” actually becomes the API for our applications while coexisting nicely with a service mesh (if you adopt a service mesh).
Navigating the service mesh landscape with Istio, Consul Connect, and LinkerdChristian Posta
The document discusses various service mesh options including Linkerd, Consul Connect, Istio, and AWS App Mesh. It provides an overview of each solution, describing their key features and strengths/opportunities. It emphasizes that the service mesh approach is useful for managing inter-service communication and that implementations are still evolving. It recommends starting simply and iteratively adopting capabilities to match needs.
Distributed microservices introduce new challenges: failure modes are harder to anticipate and resolve. In this session, we present a “Chaos Debugging” framework enabled by three open source projects: Gloo Shot, Squash, and Loop to help you increase your microservices’ “immunity” to issues.
Gloo Shot integrates with any service mesh to implement advanced, realistic chaos experiments. Squash connects powerful and mature debuggers (gdb, dlv, java debugging) to your microservices while they run in Kubernetes. Loop extends the capability of your service mesh to observe your application and record full transactions for sandboxed replay and debugging.
Come to this demo-heavy talk to see how together, Squash, Gloo Shot, and Loop allow you to trigger, replay, and investigate failure modes of your microservices in a language agnostic and efficient manner without requiring any changes to your code.
Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...Christian Posta
If you have an existing Java monolith, you know you must take care making changes to it or altering it in any negative way. Often times these monoliths are very valuable to the business and generate a lot of revenue. At the same time, since it’s difficult to make changes to the monolith it’s desirable to move to a microservices architecture. Unfortunately you cannot just do a big-bang migration to a greenfield architecture and will have to incrementally adopt microservices. In this talk, we’ll look at using Gloo proxy which is based on Envoy Proxy and GraphQL to do surgical, function-level traffic control and API aggregation to safely migrate your monolith to microservices and serverless functions.
Service-mesh options with Linkerd, Consul, Istio and AWS AppMeshChristian Posta
Service mesh abstracts the network from developers to solve three main pain points:
How do services communicate securely with one another
How can services implement network resilience
When things go wrong, can we identify what and why
Service mesh implementations usually follow a similar architecture: traffic flows through control points between services (usually service proxies deployed as sidecar processes) while an out-of-band set of nodes is responsible for defining the behavior and management of the control points. This loosely breaks out into an architecture of a "data plane" through which requests flow and a "control plane" for managing a service mesh.
Different service mesh implementations use different data planes depending on their use cases and familiarity with particular technology. The control plane implementations vary between service-mesh implementations as well. In this talk, we'll take a look at three different control plane implementations with Istio, Linkerd and Consul, their strengths, and their specific tradeoffs to see how they chose to solve each of the three pain points from above. We can use this information to make choices about a service mesh or to inform our journey if we choose to build a control plane ourselves.
The document summarizes the new features of Istio 1.1, an open-source service mesh. Some key highlights include improved performance and scalability, namespace isolation, multi-cluster capabilities, easier installation with Helm, and locality-aware load balancing. A new Sidecar resource was introduced to improve performance by configuring resources for individual proxies. The presentation demonstrates performance improvements with the Sidecar resource and highlights additional functionality in Istio like traffic control and metrics collection.
API Gateways are going through an identity crisisChristian Posta
API Gateways provide functionality like rate limiting, authentication, request routing, reporting, and more. If you've been following the rise in service-mesh technologies, you'll notice there is a lot of overlap with API Gateways when solving some of the challenges of microservices. If service mesh can solve these same problems, you may wonder whether you really need a dedicated API Gateway solution?
The reality is there is some nuance in the problems solved at the edge (API Gateway) compared to service-to-service communication (service mesh) within a cluster. But with the evolution of cluster-deployment patterns, these nuances are becoming less important. What's more important is that the API Gateway is evolving to live at a layer above service mesh and not directly overlapping with it. In other words, API Gateways are evolving to solve application-level concerns like aggregation, transformation, and deeper context and content-based routing as well as fitting into a more self-service, GitOps style workflow.
In this talk we put aside the "API Gateway" infrastructure as we know it today and go back to first principles with the "API Gateway pattern" and revisit the real problems we're trying to solve. Then we'll discuss pros and cons of alternative ways to implement the API Gateway pattern and finally look at open source projects like Envoy, Kubernetes, and GraphQL to see how the "API Gateway pattern" actually becomes the API for our applications while coexisting nicely with a service mesh (if you adopt a service mesh).
KubeCon NA 2018: Evolution of Integration and Microservices with Service Mesh...Christian Posta
Cloud-native describes a way of building applications on a cloud platform to iteratively discover and deliver business value. We now have access to a lot of similar technology that the large internet companies pioneered and used to their advantage to dominate their respective markets. What challenges arise when we start building applications to take advantage of this new technology?
In this talk we'll explore the role of service meshes when building distributed systems, why they make sense, and where they don't make sense. We will look at a class of problem that crops up that service mesh cannot solve, but that frameworks and even new programming languages like Ballerina are aiming to solve
Service-mesh technology promises to deliver a lot of value to a cloud-native application, but it doesn't come without some hype. In this talk, we'll look at what is a "service mesh", how it compares to similar technology (Netflix OSS, API Management, ESBs, etc) and what options for service mesh exist today.
Sami provided a beginner-friendly introduction to Amazon Web Services (AWS), covering essential terms, products, and services for cloud deployment. Participants explored AWS' latest Gen AI offerings, making it accessible for those starting their cloud journey or integrating AI into coding practices.
Thinking about freelancing? It's not just about coding solo and avoiding coworkers. Join me as I share insights from my 15-year freelance journey, covering everything from managing invoices to client communication styles. This session blends ColdFusion-specific tips with general freelance and consulting advice, with time for audience Q&A.
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...Hironori Washizaki
Hironori Washizaki, "Charting a Course for Equity: Strategies for Overcoming Challenges and Promoting Inclusion in the Metaverse", IEEE COMPSAC 2024 D&I Panel, 2024.
Are you wondering how to migrate to the Cloud? At the ITB session, we addressed the challenge of managing multiple ColdFusion licenses and AWS EC2 instances. Discover how you can consolidate with just one EC2 instance capable of running over 50 apps using CommandBox ColdFusion. This solution supports both ColdFusion flavors and includes cb-websites, a GoLang binary for managing CommandBox websites.
CommandBox was highlighted as a powerful web hosting solution, perfect for developers and businesses alike. Featuring a built-in server and command-line interface, CommandBox simplified web application management. Developers could deploy multiple application instances simultaneously, optimizing development workflows. CommandBox's efficient deployment processes ensured reliable web hosting, seamlessly integrating into existing workflows for scalability and feature enhancements.
In this session, we explored setting up Playwright, an end-to-end testing tool for simulating browser interactions and running TestBox tests. Participants learned to configure Playwright for applications, simulate user interactions to stress-test forms, and handle scenarios like taking screenshots, recording sessions, capturing Chrome dev tools traces, testing login failures, and managing broken JavaScript. The session also covered using Playwright with non-ColdBox sites, providing practical insights into enhancing testing capabilities.
Navigating the New Era of Adaptive PPM with OnePlan - Webinar 27Jun24.pdfOnePlan Solutions
The landscape of Project Portfolio Management (PPM) is undergoing a significant transformation, driven by the need for more adaptive, responsive approaches to managing projects and portfolios in a rapidly changing business environment. This evolution calls for tools and methodologies that can support dynamic decision-making, flexible resource allocation, and real-time strategic alignment.
Join us as we delve into how OnePlan emerges as a pivotal solution in this new era. Discover how OnePlan empowers organizations to navigate complexity and embrace change effectively, ensuring project success and strategic alignment. Learn how OnePlan enhances the capabilities of the tools your organization has already invested in, breaking down data silos and providing a unified view of project information.
Discover BoxLang, the innovative JVM programming language developed by Ortus Solutions. Designed to harness the power of the Java Virtual Machine, BoxLang offers a modern approach to application development with robust performance and scalability. Join us as we explore the capabilities of BoxLang, its syntax, and how it enhances productivity in software development.
we delve into the power of headless CMS—a versatile solution separating content creation from presentation. Explore its benefits: multi-channel delivery, accelerated time-to-market, content reusability, scalability, technology flexibility, and enhanced security. Discover how headless CMS transforms digital content management, empowering efficient and flexible content delivery across diverse platforms.
LIVE DEMO: CCX for CSPs, a drop-in DBaaS solutionSeveralnines
This webinar aims to equip Cloud Service Providers (CSPs) with the knowledge and tools to differentiate themselves from hyperscalers by offering a Database-as-a-Service (DBaaS) solution. The session will introduce and demonstrate CCX, a drop-in, premium DBaaS designed for rapid adoption.
Learn more about CCX for CSPs here: https://bit.ly/3VabiDr
Learn to manage your web form's question flow with RuleBox in this session. Simplify complex conditional statements by structuring logic in a readable and testable Given-When-Then format. Discussion covers prototyping tips, writing test cases, integrating external data, and managing multiple form versions with a single set of rules. Ideal for ColdFusion web developers exploring TestBox and/or RuleBox, with a demo featuring ColdBox and cborm, though not required.
ERP software interfaces and computerizes different corporate exercises. The system proficiently eliminates manual following, which wipes out human blunders. ERP system liberates crucial time and assets by digitizing redundant managerial undertakings. Know more information here: https://medium.com/@nyggsautomation/why-does-your-business-need-to-implement-erp-software-and-what-are-its-essential-modules-f7bba45be731
Join me for an insightful journey into task scheduling within the ColdBox framework. In this session, we explored how to effortlessly create and manage scheduled tasks directly in your code, enhancing control and efficiency in applications and modules. Attendees experienced a user-friendly dashboard for seamless task management and monitoring. Whether you're experienced with ColdBox or new to it, this session provided practical knowledge and tips to streamline your development workflow.
Austere Systems Company Portfolio (ASPL).pdfsupport433113
Austere Systems Pvt. Ltd. is a leading IT services provider specializing in a wide range of technology solutions. We help businesses leverage the power of IT to achieve their strategic goals and gain a competitive edge.
Our Expertise:
IT Staff Augmentation: We provide skilled and experienced IT professionals across various domains like SAP, Java, .Net, PHP and PowerBi.
Application Development: Our team builds robust mobile, Web and desktop applications to meet your specific business needs.
Product Re-engineering & Maintenance: We breathe new life into existing software and ensure its smooth operation.
Infrastructure Management: We take care of your IT infrastructure, including servers, networks, and security.
Support Services: Our L1/L2 support centers offer prompt and reliable assistance for your IT issues.
Digital Marketing & SEO: We help you reach your target audience and boost your online presence.
Why Choose Austere Systems?
Skilled & Experienced Professionals: Our team possesses in-depth knowledge and expertise in various technologies.
Focus on Client Satisfaction: We prioritize building strong relationships and exceeding client expectations.
Innovative Solutions: We deliver cutting-edge solutions tailored to your unique business challenges.
Cost-Effective Services: We offer competitive rates and ensure value for your investment.
How to debug ColdFusion Applications using “ColdFusion Builder extension for ...Ortus Solutions, Corp
Unlock the secrets of seamless ColdFusion error troubleshooting! Join us to explore the potent capabilities of Visual Studio Code (VS Code) and ColdFusion Builder (CF Builder) in debugging. This hands-on session guides you through practical techniques tailored for local setups, ensuring a smooth and efficient development experience.
This is why a security assessment is valuable for your organization.
It is important for organizations to continue investing in a well-secured Microsoft environment.
The better the security, the better data is protected and the smaller the chance of data leaks and cyber-attacks.
In addition, it contributes to maintaining a good reputation and leads to a more efficient working environment.
A SECA is an excellent way for companies to gain more insight into the security of their Microsoft environment.
What is a SECA?
A SECA, or security assessment, is an evaluation of the security of your Microsoft environment. During a security assessment, a SECA expert examines your current digital environment to detect possible risks. The purpose of a SECA is to improve the security of your IT infrastructure and reduce the chance of cyber-attacks and other problems.
What are the benefits of a security assessment for organizations?
A security assessment is very valuable if you want to optimally secure your Microsoft environment, to minimize the risk of annoying security leaks and problems. But that's not the only plus. A SECA offers many advantages for commercial and also government organizations, making it certainly worth considering.
A security assessment from Q-Advise.
Are you looking for a SECA expert? Then the experts at Q-Advise would be happy to discuss a collaboration. Thanks to our extensive experience and expertise and partner channel, we can help you get started. We are happy to help you make your Microsoft environment as secure as possible.
You can send the team an email…info@q-advise.com
Discover Passkeys, the next evolution in secure login methods that eliminate traditional password vulnerabilities. Learn about the CBSecurity Passkeys module's installation, configuration, and integration into your application to enhance security.
Alluxio Webinar | 10x Faster Trino Queries on Your Data PlatformAlluxio, Inc.
Alluxio Webinar
June. 18, 2024
For more Alluxio Events: https://www.alluxio.io/events/
Speaker:
- Jianjian Xie (Staff Software Engineer, Alluxio)
As Trino users increasingly rely on cloud object storage for retrieving data, speed and cloud cost have become major challenges. The separation of compute and storage creates latency challenges when querying datasets; scanning data between storage and compute tiers becomes I/O bound. On the other hand, cloud API costs related to GET/LIST operations and cross-region data transfer add up quickly.
The newly introduced Trino file system cache by Alluxio aims to overcome the above challenges. In this session, Jianjian will dive into Trino data caching strategies, the latest test results, and discuss the multi-level caching architecture. This architecture makes Trino 10x faster for data lakes of any scale, from GB to EB.
What you will learn:
- Challenges relating to the speed and costs of running Trino in the cloud
- The new Trino file system cache feature overview, including the latest development status and test results
- A multi-level cache framework for maximized speed, including Trino file system cache and Alluxio distributed cache
- Real-world cases, including a large online payment firm and a top ridesharing company
- The future roadmap of Trino file system cache and Trino-Alluxio integration
2. RED HAT | ADD NAME2
Agenda
• DevOps… What is that?
• Enterprise Integration
• Automated Delivery
• To the Cloud
3. RED HAT | ADD NAME3
Your speaker
Christian Posta
http://christianposta.com/blog
@christianposta
christian@redhat.com
• Principal Middleware Specialist
• Based in Phoenix, AZ
• Committer on Apache Camel, ActiveMQ, Apollo, PMC on ActiveMQ
• Author: Essential Camel Components DZone Refcard
4. Shorten the lifecycle from inception
to production so that the business
can make money!
Developers and Operations must
cooperate.
Rely on tools/frameworks to
automate, automate, and automate.
6. RED HAT | ADD NAME6
“WTF is DevOps?”
• IT is a core competency
• Set of principles
• There’s more to applications than coding!
• Feedback
• Repetition
• Communication
• People!
7. RED HAT | ADD NAME7
Shorten the feedback loop
• Developers
• QA
• Operations
• Business
• Customers!!!
10. RED HAT | ADD NAME10
• Off the shelf? Home Grown? Acquisition?
• Platforms
• Protocols / Data Formats
• Data Formats
• Timing
• Organizational mismatch
Why is integration hard?
11. RED HAT | ADD NAME11
• Light-weight integration library
• Domain Specific Language
• Enterprise Integration Patterns
• Components
• Routing and Mediation (like an ESB?)
• Runs in any container (or stand alone)
What is Apache Camel?
12. RED HAT | ADD NAME12
• Message Routing
• Transformation
• Aggregation
• Splitting
• Resequencer
• Routing Slip
• Enricher
• All 65 from the book!
Enterprise Integration Patterns
13. RED HAT | ADD NAME13
Components
• ActiveMQ, Websphere, Weblogic (JMS)
• AMQP
• ATOM feeds
• AWS (S3, SQS, SNS, others)
• Bean
• Cache (EHCache)
• CXF (JAX-WS, JAX-RS)
• EJB
• Drools
• File
• FTP/SFTP
• Google App Engine
• GMail
• HTTP
• IRC
• jclouds
• JDBC
• Jetty
• Twitter
• MQTT
• MyBatis
• JPA
• Spring Integration
• Spring Web Services
http://camel.apache.org/components.html
To see list of all
components!!
14. RED HAT | ADD NAME14
Java DSL
public class OrderProcessorRouteBuilder extends RouteBuilder {
@Override
public void configure() throws Exception {
from(“activemq:orders”)
.choice()
.when(header(“customer-rating”).isEqualTo(“gold”))
.to(“ibmmq:topic:specialCustomer”)
.otherwise()
.to(“ftp://user@host/orders/regularCustomers”)
.end()
.log(“received new order ${body.orderId}”)
.to(“ibatis:storeOrder?statementType=Insert”);
}
}
15. RED HAT | ADD NAME15
Spring XML DSL
<route id=“processOrders”>
<from uri=“activemq:orders”/>
<choice>
<when>
<simple>${header.customer-rating} == ‘gold’</simple>
<to uri=“ibmmq:topic:specialCustomer”>
</when>
<otherwise>
<to uri=“ftp://user@host/orders/regularCustomers” />
</otherwise>
</choice>
<log message=“received new order ${body.orderId}”/>
<to uri=“ibatis:storeOrder?statementType=Insert”/>
</route>
16. RED HAT | ADD NAME16
• Batch file transfers
• Shared Database
• RPC
• Messaging
Integration Options
17. RED HAT | ADD NAME17
• Asynchronous architectures
• Reliable message passing
• Loose coupling
• Heterogeneous integration
• Fault tolerant
• Scalable
• Real-time data
Why messaging?
18. RED HAT | ADD NAME18
• The most widely used open-source
messaging broker
• Highly configurable
• Friendly license (no license fees!)
• Vibrant community (TLP)
• Backbone of top enterprises in retail, e-retail,
financial services, shipping, many others!
Apache ActiveMQ
19. RED HAT | ADD NAME19
• High performance
• High availability
• Light-weight
• Multi-protocol (AMQP, MQTT, STOMP)
• Multi-transport (TCP,SSL,WS,VM,HTTP)
• JMS compliant
• Supported in production by Red Hat!
ActiveMQ Features
21. RED HAT | ADD NAME21
High-performance, real-time systems
Head Office
STORE
STORE
outlet
shop
24 houra
STORE
shop
Broker
Clusters
• Clustering, Network of Brokers
• Client-aware failover
• Master/Slave HA
• Fabric
22. RED HAT | ADD NAME22
Ingestion for BigData Architecture
Broker
Cluster
Broker
Cluster
Web Servers
Web Servers
Web Servers
Web Servers
Camel
HTTP Log
Reader
Camel
HTTP Log
Reader
Camel
HTTP Log
Reader
Camel
HTTP Log
Reader
ESB
ESB
Camel
HBase/HDFS
Camel
Hbase/HDFS
HDFS
23. RED HAT | ADD NAME23
Integration Everywhere – Internet of Things
Connecting Things
• mobile devices
• meters
• industrial controls
• smart buildings
• asset tracking
• traffic control
• monitors
• sensors
• actuators
Broker
Clusters
Arrival
Airport 1
24. RED HAT | ADD NAME24
JBoss Fuse
Integrate Everything!
25. RED HAT | ADD NAME25
More info on JBoss Fuse…
https://www.redhat.com/products/jbossenterprisemiddleware/fuse/
http://www.jboss.org/products/fuse
29. RED HAT | ADD NAME29
Current issues with deployments
• Installation
• SSH, download, unpack, install, permissions, ENV var
• Configuration
• Container, individual apps/services, brokers, security
• Upgrading
• Install, configure, rollback
• Introspection
• Big picture statistics (JVM/OS memory usage, CPU)
• JMX (jconsole, visualVM)
30. RED HAT | ADD NAME30
Clients aware of Topology
• Brokers
• failover:(tcp://host1:port1,tcp://host2:port2)
• Camel endpoints
• from(“jetty:http://22.33.44.55:8080/endpoint”)
• to(“http4://11.22.33.44:9000/endpoint”)
• Web service endpoints
• Access a specific endpoint: http://22.33.44.55:9091/endpoint
31. RED HAT | ADD NAME31
Current best practices
• Use templates for configuration w/ template engine
• Puppet/Chef and/or Capistrano/Ansible
• Store configuration (templates + values) in SCM
• Separate configuration from binary deployments
• Verifiable build and release process
32. RED HAT | ADD NAME32
• Simplifies deployments
• Provides management tools for centralized
configuration
• Visualize your middleware with HawtIO
• Polycontainer
• Blurs the line of PaaS
http://fabric8.io
33. RED HAT | ADD NAME33
• Provides cluster capabilities, coordination
• Service discovery, load balancing, failover
• Deploy to cloud (IaaS, PaaS)
• Supported as JBoss Fuse (managed)
• fabric8 1.0 is in Fuse 6.1
• 1.1 about to be released
• RC1 released 6/10
http://fabric8.io
34. RED HAT | ADD NAME34
fabric8 1.1 release
• Java Container
• Tomcat, TomEE, Jetty
• Spring Boot
• fabric:watch * with various containers
• Profile import/export
• fabric8 maven plugin enhancements
35. RED HAT | ADD NAME35
Architecture
Clustered Registry
Agent
Agent
Zookeeper
Agent
Zookeeper
Agent
Zookeeper
Version 1.0
Profile CXFProfile Camel
Profile Default
Features Configuration
Registry Content
containers that form an “ensemble”
Pulls
profile
data
Registers
/
Listens
for
changes
36. RED HAT | ADD NAME36
Core concepts
• Fabric registry
• Holds all configuration data
• Runtime registry for looking up distributed services
• Profile
• Describes the container set up
• Features, Bundles, ConfigAdmin PIDs, system properties
• Hierarchical structure
• Container versioning
• Agent
• Runs on each container
• Communicates with registry to make sure container provisioned
correctly
37. RED HAT | ADD NAME37
How’s fabric8 different than Puppet/Chef?
• Middleware centric
• Container agnostic
• Deep knowledge about the running processes
• Consistent configuration
• Visualizations
• Versioning built in, upgrades, rollbacks, selective
upgrade, etc
• Use puppet/chef to provision your machines, use
fabric8 to provision and manage your middleware/
apps
44. RED HAT | ADD NAME44
Continuous Delivery
• Builds on continuous integration
• Establish a concrete pipeline to production
• Build/Test/Release often!
• Bottlenecks?
• Involves Dev and Ops to be successful
• Every build is a “release candidate”
45. RED HAT | ADD NAME45
Automate everything!
• Developers
• Unit tests
• Integration tests
• Builds
• Deployments in dev
• Operations
• VMs
• Provisioning software
• Deployments in QA/UAT/PROD
46. RED HAT | ADD NAME46
Tools for a CD pipeline
• Puppet/Chef to provision VMs
• Git for SCM
• Gerrit/Gitlab for code reviews
• Maven
• Jenkins + plugins
• and of course… Fabric8!
• What’s this Docker thingy?
48. RED HAT | ADD NAME48
Sample flow
• Check your code in
• Gerrit for code reviews
• Jenkins for build + CD pipeline
• Use fabric8:zip to deploy profiles to Maven repo
• Use fabric8:branch to automate deploying multiple
profiles to QA/UAT/PROD
• Can use profile-import to manually import zips
• Build the binary once!
58. RED HAT | ADD NAME58
Flavors
origin
Public
Cloud
Service
On-
premise
or Private
Cloud
Software
Open
Source
Project
59. RED HAT | ADD NAME59
Terminology
l Broker – Management host, orchestration of Nodes
l Node – Compute host containing Gears
l Gear – Allocation of fixed memory, compute, and
storage resources for running applications
l Cartridge – A technology/framework (PHP, Perl, Java/
JEE, Ruby, Python, MySQL, etc.) to build applications
l Application – Instantiation of a Cartridge
l Client Tools – CLI, Eclipse Plugin, Web Console,
Java API, REST API
60. RED HAT | ADD NAME60
xPaaS/iPaaS
l xPaaS
- Cartridges for Entire JBoss & Fuse Portfolio
- EAP and EWS Enterprise Cartridges Today
- Several Community Cartridges Available Today
l iPaaS
- Integration PaaS
- Fuse/Fabric Cartridge
61. RED HAT | ADD NAME61
More Info
http://fabric8.io
http://hawt.io
http://docker.io
http://jboss.org/products/fuse
http://activemq.apache.org
http://camel.apache.org
62. RED HAT | ADD NAME62
Questions?
http://christianposta.com/blog
@christianposta
christian@redhat.com