Sas 2015 event_driven

These slides describe how Coursera created its next generation of long-running job scheduler by integrating Docker and Mesos.

This document discusses how Docker can be used with CloudStack. It provides several options: 1) Running Docker in VMs on CloudStack templates that include Docker, 2) Using Docker-optimized OS templates, 3) Launching containers through a container service API, 4) Using CloudStack plugins within the Docker ecosystem like Docker Machine. The document concludes that CloudStack should not try to write a Docker hypervisor plugin, but instead focus on Docker-optimized OS templates and deploying application frameworks to orchestrate Docker.

This document summarizes serverless design patterns and tools. It begins with a brief history of cloud computing and an introduction to serverless computing. Common serverless use cases like event-driven applications and stream processing are described. Several serverless patterns are then outlined, such as hosting a static website or REST API using AWS Lambda and API Gateway. Finally, the document demonstrates a serverless application and discusses future directions for serverless technologies.

Presentation given to NashJS (http://meetup.com/nashjs) on 6/14/2018 about serverless architecture in AWS using the Serverless framework (http://serverless.com).

Making it easy to integrate legacy and iterative microservices with REST/CQRS and deploy to Docker/Kubernetes/OpenShift all on a developer laptop!

Common considerations on Serverless architecture, AWS Lambda (including Serverless Framework) and ECS. Also introduces Guanyu, an open-sourced wrapper to Sophos-AV Free edition, as example to demonstrate patterns and tradeoffs in architecture.

Conflicts we at 104 Corp. faced in cloud migration. Most of the contents are in English except for presenter notes and title pages.

Tired of having users email you that your Svelte application is broken? Turns out that building reliable applications is hard and requires a lot of testing. You can write unit tests but quite often these all pass and the application is still broken. Why? Because they test parts of the application in isolation. But for a reliable application we need more. We need to make sure that all parts, including the backend API’s, work together as intended. Cypress is a great tool to achieve this. It will test you complete web application in the browser and use it like a real world user would. In this session Maurice will show you how to use Cypress during development and on the CI server with Svelte. He will share tips and tricks to make your tests more resilient and more like how an actual end user would behave.

Talk at JAWS DAYS '17 at Tokyo as the organizer to AWS User Group Taiwan. Covers Guanyu and kms-local a bit, both internal projects at 104 Corp. that will be opensourced.

In this session we'll discuss and demonstrate key concepts and design patterns for continuous deployment and integration using technologies like AWS OpsWorks and Chef to enable better control of applications and infrastructures.

It's the architect session on AWSome Day to introduce the cloud principles which you need to follow when developing cloud services.

In deploying apps that have been containerized, you have a lot to think about regarding what to use in production. There are a lot of things to manage, so orchestrators become a huge help. providing many services together such as scheduling, container communication, scaling, health, and more. There are major platforms to consider from Kubernetes, Swarm to ECS. In this talk we'll go through the overview of orchestrators and some of the differences between the big players. You should come out of the talk knowing where to go next in determining your orchestrator needs.

This document provides an overview of Stacktician, which is a tool that allows users to deploy and manage infrastructure on CloudStack using templates similar to AWS CloudFormation. It discusses the history and architecture of Stacktician, including its two main components - StackMate for executing templates and Stacktician for the web interface. It covers the current state including improvements made for error handling, rollbacks, metadata handling and scaling. Finally, it discusses some planned future enhancements such as better plugin support, nested stacks, and stack updates.

We highlight solutions to common Docker challenges that you may encounter as you move from initial experiments toward full-fledged Docker adoption. At RightScale, we’ve been sharing our lessons learned as we move toward a fully containerized environment leveraging a “sea of containers.” We’re now in the middle stages of that journey and will share some of the challenges we’ve encountered and how we’ve overcome them.

The document discusses Rohit Yadav and his work with Apache CloudStack. It provides an agenda for understanding CloudStack internals, including getting started as a user or developer, a guided tour of the codebase, common development patterns, and deep dives into key areas like system VMs, networking implementation, and plugins. The document outlines ways to join the CloudStack community and how to contribute code through GitHub pull requests.

This document is a summary of a webinar about infrastructure as code. It introduces the speaker, Srirajan, and discusses how automation tools like Chef, Puppet, Ansible and others can be used to define infrastructure in code. Key benefits of infrastructure as code include automation, repeatability, and disaster recovery. The webinar also discusses testing infrastructure code and version controlling code changes.

Using apache camel for microservices and integration then deploying and managing on Docker and Kubernetes. When we need to make changes to our app, we can use Fabric8 continuous delivery built on top of Kubernetes and OpenShift.

This document provides an overview of using the Vert.x reactive application platform at the European advertising network zanox. It discusses how zanox used Vert.x to build a new core system requiring low latency and high throughput. The document covers getting started with Vert.x, best practices like encapsulating common code in modules, deployment strategies including fat jars and Docker, and integrating Vert.x with messaging systems like Apache Kafka using available modules. Metrics showed the Vert.x system at zanox could handle 18,000-28,000 requests per second on average with response times under 2ms.

Nach wochenlanger Arbeit und nervenaufreibendem Approval-Prozess ist die Killer-App jetzt endlich im Store gelandet. Und jetzt? Die Downloadzahlen sind deutlich unter den Erwartungen, und etwas muss getan werden! In diesem Talk wird dargestellt, welche Möglichkeiten der Bewerbung existieren und welche Fallstricke es auf den unterschiedlichen Plattformen zu umschiffen gilt.

JBoss 7, Blazing Fast!

Sascha Möllering discusses how his company moved from manual server setup and deployment to automated deployments using infrastructure as code and continuous delivery. They now deploy whenever needed using tools like Chef and JBoss to configure servers. Previously they faced challenges like manual processes, difficult rollbacks, and biweekly deployment windows. Now deployments are automated, safer, and can happen continuously.

This document discusses scaling applications in the AWS cloud. It begins with an overview of AWS services like EC2, S3, RDS, and ELB. It then walks through creating a simple cloud application and database, and improving it by separating components, adding redundancy, caching, and autoscaling. A real-world example is shown using Vert.x, Kinesis, Docker, and deployment scripts to dynamically scale a streaming data application across Availability Zones.

This document provides an overview of CloudFormation best practices: - It discusses organizing infrastructure using CloudFormation stacks by layers, environments, and services to promote reuse and decoupling. - It recommends starting with existing templates, validation tools, parameter types, and IAM roles to prevent errors. - Debugging tips include viewing stack events, using wait conditions, and logging to CloudWatch. - Safe stack updates involve change sets to review impacts and choosing update styles for minimal disruption.

Log analytics is a common big data use case that allows you to analyze log data from websites, mobile devices, servers, sensors, and more for a wide variety of applications including digital marketing, application monitoring, fraud detection, ad tech, gaming, and IoT. In this tech talk, we will walk you step-by-step through the process of building an end-to-end analytics solution that ingests, transforms, and loads streaming data using Amazon Kinesis Firehose, Amazon Kinesis Analytics and AWS Lambda. The processed data will be saved to an Amazon Elasticsearch Service cluster, and we will use Kibana to visualize the data in near real-time. Learning Objectives: 1. Reference architecture for building a complete log analytics solution 2. Overview of the services used and how they fit together 3. Best practices for log analytics implementation

This document discusses infrastructure as code. It describes using VMware and SDKs like VMware vSphere and VI Java SDK to programmatically manage virtual machines. It also discusses using Chef to automate the installation of Linux, packages, and middleware like JBoss. The document provides examples of using Chef to configure JBoss and links to GitHub repositories for VIAutomator and autoimport samples. It concludes with a Q&A section.

A brief example that demonstrates asic usage of Kafka and Spring Integration when you build a microservice.

Independent of the source of data, the integration of event streams into an Enterprise Architecture gets more and more important in the world of sensors, social media streams and Internet of Things. Events have to be accepted quickly and reliably, they have to be distributed and analysed, often with many consumers or systems interested in all or part of the events. Dependent on the size and quantity of such events, this can quickly be in the range of Big Data. In this session an architecture with a central log structured storage is presented where anybody can store and subscribe for events. This can be implemented using frameworks such as Kafka, Storm, Samza and Spark Streaming.

This document provides an overview of cloud architecture and best practices for deploying applications in the AWS cloud. It begins with an introduction to key AWS services like EC2, ELB, RDS, and Auto Scaling. It then walks through creating a basic cloud deployment and improving it by separating concerns, adding redundancy, caching, and autoscaling. Finally, it discusses a real-world example using services like Kinesis and deploying containers with ECS.

This document discusses using Docker containers in the cloud. It begins with an introduction to Docker and Amazon Web Services (AWS). It then covers deploying Docker containers to AWS using services like OpsWorks, Elastic Beanstalk, and EC2 Container Service. It also discusses the immutable server pattern and using EC2 Container Service to manage Docker containers on EC2 instances.

This document discusses deploying Docker containers on Amazon Web Services. It covers using AWS services like EC2, OpsWorks and Elastic Beanstalk that support Docker. It describes using the EC2 Container Service for container management and deploying containers across a cluster of EC2 instances. It also discusses the immutable server pattern of deploying to new infrastructure with each release rather than changing existing servers.

This document discusses using Docker on AWS. It describes using Docker to deploy highly scalable applications across multiple AWS regions and availability zones. It also discusses using a private Docker registry hosted on EC2 and S3 to store custom Docker images. Finally, it summarizes using Amazon EC2 Container Service (ECS) for container management on AWS, including concepts like clusters, tasks, and container instances.

Lift Urban Entrepreneurs is designed to create an urban transversal vision through a series of events bringing together entrepreneurs, designers, developers, academics and corporates to explore new solutions to address these urban challenges. The mission is to foster new ideas in order to co-create, prototype and produce solutions for a better urban future. This is the storyboard of the Seoul Workshop held on Nov. 12, 2016 at Google Campus Seoul. http://www.urbanentrepreneurs.io

This document discusses Vert.x, an open source toolkit for building reactive applications on the JVM. It introduces Vert.x and describes how it was used at Zanox to build a new request processing system with requirements of low latency, high throughput, scalability, resilience, and responsiveness. The document outlines how to start with Vert.x, best practices, using the infrastructure as code and module system, integrating Kafka messaging, and metrics from Zanox's Vert.x project.

This document provides steps to integrate Jenkins with Amazon S3 for artifact storage. It demonstrates installing the Jenkins S3 plugin, configuring credentials for an IAM user with S3 access, and configuring a Jenkins job to upload build artifacts like an index.html file to an S3 bucket after a build. With this integration, artifacts can be reliably stored on S3, which is cheaper for storage than other options and allows easy tracking and management of files.

Sascha Möllering discusses infrastructure as code and provides an overview of VMware SDKs, Chef, and using Chef to configure JBoss middleware. He explains that VMware has multiple SDKs and that the VI Java SDK simplifies development. Chef is introduced as a tool to automate and standardize server configurations. The presentation then covers using Chef recipes to deploy and configure JBoss application servers and integrating with JBoss Operations Network for monitoring.

Discovery Communications migrated 80% of their IT infrastructure to AWS to reduce costs, improve scalability and performance, and gain access to AWS services and innovations; they worked with Cloudreach to assess over 800 workloads, design the cloud environment and migration process, and execute the datacenter migrations using a dedicated team and standardized "migration factory" approach. Discovery has already migrated workloads from datacenters in Paris, Miami, and the US with more migrations ongoing to achieve their cloud goals.

This document discusses microservices architecture and how it enables agility. It defines microservices as small, independent units that can be developed and deployed autonomously. The document argues that microservices align with the principles of the Agile Manifesto by allowing teams to work independently, facilitating continuous delivery, and making the architecture adaptable to change. Some benefits outlined are improved scalability, maintainability, and ability to replace services easily. The conclusion is that by structuring an organization and software architecture around microservices, greater agility can be achieved compared to a monolithic architecture.

The document discusses Yelp's use of Docker containers and microservices on their open source PaaSTA platform. It describes how Yelp uses Jenkins build pipelines with multiple steps, including a security-check step that runs tests to check for things like up-to-date packages, best practices for Docker containers, and known vulnerabilities. Any failures in the security tests will create a Jira ticket and notify teams so issues can be addressed quickly. The takeaways are that organizations can implement similar security testing in their own build pipelines to help service owners keep their software secure.

Apache Kafka is a high-throughput distributed messaging system that can be used for building real-time data pipelines and streaming apps. It provides a publish-subscribe messaging model and is designed as a distributed commit log. Kafka allows for both push and pull models where producers push data and consumers pull data from topics which are divided into partitions to allow for parallelism.

These are the slides of my Kafka talk at Apache: Big Data Europe in Budapest, Hungary. Enjoy! --Michael Apache Kafka is a high-throughput distributed messaging system that has become a mission-critical infrastructure component for modern data platforms. Kafka is used across a wide range of industries by thousands of companies such as Twitter, Netflix, Cisco, PayPal, and many others. After a brief introduction to Kafka this talk will provide an update on the growth and status of the Kafka project community. Rest of the talk will focus on walking the audience through what's required to put Kafka in production. We’ll give an overview of the current ecosystem of Kafka, including: client libraries for creating your own apps; operational tools; peripheral components required for running Kafka in production and for integration with other systems like Hadoop. We will cover the upcoming project roadmap, which adds key features to make Kafka even more convenient to use and more robust in production.

Habitat intro for CodeMonsters. For the code used in the demo, see https://github.com/lnxchk/container_sched_backend/

This document discusses different cloud computing layers (IaaS, PaaS, SaaS) and how IBM Integration Bus can integrate with them. It describes how tools like Chef, IBM UrbanCode Deploy, and Bluemix PaaS can be used to automate deployment and management of IIB in cloud environments. The document also discusses how IIB can connect to SaaS applications and provide APIs to expose integration services as cloud applications.

Nowadays "cloud" and "microservice" terms are used all the time, even overused. Does any system must be the "microservices" deployed in the "cloud"? Definitely not! However once you see that your system may benefit from that architecture, the next question is how to get there - how to fly to the clouds? Spring was always about simplifying the complicated aspects of your enterprise system. Netflix went to microservice architecture long before this term even was created. Both are very much contributed to open source software. How can you benefit from joint forces of the both?

Nowadays "cloud" and "microservice" terms are used all the time, even overused. Does any system must be the "microservices" deployed in the "cloud"? Definitely not! However once you see that your system may benefit from that architecture, the next question is how to get there - how to fly to the clouds? Spring was always about simplifying the complicated aspects of your enterprise system. Netflix went to microservice architecture long before this term even was created. Both are very much contributed to open source software. How can you benefit from joint forces of the both?

One of the great things about running applications in the cloud is that you only pay for the resources that you use. But that also makes it more important than ever for our applications to be resource-efficient. This becomes even more critical when we use serverless functions. Micronaut is an application framework that provides dependency injection, developer productivity features, and excellent support for Apache Kafka. By performing dependency injection, AOP, and other productivity-enhancing magic at compile time, Micronaut allows us to build smaller, more efficient microservices and serverless functions. In this session, we'll explore the ways that Apache Kafka and Micronaut work together to enable us to build fast, efficient, event-driven applications. Then we'll see it in action, using the AWS Lambda Sink Connector for Confluent Cloud.

Automation of infrastructure is one of the key tenants of DevOps. Chef has been at the vanguard of "Infrastructure as Code", where the configuration and management of your applications and servers is automated and tracked as source code. This infrastructure source code may be tested, shared and tracked just like any other software project. Traditionally configuration management has meant physical, virtual and cloud servers but Cisco and Chef are working together to extend this into networking. This session will provide an introduction to Chef and the current state of Cisco integrations, network automation scenarios and the roadmap ahead.

Chef is an open source framework for managing infrastructure as code. It uses a declarative language and recipes to describe and configure systems to match a desired state. Chef works by having clients pull policies from a server and apply them locally, then reporting the state back. Cisco and Chef are working together to bring official Chef support to Cisco's NX-OS and IOS-XR platforms through Omnibus packages and a Cisco cookbook with resources for managing Cisco devices. This will allow network infrastructures to be managed as code through testing, versioning, and continuous delivery practices.

Kafka is primarily used to build real-time streaming data pipelines and applications that adapt to the data streams. It combines messaging, storage, and stream processing to allow storage and analysis of both historical and real-time data.

The document summarizes an agenda for an HBase Meetup at Cask HQ. The agenda includes announcements about Cask's newly open sourced projects - CDAP (Cask Data Application Platform), Coopr (cluster provisioning), and Tigon (real-time streaming on YARN and HBase). It also lists talks on using HBase at Flipboard and master topologies after HBase 1.0. Cask is now fully open source and aims to build communities around these projects to help more developers build applications on Hadoop platforms.

This document provides an overview of the SharePoint Framework (SPFx) including its development and deployment process. It discusses setting up the development environment with Node.js, Yeoman and Gulp. It describes building, bundling and packaging solutions as well as enabling the app catalog and deploying SPFx apps. It also covers using Azure DevOps for continuous integration and delivery pipelines along with its predefined variables and Office 365 CLI commands.

DevOps, Continuous Integration & Deployment on AWS discusses practices for software development on AWS including DevOps, continuous integration, continuous delivery, and continuous deployment. It provides an overview of AWS services that can be used at different stages of the software development lifecycle such as CodeCommit for source control, CodePipeline for release automation, and CodeDeploy for deployment. National Novel Writing Month (NaNoWriMo) maintains its websites and services on AWS to support its annual writing challenge. It migrated to AWS to improve uptime and scalability. Its future goals include porting older sites to Rails, using Amazon SES for email, load balancing with ELB, implementing auto scaling, and using services like CodeDeploy, SNS

Organizations around the globe are leveraging the cloud to accomplish world-changing missions. This session will address how AWS can help organizations put more money toward their mission and scale outreach and operations to achieve more with less. Hear some of AWS’s most advanced customers on how their organizations handle DevOps, continuous integration and deployment. Learn how these practices allow them to rapidly develop, iterate, test and deploy highly-scalable web applications and core operational systems on AWS. The discussion will focus on best practices, lessons learned, and the specific technologies and services they use.

1) The document discusses using Spring Cloud Netflix to build cloud native applications. It describes key concepts like microservices, twelve-factor apps, and Netflix OSS projects like Eureka and Hystrix that Spring Cloud Netflix integrates. 2) Spring Cloud Netflix enables common patterns for distributed systems through annotations, including service discovery with Eureka, circuit breaking with Hystrix, load balancing with Ribbon, and declarative REST clients with Feign. 3) The document provides a demo example of a microservices architecture using Spring Cloud Netflix with separate services for news and articles that are discovered and load balanced, with circuit breaking protection.

This document discusses integration in the age of DevOps. It describes how microservices help solve the problem of decoupling services and teams to move quickly at scale. Apache Camel is presented as a solution for integration that allows for reliable and distributed integration through mechanisms like messaging. Kubernetes and Docker are discussed as platforms that help develop and run microservices locally and at scale by providing automation, configuration, isolation and service discovery capabilities.

One of the great things about running applications in the cloud is that you only pay for the resources that you use. But that also makes it more important than ever for our applications to be resource-efficient. This becomes even more critical when we use serverless functions. Micronaut is an application framework that provides dependency injection, developer productivity features, and excellent support for Apache Kafka. By performing dependency injection, AOP, and other productivity-enhancing magic at compile time, Micronaut allows us to build smaller, more efficient microservices and serverless functions. In this session, we'll explore the ways that Apache Kafka and Micronaut work together to enable us to build fast, efficient, event-driven applications. Then we'll see it in action, using the AWS Lambda Sink Connector for Confluent Cloud.

* Quick Intro to Bigtop * Trend Micro Big Data Platform * Mission-specific Platform * Big Data Landscape (3p) * Bigtop 1.1 Release (6p)

A case study of the tools and techniques used at Gilt Groupe to develop and deploy a system composed of over 200 micro-services.

Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states. In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing. Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.

This is a powerpoint that features Microsoft Teams Devices and everything that is new including updates to its software and devices for May 2024

Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment. How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.

The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.

To help you choose the best DiskWarrior alternative, we've compiled a comparison table summarizing the features, pros, cons, and pricing of six alternatives.

Today’s digitally connected world presents a wide range of security challenges for enterprises. Insider security threats are particularly noteworthy because they have the potential to cause significant harm. Unlike external threats, insider risks originate from within the company, making them more subtle and challenging to identify. This blog aims to provide a comprehensive understanding of insider security threats, including their types, examples, effects, and mitigation techniques.