Cloud foundry: The Platform for Forging Cloud Native Applications

This document summarizes Netflix's use of Kafka in their data pipeline. It discusses how Netflix evolved from using S3 and EMR to introducing Kafka and Kafka producers and consumers to handle 400 billion events per day. It covers challenges of scaling Kafka clusters and tuning Kafka clients and brokers. Finally, it outlines Netflix's roadmap which includes contributing to open source projects like Kafka and testing failure resilience.

This document summarizes a presentation about how Netflix shifts the curve on availability and rate of change. It discusses establishing a culture of freedom and responsibility, optimizing tools for continuous delivery, and using techniques like red/black deployment, canary analysis, and "monkey" testing to improve resilience and recovery from failures in the cloud. The presentation outlines how Netflix's tooling and culture work together to enable rapid innovation while maintaining high availability across multiple cloud regions.

A look at how observability relates to testing and more specifically how understanding the data collection behind it is key.

LINE uses Redis for caching and primary storage of messaging data. It operates over 60 Redis clusters with over 1,000 machines and 10,000 nodes to handle 25 billion messages per day. LINE developed its own Redis client and monitoring system to support client-side sharding without a proxy, automated failure detection, and scalable cluster monitoring. While the official Redis Cluster was tested, it exhibited some issues around memory usage and maximum node size for LINE's large scale needs.

Most databases are based on architectures that pre-date advances to modern hardware. This results in performance issues, the need to overprovision, and a high total cost of ownership. In this webinar we will discuss the advances to modern server technology and take a deep dive into Scylla’s shard-per-core architecture and our asynchronous engine, the Seastar framework. Join us to learn how Seastar (and Scylla): Avoid locks and contention on the CPU level Bypass kernel bottlenecks Implement its per-core shared-nothing autosharding mechanism Utilize modern storage hardware Leverage NUMA to get the best RAM performance Balance your data across CPUs and nodes for best and smoothest performance Plus we’ll cover the advantages of unlocking vertical scalability.

The presentation from our online webinar "Design patterns for microservice architecture". Full video from webinar available here: https://www.youtube.com/watch?v=826aAmG06KM If you’re a CTO or a Lead Developer and you’re planning to design service-oriented architecture, it’s definitely a webinar tailored to your needs. Adrian Zmenda, our Lead Dev, will explain: - when microservice architecture is a safe bet and what are some good alternatives - what are the pros and cons of the most popular design patterns (API Gateway, Backend for Frontend and more) - how to ensure that the communication between services is done right and what to do in case of connection issues - why we’ve decided to use a monorepo (monolithic repository) - what we’ve learned from using the remote procedure call framework gRPC - how to monitor the efficiency of individual services and whole SOA-based systems.

By Tom Wilkie, delivered at London Microservices User Group on 2/12/15 The rise of microservice-based applications has had many knock-on effects, not least on the complexity of monitoring your application. Order-of-magnitude increase in the number of moving parts and rate of change of the application require us to reassess traditional monitoring techniques. In this talk we will discuss some different approaches to monitoring, visualising and tracing containerised, microservices-based applications. We’ll present different techniques to some of the emergent problems, and try not to rant too much.

From DataEngConf 2017 - Everybody wants to get to data faster. As we move from more general solution to specific optimization techniques, the level of performance impact grows. This talk will discuss how layering in-memory caching, columnar storage and relational caching can combine to provide a substantial improvement in overall data science and analytical workloads. It will include a detailed overview of how you can use Apache Arrow, Calcite and Parquet to achieve multiple magnitudes improvement in performance over what is currently possible.

This document discusses DevOps practices at Amazon, including: 1. Amazon uses DevOps practices like continuous integration, deployment, and automation to deploy code changes frequently and reliably, with mean deployment times of 11.6 seconds and up to 10,000 deployments in an hour. 2. Adopting DevOps practices has led to a 75% reduction in outages from software deployments and a 90% reduction in outage minutes since 2006. 3. The document outlines DevOps tools and practices used at Amazon like AWS services for version control, continuous integration, deployment automation, and monitoring.

Observability refers to the ability to infer the internal state of a system from its external outputs. It is a property of the system, not an action like monitoring. For a system to be observable, it must externalize its state through logs, metrics, and events. Improving observability involves monitoring all components of an application from the front-end to backend services to infrastructure. Common metrics include requests processed, errors encountered, and response times for applications as well as CPU usage, disk I/O, and network traffic for infrastructure. Observability extends monitoring by helping understand why a system is not working in addition to whether it is working.

Sizing a database cluster makes or breaks your application. Too small and you could sustain spikes in usage and recover from a node loss or an operational slowdown. Too big and your cluster will cost more and waste valuable human resources. Since different workloads have different requirements, successful sizing of your application should be optimized for both throughput and latency performance. However, in many cases, the requirements for each contradicts each other. In this webinar, we explain how to remediate the contradicting forces and build a sustainable cluster to meet both performance and resiliency requirements.

Microservices and containers are now influencing application design and deployment patterns. Sixty percent of all new applications will use cloud-enabled continuous delivery microservice architectures and containers. Service discovery, registration, and routing are fundamental tenets of microservices. Kubernetes provides a platform for running microservices. Kubernetes can be used to automate the deployment of Microservices and leverage features such as Kube-DNS, Config Maps, and Ingress service for managing those microservices. This configuration works fine for deployments up to a certain size. However, with complex deployments consisting of a large fleet of microservices, additional features are required to augment Kubernetes.

From the monitoring, organization type, on call, incident response, RCA to discuss how to build a healthy On-Call Culture

Slides for Amey Banarse's, Principal Data Architect at Yugabyte, "Distributed Databases Deconstructed: CockroachDB, TiDB and YugaByte DB" webinar recorded on Oct 30, 2019 at 11 AM Pacific. Playback here: https://vimeo.com/369929255

Communication between Microservices is inherently unreliable. These integration points may produce cascading failures, slow responses, service outages. We will walk through stability patterns like timeouts, circuit breaker, bulkheads and discuss how they improve stability of Microservices.

In a microservices world, applications consist of dozens, hundreds, or even thousands of components. Manually deploying and verifying deployment quality in production is virtually impossible. Kubernetes, which natively supports rolling updates, enables blue-green application deployments with Spinnaker. However, gradual rollouts is a feature that doesn't come out-of-the-box but can be achieved by adding Istio and Prometheus to the equation. During this meetup, Slava Koltovich, CEO of Kublr, and Oleg Atamanenko, Senior Software Architect, discussed canary release implementations on Kubernetes with Spinnaker, Istio, and Prometheus. They examined the role of each tool in the process and how they are all connected. During a demo, they demonstrated a successful and a failed canary release, and how these tools enable IT teams to properly roll out changes to their customer base without any downtime.

Building Cloud-Native App Series - Part 9 of 11 Microservices Architecture Series CI-CD Jenkins, GitHub Actions, Tekton

The document discusses the shift towards cloud platforms and microservices architectures to enable continuous delivery. It argues that platforms are needed to manage the increasing complexity of distributed systems and provide services like deployment, scaling, and monitoring. The Cloud Foundry platform is presented as fulfilling this need by automating operations and allowing developers to focus on building applications instead of infrastructure. The vision is for a ubiquitous, flexible, portable, and interoperable cloud computing environment underpinning a large ecosystem of applications.

Cloud Foundry is an open source platform that allows developers to build, deploy, and manage cloud applications. It provides tools for continuous integration, deployment, and scaling of applications. The platform handles tasks like provisioning infrastructure, load balancing, and managing services so developers can focus on their code. Cloud Foundry uses containers and a buildpack system to make applications portable and scalable across different cloud environments.

For the DevOps LA Meetup January 18, 2017 - An introduction to Habitat, a new open source application by Chef for application automation.

This document discusses secrets of successful adoptions of Cloud Foundry. It provides examples of companies that have used Cloud Foundry to improve operations, increase developer productivity, and enhance security. Specific outcomes mentioned include reducing wait times, increasing revenue, and performing updates more frequently. It also discusses metrics for measuring the success of digital transformations and emphasizes the importance of measuring the right metrics.

Presentazione dello speech tenuto da Carmine Spagnuolo (Postdoctoral Research Fellow - Università degli Studi di Salerno/ ACT OR) dal titolo "Technology insights: Decision Science Platform", durante il Decision Science Forum 2019, il più importante evento italiano sulla Scienza delle Decisioni.

Do more with less, the pain of the modern architect. High cohesion & low coupling, high availability & scale, ease of DevOps. Our systems need to support all these quality attributes, while providing more functionality with less resources. We need to be agile, we need to embrace changes, we need to have a better way! Micro-Service-Architecture (MSA) promises to bring cure to the architect's pains, but does it really deliver? This lecture presents the essence of MSA, how does it answer main concerns of modern distributed systems, how to get started, how to migrate current solutions to MSA by adopting an evolution migration path. What to be careful about and the signs that we are on the right track. We will talk about SA evolution, the CAP theorem and eventually consistency, MSA principles, hosting. containers, versioning, orchestrators & decoupling business processes. By the end of this lecture the participant will have a better understanding of why, when and how to embrace MSA.

CEO Sam Ramji LinuxCon keynote on the end of competitive advantage and the increase in foundations across the software ecosystem.

Financial services organizations are adopting new technologies like cloud computing, big data, artificial intelligence, blockchain, and Internet of Things to improve business agility, reduce costs, and gain new insights from data. MongoDB is helping in areas like cloud data strategy, blockchain applications, mainframe offloading, and powering Internet of Things applications by providing a flexible, scalable database that can be deployed across on-premises, private cloud, and public cloud environments.

IBM’s Steve Barbieri and Chad Holliday show how enterprise customers are using blueprints to develop their infrastructure and application layers across different cloud environments - helping them "make the move to cloud" in 2017.

This document outlines an agenda for a .NET cloud-native bootcamp. The bootcamp will introduce practices, platforms and tools for building modern .NET applications, including microservices, Cloud Foundry, and cloud-native .NET technologies and patterns. The agenda includes sessions on microservices, Cloud Foundry, hands-on exercises, and a wrap up. Break times are scheduled between sessions.

From Multi-Cloud and MicroServices to12-Factor Apps, Cloud-Native Applications are designed to be fast, tested and fail safe with continuous deployment to production. Simple policy declaration and enforcement across your stack allow you to move at greater speed, safety, and scale.

DevOps in Practices document provides an overview of DevOps practices and microservice architecture. It discusses that DevOps aims to reduce the time between introducing changes to a system and deploying those changes in a production environment. Microservices architecture breaks applications into smaller, independent services that are built around business capabilities. Netflix is highlighted as an example that pioneered this approach at a large scale using AWS. Key aspects of DevOps like continuous integration, infrastructure as code, and automated testing are explained in the context of enabling faster delivery with microservices.

Surush Samani of CloudNative Labs gives a demo on how to use Operators on Kubernetes to deploy a camunda engine and maintain it.

El desarrollo orientado hacia la nube es una realidad. Muchas empresas han reemplazado sus herramientas y modificado sus operaciones para obtener beneficios ofrecidos por este nuevo paradigma. Durante esta sesión se pretende abordar temas relacionados con el surgimiento de estas tecnologías. Entre los cuales destacan los distintos modelos de servicio y despliegue, estrategias para la adopción y el uso de herramientas existentes como Kubernetes.

The document discusses cloud-native application architectures and how they enable speed, safety, and scale through approaches like twelve-factor applications and microservices. It outlines the cloud-native stack and where governance is needed to secure different components like code, orchestration tools, containers, services, and infrastructure. The document argues that while cloud-native approaches are well-suited for technology companies, traditional enterprises face challenges in fully adopting these architectures due to differences in priorities, skills, and scale.

The document discusses microservices and provides information on: - The benefits of microservices including faster time to market, lower deployment costs, and more revenue opportunities. - What defines a microservice such as being independently deployable and scalable. - Differences between monolithic and microservice architectures. - Moving applications to the cloud and refactoring monolithic applications into microservices. - Tools for building microservices including Azure Service Fabric and serverless/Functions. - Best practices for developing, deploying, and managing microservices.

This presentation talks about - What is DevOps? Why it's required for the Information Technology industry? And, more importantly, what are the DevOps trend in 2019 and later.

Banjot Chanana is Senior Director of Product Management at Docker bringing solutions for enterprises to build, ship and run Docker applications on-premise or in their virtual private clouds.

The document discusses 4 reasons to use a cloud-native Kafka service like Confluent Cloud instead of managing Kafka yourself. It notes that managing Kafka requires significant investment of time and resources for tasks like architecture planning, cluster sizing, software upgrades, and more. A cloud-native service handles all operational overhead automatically so you can focus on your core business. Confluent Cloud specifically offers elastic scaling, infinite data retention, global access across clouds, and integrations to make it a complete data streaming platform.

Today it’s all about delivering velocity without compromising on quality, yet it’s becoming increasingly difficult for organisations to keep up with the challenges of current release management and traditional operations. The demand for developers to own the end-to-end delivery, including operational ownership, is increasing. A “you build it, you own it” development process requires tools that developers know and understand. So I’d like to introduce “GitOps”- an agile software lifecycle for modern applications. In this session, I will discuss these industry challenges, including current CICD trends and how they’re converging with operations and monitoring. I’ll also illustrate the GitOps model, identify best practices and tools to use, and explain how you can benefit from adopting this methodology inherited from best practices going back 10-15 years.

The document discusses how Cloud Foundry, an open source cloud application platform, helps companies innovate faster. It provides examples of how Comcast was able to upgrade infrastructure with zero downtime and how a large advertising network reduced time to market for new campaigns by 5 times using Cloud Foundry. The platform allows for diverse contributors to collaborate and partners to provide support and services to customers.

Cloud Foundry is an open source platform that provides a scalable and secure environment for running applications. It offers great developer experience through features like cf push that allow developers to deploy applications with a simple command. The platform supports multiple programming languages, frameworks, databases and services through its use of containers and integration with tools like Docker and Kubernetes. It also provides security features like CredHub for managing secrets and policies across applications and infrastructure.