OpenTelemetry For Architects

This is my talk at Indonesia Open Infrastructure Day 2020 - Adopting Open Telemetry on Your Microservices and Use It as Your Trace Capture

This document provides an overview of OpenTelemetry, including: - OpenTelemetry is an observability framework that assists in generating and capturing telemetry data from cloud-native software across traces, metrics, and logs. - It includes vendor-agnostic APIs, SDKs, and tools for generating, collecting, and exporting telemetry data to analysis tools. - OpenTelemetry has reached general availability for tracing and is in release candidate for metrics, with client libraries available for many popular programming languages.

Slide deck from the ASEAN Cloud Summit meetup on 27 January 2022. The session cover the following topics 1 - Centralized Loggin with Elasticsearch, Fluentbit and Kibana 2 - Monitoring and Alerting with Prometheus and Grafana 3 - Exception aggregation with Sentry The live demo showcased these aspects using Azure Kubernetes Service (AKS)

- OpenTelemetry is a unified observability library that provides APIs, SDKs, and middleware for generating and exporting traces, metrics, and logs. It combines OpenCensus and OpenTracing into a single open source project. - The OpenTelemetry architecture includes APIs, SDKs, a collector, and exporters. The SDK implements the API to generate traces and metrics. The collector receives telemetry data from the SDK and exports it to backends like Jaeger and Prometheus. - The document demonstrates how to instrument a Go application with OpenTelemetry tracing by creating spans from HTTP requests and exporting them to Jaeger for analysis.

This document discusses using OpenTelemetry, Grafana, and Promscale to gain insights into distributed systems. It summarizes OpenTelemetry for instrumentation, Promscale as an observability backend built on TimescaleDB that allows analyzing metrics, traces and business data together, and demonstrates this using a lightweight microservices demo that generates absurd passwords. The demo can be run locally and visualized in Grafana.

This presentation has been presented at the "Vienna DevOps & Security Meetup" in 2021. It discusses the state of monitoring, what Opentelemetry is and why should you care about it. Concepts and basics are discussed and presented in a full example extracting traces, metrics and logs. Demo: https://github.com/secustor/opentelemetry-meetup

No production system is complete without a way to monitor it. In software, we define observability as the ability to understand how our system is performing. This talk dives into capabilities and tools that are recommended for implementing observability when running K8s in production as the main platform today for deploying and maintaining containers with cloud-native solutions. We start by introducing the concept of observability in the context of distributed systems such as K8s and the difference with monitoring. We continue by reviewing the observability stack in K8s and the main functionalities. Finally, we will review the tools K8s provides for monitoring and logging, and get metrics from applications and infrastructure. Between the points to be discussed we can highlight: -Introducing the concept of observability -Observability stack in K8s -Tools and apps for implementing Kubernetes observability -Integrating Prometheus with OpenMetrics

As an introduction to OpenTelemetry, it can help beginners understand the related technology trends and the general body of knowledge.

With the increasing complexity that comes with adopting distributed architectures like Kafka, it is difficult to identify and triage performance anomalies. Implementing Distributed Tracing technologies like OpenTelemetry can add context to the messages flowing through your system, allowing us to visualize the data flow to pinpoint bottlenecks, bugs, and other performance issues. However, implementing end-to-end distributed tracing can be challenging for multi-request and asynchronous processes built on Kafka. This talk will walk through how to use OpenTelemetry to tell the full story of a request as it travels through your Kafka producer, queue, and consumer. First, we will learn how context propagation works in OpenTelemetry with W3C and B3 protocols. Then, we will go through the problems faced when instrumenting asynchronous requests. Finally, we will go through the context propagation implementation of the OpenTelemetry Kafka exporter, tracing multi-request and asynchronous processes.

This document provides an introduction to open source observability tools including Grafana, Prometheus, Loki, and Tempo. It summarizes each tool and how they work together. Prometheus is introduced as a time series database that collects metrics. Loki is described as a log aggregation system that handles logs at scale without high costs. Tempo is explained as a tracing system that allows tracing from logs, metrics, and between services. The document emphasizes that these tools can be run together to gain observability across an entire system from logs to metrics to traces.

Modern cloud-native applications are incredibly complex systems. Keeping the systems healthy and meeting SLAs for our customers is crucial for long-term success. In this session, we will dive into the three pillars of observability - metrics, logs, tracing - the foundation of successful troubleshooting in distributed systems. You'll learn the gotchas and pitfalls of rolling out the OpenTelemetry stack on Kubernetes to effectively collect all your signals without worrying about a vendor lock in. Additionally we will replace parts of the Prometheus stack to scrape metrics with OpenTelemetry collector and operator.

OpenTelemetry is an open source observability framework that provides standard APIs and SDKs for instrumenting applications to generate and collect telemetry data. It aims to solve the problem of different APM tools requiring re-instrumentation of code by providing a common data format. OpenTelemetry consists of SDKs that instrument code to generate traces and metrics, exporters to send that data to backends, and a collector service that can receive telemetry in a vendor-agnostic way before exporting to various APM solutions. It is supported by many cloud providers and vendors but does not include its own monitoring backend.

This is a talk on how you can monitor your microservices architecture using Prometheus and Grafana. This has easy to execute steps to get a local monitoring stack running on your local machine using docker.

This document discusses concepts related to observability including Prometheus, ELK stack, OpenTracing, and Victoria Metrics. It provides examples of setting up Prometheus and Grafana to monitor metrics from applications instrumented with exporters. It also demonstrates setting up Filebeat, Logstash and Elasticsearch (ELK stack) to monitor logs and send them to Elasticsearch. Additionally, it shows how to implement OpenTracing in a Java application and visualize traces using Jaeger. Finally, it outlines an exercise to build a microservices ecommerce application incorporating logging, metrics and tracing using the discussed tools.

2022.1.6 @ Will 保哥的技術交流中心 facebook 直播分享 隨著系統架構越來越複雜，服務數量也不停地成長，越來越多人開始使用分散式追蹤 (Distributed Tracing) 來紀錄微服務架構下的各種紀錄。OpenTelemetry 是一套雲原生的可觀測性框架，它提供了標準化的 API、SDK 與協議，支援 W3C 定義的追蹤規範 (W3C trace context specification)，同時提供在開發者可以在 Pipleline 上進行擴展，降低開發者在蒐集遙測數據上的困難度。 希望透過這次分享介紹 OpenTelemetry 這個主題，內容將涵蓋 OpenTelemetry 解決了什麼樣的問題，以及如何做到服務之間的數據追蹤、蒐集與分析，讓各位對於 OpenTelemetry 有個基本概念！ 👍 範例程式 ▶ https://github.com/marcustung/OpenTelemetrySample

This document provides an overview of GitOps and summarizes a training session on the topic. The session covered Kubernetes and Git basics, the motivation and model for GitOps, an example of GitOps in action using Flux on a training environment, progressive delivery techniques like Flagger, and challenges with GitOps adoption. The goals were to explain what GitOps is, understand benefits, gain hands-on experience, and decide if it's right for a team/project. GitOps aims to use Git as the single source of truth for infrastructure and automate deployments by reconciling production with the code repository.

Slides from OpenSource101.com Talk (https://opensource101.com/sessions/wtf-is-gitops-why-should-you-care/) If you’re interested in learning more about Cloud Native Computing or are already in the Kubernetes community you may have heard the term GitOps. It’s become a bit of a buzzword, but it’s so much more! The benefits of GitOps are real – they bring you security, reliability, velocity and more! And the project that started it all was Flux – a CNCF Incubating project developed and later donated by Weaveworks (the GitOps company who coined the term). Pinky will share from personal experience why GitOps has been an essential part of achieving a best-in-class delivery and platform team. Pinky will give a brief overview of definitions, CNCF-based principles, and Flux’s capabilities: multi-tenancy, multi-cluster, (multi-everything!), for apps and infra, and more. Pinky will cover a little of Flux’s microservices architecture and how the various components deliver this robust, secure, and trusted open source solution. Through the components of the Flux project, users today are enjoying compatibility with Helm, Jenkins, Terraform, Prometheus, and more as well as with cloud providers such as AWS, Azure, Google Cloud, and more. Join us for this informative session and get all of your GitOps questions answered by an end user in the community! Speaker: Priyanka (aka “Pinky”) is a Developer Experience Engineer at Weaveworks. She has worked on a multitude of topics including front end development, UI automation for testing and API development. Previously she was a software developer at State Farm where she was on the delivery engineering team working on GitOps enablement. She was instrumental in the multi-tenancy migration to utilize Flux for an internal Kubernetes offering. Outside of work, Priyanka enjoys hanging out with her husband and two rescue dogs as well as traveling around the globe.

The objective of the engagement is for Citi to have an understanding and path forward to monitor their Confluent Platform and - Platform Monitoring - Maintenance and Upgrade

Join this info-packed and hands-on workshop where we will cover:  Introduction to Kubernetes & GitOps talk: We'll cover the most popular path that has brought success to many users already - GitOps as a natural evolution of Kubernetes. We'll give an overview of how you can benefit from Kubernetes and GitOps: greater security, reliability, velocity and more. Importantly, we cover definitions and principles standardized by the CNCF's OpenGitOps group and what it means for you.  Get Started with GitOps: You'll have GitOps up and running in about 30 mins using our free and open source tools! We'll give a brief vision of where you want to be with those security, reliability, and velocity benefits, and then we'll support you while go through the getting started steps. During the workshop, you'll also experience in action and see demos for: * an opinionated repo structure to minimize decision fatigue * disaster recovery using GitOps * Helm charts example * Multi-cluster example * all with free and open source tools mostly in the CNCF (eg. Flux and Helm). If you have questions before or after the workshop, talk to us at #weave-gitops http://bit.ly/WeaveGitOpsSlack (If you need to invite yourself to the Slack, visit https://slack.weave.works/)

This document discusses distributed tracing and OpenTelemetry. It provides an overview of tracing concepts like spans and context propagation. It describes the OpenTelemetry architecture including specifications, instrumentation libraries, and the OpenTelemetry collector. It discusses how to instrument applications for automatic tracing and exporting telemetry data. Finally, it covers best practices for debugging distributed systems using observability data and next steps to get involved in the OpenTelemetry community.

View this video on Youtube here: https://youtu.be/tK4S8y3j5TA In this info-packed and hands-on workshop we covered:  Introduction to Kubernetes & GitOps talk: We covered the most popular path that has brought success to many users already - GitOps as a natural evolution of Kubernetes. We'll give an overview of how you can benefit from Kubernetes and GitOps: greater security, reliability, velocity and more. Importantly, we cover definitions and principles standardized by the CNCF's OpenGitOps group and what it means for you.  Get Started with GitOps: You'll have GitOps up and running in about 30 mins using our free and open source tools! We'll give a brief vision of where you want to be with those security, reliability, and velocity benefits, and then we'll support you while go through the getting started steps. During the workshop, you'll also experience in action and see demos for: - an opinionated repo structure to minimize decision fatigue - disaster recovery using GitOps - Helm charts example - Multi-cluster example - all with free and open source tools mostly in the CNCF (eg. Flux and Helm). If you have questions before or after the workshop, talk to us at #weave-gitops http://bit.ly/WeaveGitOpsSlack (If you need to invite yourself to the Slack, visit https://slack.weave.works/)

1. The document discusses developer-first workflows for building and operating microservices on Kubernetes. 2. It recommends creating self-sufficient, autonomous teams and using Kubernetes, Docker, and Envoy to provide the basic infrastructure primitives needed for distributed workflows. 3. The strategies suggested depend on the service maturity level and include using similar development and production environments for prototyping, implementing software redundancy for production services, and defining service level objectives and network observability for internal dependencies.

Prometheus is a metrics-based monitoring and alerting system and also the project with the second longest tenure within the CNCF. As such you have probably heard about it by now. We will give you a short introduction to Prometheus, what it is and why it was such a big deal when it was initially released. In all those years since then, the project has only gained speed, which provides us with the opportunity to tell you about all the exciting new features that have just been released or are in the pipeline, including opportunities to contribute to the project and its wider ecosystem. Talk at kubecon 2021

Open source is at the heart of what we do at Grafana Labs and there is so much happening! The intent of this talk to update everyone on the latest development when it comes to Grafana, Pyroscope, Faro, Loki, Mimir, Tempo and more. Everyone has had at least heard about Grafana but maybe some of the other projects mentioned above are new to you? Welcome to this talk 😉 Beside the update what is new we will also quickly introduce them during this talk.