Why reinvent the wheel at Criteo?

•Download as PPTX, PDF•

1 like•483 views

Growing toward Criteo scale means that sometimes we do need to re-invent the wheel. We will share with you real life examples of what we have done in our C# stack to achieve that scale. For instance what happens when traditional load balancing is too costly to scale ? What about migrating monolithic application to service oriented when you have got spaghetti code ? You should monitor everything right ? But can you and should you measure and monitor your code down to the task level on your production machines. Apache Kafka looks good and you want to use it but is there any good enough implementation in C# ?

Cedrick MONTOUT
2016 June 8th
A story about C# @ Criteo
Why do we reinvent
the wheel ?

2 | Copyright © 2016 Criteo
• Criteo
• Global leader on retargeting
• Scalability
• One of many groups in the Criteo R&D department
• WebScale
• We write code to help scale up the real-time Criteo Platform
• https://www.linkedin.com/in/kerdrek
• https://github.com/kerdrek
• @kerdrek
Who am I ?

4 | Copyright © 2016 Criteo
The situation then … 1/5
Footer:
App A Pool App B Pool
HA Proxy
Service Pool

5 | Copyright © 2016 Criteo
The situation then … 2/5
Footer:
App A Pool App B Pool
HA Proxy
Service Pool
Random DC Service Pool
Traffic 185k QPS
Input size ~5K bytes
Output size ~3,5K bytes
Network traffic ~12Gbits/s

6 | Copyright © 2016 Criteo
The situation then … 3/5
Footer:
App A Pool App B Pool
HA Proxy
Service Pool
Random DC Service Pool
Traffic 185k QPS
Input size ~5K bytes
Output size ~3,5K bytes
Network traffic ~12Gbits/s
Compression
30% gain

7 | Copyright © 2016 Criteo
The situation then … 4/5
Footer:
App A Pool App B Pool
HA Proxy
Service Pool
Random DC Service Pool
Traffic 185k QPS
Input size ~5K bytes
Output size ~3,5K bytes
Network traffic ~12Gbits/s
Compression
30% gain
Bonding 2 ports
Combine two physical data links into one logical link, by
connecting 2 ports of the switch to 2 network interfaces
of the HAProxy

8 | Copyright © 2016 Criteo
The situation then … 5/5
Footer:
App A Pool App B Pool
HA Proxy
Service Pool
Random DC Service Pool
Traffic 185k QPS
Input size ~5K bytes
Output size ~3,5K bytes
Network traffic ~12Gbits/s
Compression
30% gain
Bonding 2 ports
Combine two physical data links into one logical link, by
connecting 2 ports of the switch to 2 network interfaces
of the HAProxy
4 pairs

9 | Copyright © 2016 Criteo
The new wheel : Client Side Load Balancing
• Bypass HA Proxy
• Implemented inside Twitter/Finagle
• Reuse existing health Check
• Re-implement monitoring
App Pool A App Pool B
Service Pool

11 | Copyright © 2016 Criteo
The situation then …
Footer:
• Harder and harder to release
• No available memory for new feature
• Fragmented feature in production
• Tightly coupled with the HTTP stack

12 | Copyright © 2016 Criteo
The new wheel: Component
• One Input
• One Output
• On Process

13 | Copyright © 2016 Criteo
The new wheel: A Host with Services
• A collection of services

14 | Copyright © 2016 Criteo
The new wheel: DevHost
• The DevHost
• Several Components
• Several Services
• Asynchronous Process
• Transport agnostic
• Not front facing

15 | Copyright © 2016 Criteo
The new wheel: DevHost
• Still use in production
• ~45% of the windows production machine
• Next iteration will use .net core (WiP)

17 | Copyright © 2016 Criteo
• Synchronous processing everywhere
• Timeout on several pipeline (loosing money)
• No clear diagnostic on execution path
The situation then …
Footer:

18 | Copyright © 2016 Criteo
The new Wheel : Asynchronous Token Framework
• TPL was not a solution at that time
• Asynchronous Completion Token
• Delegate based
• Execution is time boxed
• Task underneath
• Timing for every thing
• Metrics available on the machine
• Metrics available aggregated on Graphite

20 | Copyright © 2016 Criteo
The situation then …
Footer:
• Syslog
• Text based
• Fire and Forget
• Single messages
• No built-in resiliency
• No API for consuming

21 | Copyright © 2016 Criteo
The situation then …
Footer:
• Apache Kafka
• Binary
• Acknowledged message
• Batched messages
• Partitioning and replication
• Consuming support
• Syslog
• Text based
• Fire and Forget
• Single messages
• No built-in resiliency
• No API for consuming

22 | Copyright © 2016 Criteo
Apache Kafka where is your C# driver ?
We looked at several drivers

23 | Copyright © 2016 Criteo
Goldilocks Conundrum all over again
We looked at several drivers
First driver never used in production

24 | Copyright © 2016 Criteo
Goldilocks Conundrum all over again
We looked at several drivers
First driver never used in production
Second driver was impossible to unit test

25 | Copyright © 2016 Criteo
Goldilocks Conundrum all over again
We looked at several drivers
First driver never used in production
Second driver was impossible to unit test
Third driver was not recently maintained

26 | Copyright © 2016 Criteo
The new Wheel: kafka-sharp
• Yet another C# driver
• Highly tuneable
• Written with perf and scale in mind
• Battle tested in production
• Available here: https://github.com/criteo/kafka-sharp

27 | Copyright © 2016 Criteo
The wheel lists
• Distributed load balancer between clients.
• Lightweight hosting server.
• Low level asynchronous execution framework.
• Yet another C# driver for Apache Kafka.

What's hot

Quick workflow of a nodejs api

Paolo Carrasco Mori

GitOps Toolkit (Cloud Native Nordics Tech Talk)

Weaveworks

GitOps with Gitkube

Tirumarai Selvan

Kubernetes GitOps featuring GitHub, Kustomize and ArgoCD

Sunnyvale

DevOps Fest 2020. Дмитрий Кудрявцев. Реализация GitOps на Kubernetes. ArgoCD

DevOps_Fest

The document discusses GitOps and ArgoCD for managing Kubernetes applications. It defines GitOps as storing the desired state of systems in Git repositories and using continuous delivery tools to ensure the live systems match that state. ArgoCD is introduced as a GitOps tool that monitors applications and ensures the running state matches the target state defined in Git. Key features of ArgoCD include a web UI, automated deployments, support for different config formats, and rollback capabilities. The document provides an example of using Kustomize to customize Kubernetes resources through overlays.

CNCF: Cross-cloud CI v1.0.0 Dashboard Release 2018-01-26

Vulk Coop

What's Coming in Apache Airflow 2.0 - PyDataWarsaw 2019

Jarek Potiuk

Gitops Hands On

Brice Fernandes

GitLab Frontend and VueJS at GitLab

Fatih Acet

Introduction of cloud native CI/CD on kubernetes

Kyohei Mizumoto

Using GitLab CI

ColCh

The Power of GitOps with Flux & GitOps Toolkit

Weaveworks

GitOps Days Community Special Watch the video here: https://youtu.be/0v5bjysXTL8 New to GitOps or been a long-time Flux user? We'll walk you through the benefits of GitOps and then demo it in action with a sneak peak into the next gen Flux and GitOps Toolkit! * Automation! * Visibility! * Reconciliation! * Powerful use of Prometheus and Grafana! * GitOps for Helm! For Flux users, Flux v1 is decoupled into Flux v2 and GitOps Toolkit. We'll demo how this decoupling gives you more control over how you can do GitOps and with fewer steps! Join Leigh Capili and Tamao Nakahara as they show you GitOps in action with Flux and GitOps Toolkit. Note to our Flux community that Flux v2 and the GitOps Toolkit is in development and Flux v1 is in maintenance mode. These talks and upcoming guides will give you the most up-to-date info and steps to migrate once we reach feature parity and start the migration process. We are dedicated to the smoothest experience possible for our Flux community, so please join us if you'd like early access and to give us feedback for the migration process. We are really excited by the improvements and want to take this opportunity to show you what the GitOps Toolkit is all about, walk you through the guides and get your feedback! For more info, see https://toolkit.fluxcd.io/. Here's our latest blog post on Flux v2 and GitOps Toolkit updates: https://www.weave.works/blog/the-road-to-flux-v2-october-update

Docker New York City: From GitOps to a scalable CI/CD Pattern for Kubernetes

Andrew Phillips

Case Study: Migration to GitLab (from Bitbucket) at AppsFlyer

Noa Harel

AppsFlyer migrated from BitBucket to GitLab for their 150 users and 680 repositories. They wanted to leave the hosted BitBucket solution due to API call limits and latency. The migration process involved converting repositories from Mercurial to Git, setting up the GitLab architecture on Amazon Web Services with an EFS file system, educating teams, and creating custom tooling like a Python script to notify Slack. Lessons learned included issues restoring backups and increasing Unicorn workers. The full technical details are available at the provided URL.

Importance of GCP: 30 Days of GCP

AnshTyagi27

Google Cloud Platform (GCP) allows developers to build and deploy applications at scale. GCP provides infrastructure like virtual machines and containers to deploy applications without hardware limitations. It also offers services for continuous integration/delivery (CI/CD) pipelines, monitoring, error handling, and machine learning/artificial intelligence to add capabilities to applications. Completing a 30 day training on GCP can help engineers become more dynamic by learning how to use GCP's full suite of tools and services to build real-world applications.

GitLab London Meetup: How Containerized Pipelines and Kubernetes Can Boost Yo...

Nico Meisenzahl

Learn how to eliminate any dependencies on your CI/CD build nodes and don’t bother yourself with multiple versions of your toolchain and any corresponding constraints. Walk away with knowledge and best practices that will help you to optimize your builds and deployments with containerized pipelines! Use containerized GitLab CI/CD pipelines and Kaniko to move your build and deployment workloads into your Kubernetes cluster. Build your apps and infrastructure without any external dependencies and constraints. If you are building a Go project, deploying an app to Kubernetes or just building your infrastructure. It doesn’t matter, anything is possible! Join Nico on a deep dive into the secrets of building hassle-free containerized build and deployment pipelines with GitLab CI/CD, Kubernetes and Kaniko.

Meetup 23 - 03 - Application Delivery on K8S with GitOps

Vietnam Open Infrastructure User Group

This document summarizes a presentation about deploying applications on Kubernetes with GitOps. The presentation covers GitOps workflows and tools like FluxCD and ArgoCD for managing Helm charts from Git repositories. It also discusses integrating continuous integration pipelines with ArgoCD and provides best practices for areas like secret management, scaling, and microservices. The presenter concludes by taking questions and inviting interested parties to join their company.

A GitOps Kubernetes Native CICD Solution with Argo Events, Workflows, and CD

Julian Mazzitelli

Presented at Kubernetes and Cloud Native meetup in Toronto on December 4, 2019 See https://www.youtube.com/watch?v=YmIAatr3Who for a video recording of a similar talk. Are you looking to get more flexibility out of your CICD platform? Interested how GitOps fits into the mix? Learn how Argo CD, Workflows, and Events can be combined to craft custom CICD flows. All while staying Kubernetes native, enabling you to leverage existing observability tooling.

Knative CloudEvents

Nobuhiro Sue

Kamesh is an active open source contributor who has contributed to projects like Knative, Minishift, Eclipse Che, and fabric8. He created the vert.x-maven-plugin. Knative provides a Kubernetes-based platform for building, deploying, and managing serverless workloads and includes components for serving, building, and eventing that are essential for modern applications. It supports running workloads on-premises, in the cloud, or in third-party data centers.

CI:CD in Lightspeed with kubernetes and argo cd

Billy Yuen

Enterprises have benefited greatly from the elastic scalability and multi-region availability by moving to AWS, but the fundamental deployment model remains the same. At Intuit, we have adopted k8s as our new saas platform and re-invented our CI/CD pipeline to take full advantage of k8s. In this presentation, we will discuss our journey from Spinnaker to Argo CD. 1. Reduce CI/CD time from 60 minutes to 10 minutes. 2. Reduce production release (or rollback) from 10 minutes to 2 minutes. 3. Enable concurrent deployment using spinnaker and argo cd as HA/DR to safely adopt the new platform with no downtime. 4. Be compatible with the existing application monitoring toolset.

What's hot (20)

Quick workflow of a nodejs api

GitOps Toolkit (Cloud Native Nordics Tech Talk)

GitOps with Gitkube

Kubernetes GitOps featuring GitHub, Kustomize and ArgoCD

DevOps Fest 2020. Дмитрий Кудрявцев. Реализация GitOps на Kubernetes. ArgoCD

CNCF: Cross-cloud CI v1.0.0 Dashboard Release 2018-01-26

What's Coming in Apache Airflow 2.0 - PyDataWarsaw 2019

Gitops Hands On

GitLab Frontend and VueJS at GitLab

Introduction of cloud native CI/CD on kubernetes

Using GitLab CI

The Power of GitOps with Flux & GitOps Toolkit

Docker New York City: From GitOps to a scalable CI/CD Pattern for Kubernetes

Case Study: Migration to GitLab (from Bitbucket) at AppsFlyer

Importance of GCP: 30 Days of GCP

GitLab London Meetup: How Containerized Pipelines and Kubernetes Can Boost Yo...

Meetup 23 - 03 - Application Delivery on K8S with GitOps

A GitOps Kubernetes Native CICD Solution with Argo Events, Workflows, and CD

Knative CloudEvents

CI:CD in Lightspeed with kubernetes and argo cd

Viewers also liked

Introduction Criteo - 2.0

Scott Turecek

This document discusses Criteo's performance advertising solutions. It highlights that Criteo works with over 9,000 publishers and 7,000 advertisers across 130+ countries. Criteo uses a prediction engine and dynamic creative optimization to deliver measurable ROI at unmatched scale across desktop, mobile, in-app, and social platforms. The document also outlines Criteo's solutions for display, email, and cross-device marketing and notes that setup to go live typically takes about 4 weeks with world-class support included.

EN - Criteo - BD Deck -July 2014 - rebrand

Djilali Zitouni

Criteo provides targeted display advertising that is able to predict user clicks. It buys large advertising inventories from publishers using a cost-per-million (CPM) model and sells campaigns to advertisers on a cost-per-click (CPC) model. Criteo's Real-Time Audience solution allows it to drop cookies and show targeted ads to iOS and Safari users. This solution has increased revenue for publishers by over 100% on average by enabling more accurate frequency capping, higher volume, and price points. Integrating Criteo's RTA solution is described as easy and having no impact on page loading times.

Criteo. Reach people, not devices!

HybridRussia

Ключевым условием успеха в онлайн-продажах является возможность объединения профиля пользователя на различных устройствах, браузерах и приложениях. Из доклада вы узнаете, почему cross-device решения являются сегодня необходимыми, и как Criteo при помощи Universal Match решает эти проблемы.

New challenges for scalable machine learning in online advertising

Olivier Koch

The document discusses challenges and opportunities for machine learning in online advertising at scale. It notes that while ML has helped with tasks like bidding and recommendations, challenges remain around long-term effects, overfitting, personalization across devices, and optimal credit assignment and metrics. The document proposes that reinforcement learning, counterfactual analysis, transfer learning and factorization could help address issues like optimal bidding strategies, offline evaluation, and modeling long tail users and products. It concludes by inviting others to help solve remaining open challenges.

Making advertising personal, 4th NL Recommenders Meetup

Olivier Koch

Criteo is a performance advertising company that buys ad inventory and sells clicks at scale. They use real-time personalized product recommendations to select which ads to display to each user from billions of products. Their recommendation system retrieves candidate products for each user based on their browsing history and scores products from multiple data sources to select the top recommendations within 8 milliseconds to support their high traffic levels across many servers and data centers globally. They discuss challenges maintaining large user profiles, improving product data, and optimizing response time and independence of recommendations.

Couchbase live 2016

Pierre Mavro

Pierre Mavro from Criteo discussed Couchbase usage at their company. Criteo has over 100 Couchbase clusters storing over 90TB of data serving up to 25 million queries per second. They benchmarked Couchbase and found network bandwidth and replicas increased latency. To improve, Criteo monitored latency, split workloads across clusters, automated operations, and tuned Couchbase and systems. Their changes helped Couchbase scale for Criteo's large workload.

RecsysFR: Criteo presentation

recsysfr

1) Criteo is an advertising technology company that uses machine learning to target personalized ads to users across devices. 2) It has over 2000 employees, works with 16,000 publishers and 11,000 advertisers worldwide, and generated $11 billion for its clients in 2015. 3) Criteo's machine learning challenges include selecting the best products and ads to show users from huge catalogs in milliseconds, building a graph connecting billions of devices to identify users, and extensively testing new models and systems.

criteo-performance-advertising-playbook-2015

Carolyn Bednarz

The document provides tips for running successful performance display marketing campaigns. It outlines 5 common mistakes to avoid: 1) Not defining clear goals and KPIs, 2) Focusing on placements over understanding customer journeys, 3) Not asking the right questions of data to gain insights, 4) Allowing low quality inventory like bot traffic, and 5) Not optimizing campaigns for mobile users. Following people across devices, prioritizing premium inventory, and gaining insights from smart data analysis are some of the keys to driving maximum performance.

Средний чек как показатель эффективности персонала аптеки

Yaroslav Shulga

Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers

Criteo

Big data analytics has shown that the conventional wisdom about display ad clicks being worthless was incorrect. It revealed that people who click on ads (clickers) actually buy 3 times more frequently than non-clickers, and half of all clicks and sales come from just 20% of users. Real-time big data is now being used to serve highly targeted display ads to the right users at the right time with the right message, making the ads actually worth clicking on and representing a $20 billion opportunity for the display advertising industry.

RecSys 2015: Large-scale real-time product recommendation at Criteo

Romain Lerallut

Criteo - NOAH13 London

NOAH Advisors

Criteo is a technology company that uses machine learning to deliver personalized display advertisements to users. It analyzes large amounts of shopping behavior data to determine the right products to show users at the right time. Criteo has grown rapidly, with over $350 million in revenue in 2012 and a compound annual growth rate of 104% since 2010. It has a global presence with over 750 employees across 15 offices worldwide serving over 4,000 retailer clients. Criteo is also pursuing opportunities in mobile advertising given the growth of mobile commerce.

Damien Lefortier, Senior Machine Learning Engineer and Tech Lead in the Predi...

MLconf

Machine Learning for Display Advertising @ Scale: In this talk, we will briefly introduce the display advertising marketplace, its stakeholders and the key performance metrics. We will then present the models we have developed at Criteo for bidding in real-time auctions, product recommendation, and look & feel optimization at scale (1B+ monthly users, 3B+ products in our catalog, and 30K ad displayed / sec at peak traffic). For these tasks, we’ve moved over time from predicting rare, binary events (clicks) to predicting very rare events (sales) and continuous events (sales amounts), all of them being quite noisy, and we’ll discuss the different methods that we have tried to build these models (such as generalized linear models, trees or factorization machines). We’ll continue by discussing how we evaluate these models both offline and online. We will describe the infrastructure for large-scale distributed data processing that these algorithms rely upon and discuss different optimization techniques we have experimented with (such as SGD, L-BFGS, SVRG). Finally, we will conclude with future areas of research and discuss open challenges we are currently facing.”

LUMA Digital Brief 012 - Market Report Q4 2016

LUMA Partners

LUMA provides expertise at the intersection of media, marketing and technology. This includes digital content like online publishers, mobile gaming companies, and video platforms. It also includes advertising technology for display, video, mobile and other formats. Additionally, it includes marketing technology for data, analytics, sales automation and other tools. Mobile proliferation has had a major impact on these industries. In 2016, there was significant consolidation activity across digital media and marketing sectors through mergers and acquisitions. The deal activity was driven by growth in the pool of strategic buyers including telecommunications companies, international firms, and private equity investors.

STRATEGIC BUYER LUMAscape

LUMA Partners

DISPLAY LUMAscape

LUMA Partners

Viewers also liked (16)

Introduction Criteo - 2.0

EN - Criteo - BD Deck -July 2014 - rebrand

Criteo. Reach people, not devices!

New challenges for scalable machine learning in online advertising

Making advertising personal, 4th NL Recommenders Meetup

Couchbase live 2016

RecsysFR: Criteo presentation

criteo-performance-advertising-playbook-2015

Средний чек как показатель эффективности персонала аптеки

Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers

RecSys 2015: Large-scale real-time product recommendation at Criteo

Criteo - NOAH13 London

Damien Lefortier, Senior Machine Learning Engineer and Tech Lead in the Predi...

LUMA Digital Brief 012 - Market Report Q4 2016

STRATEGIC BUYER LUMAscape

DISPLAY LUMAscape

Similar to Why reinvent the wheel at Criteo?

DataEngConf: Apache Kafka at Rocana: a scalable, distributed log for machine ...

Hakka Labs

This document discusses Apache Kafka, a distributed streaming platform developed at LinkedIn and open sourced. It provides the following key points: - Kafka was designed at LinkedIn to act as a centralized data bus that scales horizontally, delivers events in order, decouples producers and consumers, and has low latency. - It uses a partitioned log structure to store streams of records immutably, replicating across nodes for fault tolerance. Producers publish records to topics which are divided into partitions. - Consumers can independently consume data from Kafka topics in a pull-based manner without relying on a shared storage layer. This allows for scaling of consumers without additional coordination.

The Truth About the Service Mesh Data Plane

Christian Posta

The exploration of service mesh for any organization comes with some serious questions. What data plane should I use? How does this tie in with my existing API infrastructure? What kind of overhead do sidecar proxies demand? As I've seen in my work with various organizations over the years "if you have a successful microservices deployment, then you have a service mesh whether it’s explicitly optimized as one or not." In this talk, we seek to understand the role of the data plane and how to pick the right component for the problem context. We start off by establishing the spectrum of data-plane components from shared gateways to in-code libraries with service proxies being along that spectrum. We clearly identify which scenarios would benefit from which part of the data-plane spectrum and show how modern service meshes including Istio, Linkerd, and Consul enable these optimizations.

Http Services in Rust on Containers

Anton Whalley

Fluent 2018: Tracking Performance of the Web with HTTP Archive

Paul Calvano

Have you ever thought about how your site’s performance compares to the web as a whole? Or maybe you’re curious how popular a particular web feature is. How much is too much JavaScript? The HTTP Archive has been keeping track of how the web is built since 2010. It enables you to find answers to questions about the state of the web past and present. Paul Calvano explores how the HTTP Archive works, how people are using this dataset, and some ways that Akamai has leveraged data within the HTTP Archive to help its customers.

Kubernetes Ingress to Service Mesh (and beyond!)

Christian Posta

Kubernetes users need to allow traffic to flow into and within the cluster. Treating the application traffic separately from the business logic allows presents new possibilities in how service to service traffic is served, controlled and observed — and provides a transition to intra cluster networking like Service Mesh. With microservices, there is a concept of both North / South traffic (incoming requests from end users to the cluster) and East / West (intra cluster) communication between the services. In this talk we will explain how Envoy Proxy works in Kubernetes as a proxy for both of these traffic directions and how it can be leveraged to do things like traffic shaping, security, and integrate the north/south to east/west behavior. Christian Posta (@christianposta) is Global Field CTO at Solo.io, former Chief Architect at Red Hat, and well known in the community for being an author (Istio in Action, Manning, Istio Service Mesh, O'Reilly 2018, Microservices for Java Developers, O’Reilly 2016), frequent blogger, speaker, open-source enthusiast and committer on various open-source projects including Istio, Kubernetes, and many others. Christian has spent time at both enterprises as well as web-scale companies and now helps companies create and deploy large-scale, cloud-native resilient, distributed architectures. He enjoys mentoring, training and leading teams to be successful with distributed systems concepts, microservices, devops, and cloud-native application design.

OpenTSDB for monitoring @ Criteo

Nathaniel Braun

OpenTSDB is used at Criteo for monitoring their large Hadoop infrastructure which includes over 2500 servers running many different services like HDFS, YARN, HBase, Kafka, and Storm. OpenTSDB was chosen because it can handle the scale of metrics collected, store metrics for long periods of time with fine-grained resolution, and is easily extensible to add new metrics. It uses HBase for storage which is optimized for the time series data stored in OpenTSDB and can scale to meet Criteo's needs of storing billions of data points and handling high query loads.

SD Times - Docker v2

Alvin Richards

This document discusses using containers and databases together from development to production. It addresses challenges like data redundancy, dynamic cluster formation and healing when containers start and stop. It proposes that Aerospike database combined with containers can provide data persistence, scalability, self-organization and efficient resource utilization to meet these challenges. Examples are given of building an app with Python, Aerospike and Docker Compose in development and deploying it to production behind HAProxy, including scaling the web tier and Aerospike cluster using Docker networking and the Interlock plugin.

01282016 Aerospike-Docker webinar

Aerospike, Inc.

This document discusses using Docker containers with the Aerospike NoSQL database to simplify deployment from development to production. It provides examples of building a Python/Flask application with Aerospike in Docker for development and deploying it behind a load balancer to a Docker Swarm cluster for production. It also demonstrates scaling the web and Aerospike tiers independently by launching additional Docker containers.

Using Databases and Containers From Development to Deployment

Aerospike, Inc.

This document discusses using containers and databases together from development to production. It addresses challenges like data redundancy, dynamic cluster formation and healing when containers start and stop. It proposes that existing architectures are broken and presents Aerospike as a solution, being self-organizing, self-healing and optimized for flash storage. It demonstrates building an app with Python, Aerospike and Docker, deploying to a Swarm cluster, and scaling the database and web tiers through containers.

Running Consul on Kubernetes and Beyond

Mitchell Pronschinske

Verifying Apache Kafka-Based Data Pipelines With Subhangi Agarwala | Current ...

HostedbyConfluent

Verifying Apache Kafka-Based Data Pipelines With Subhangi Agarwala | Current 2022 Trading activity is at an all time high in the financial markets. In order to handle increasing volumes/velocity of data in real time to meet clients' expectations of capturing market moves with minimal latency, we recently migrated our financial data generation pipelines from batch to stream processing. It is important to have Integration Tests in place to ensure that our system is behaving as intended and provides greater confidence in maintaining the system over time, as it can simulate potential issues that could arise in production. A single software or network glitch in processing critical financial information can lead to erroneous data and outages. Testing stream-based applications to avoid these glitches is challenging because data can come in from an array of sources and in a variety of formats like json, avro or others. In this talk, we aim to highlight the importance of integration testing, a critical verification method for stable and reliable large-scale distributed streaming applications. We will also provide a high level overview of our system, challenges faced in moving to a streaming infrastructure, explore alternatives to our proposed solution and talk through the lessons learned while working with TestContainers library to test our KafkaStreams application.

What You Missed: Red Hat Summit 2016

NetApp

Couchbase Connect 2016: Monitoring Production Deployments The Tools – LinkedIn

Michael Kehoe

Good monitoring can be the difference between a great night's sleep or hearing your phone go off at 2:37 a.m. because of a production outage. Couchbase Server provides a large number of metrics which can be overwhelming if you do not know the critical things to focus on or how to expose that information to your monitoring system. In this talk we will look at example production incidents, going in depth around specific things to monitor, and how this information can be used to find issues, work out root cause, and discover trends.

Kafka Practices @ Uber - Seattle Apache Kafka meetup

Mingmin Chen

Building data pipelines is pretty hard! Building a multi-datacenter active-active real time data pipeline for multiple classes of data with different durability, latency and availability guarantees is much harder. Real time infrastructure powers critical pieces of Uber (think Surge) and in this talk we will discuss our architecture, technical challenges, learnings and how a blend of open source infrastructure (Apache Kafka and Flink) and in-house technologies have helped Uber scale.

Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...

Docker, Inc.

The SDACK architecture stands for Spark, Docker, Akka, Cassandra, and Kafka. At TrendMicro, we adopted the SDACK architecture to implement a security event inspection platform for APT attack analysis. In this talk, we will introduce SDACK stack with Spark lambda architecture, Akka and Kafka for streaming data pipeline, Cassandra for time series data, and Docker for microservices. Specifically, we will show you how we Dockerize each SDACK component to facilitate the RD team of algorithms development, help the QA team test the product easily, and use the Docker as a Service strategy to ship our products to customers. Next, we will show you how we monitor each Docker container and adjust the resource usage based on monitoring metrics. And then, we will share our Docker security policy which ensures our products are safety before shipping to customers. After that, we'll show you how we develop an all-in-one Docker based data product and scale it out to multi-host Docker cluster to solve the big data problem. Finally, we will share some challenges we faced during the product development and some lesson learned.

Kubernetes sidecar pattern as a swiss-army knife for microservices.pptx

Abhishek Tiwari

Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...

Christian Posta

If you have an existing Java monolith, you know you must take care making changes to it or altering it in any negative way. Often times these monoliths are very valuable to the business and generate a lot of revenue. At the same time, since it’s difficult to make changes to the monolith it’s desirable to move to a microservices architecture. Unfortunately you cannot just do a big-bang migration to a greenfield architecture and will have to incrementally adopt microservices. In this talk, we’ll look at using Gloo proxy which is based on Envoy Proxy and GraphQL to do surgical, function-level traffic control and API aggregation to safely migrate your monolith to microservices and serverless functions.

Brushing skills on SignalR for ASP.NET developers

ONE BCG

Let’s take a quick peek at SignalR and how we use it. SignalR is an open-source library available in ASP.NET, to help developers add real-time features in web applications and more. Chat is the most common application that you can think of, there is a lot more you can do. SignalR can be used to add any sort of “real-time” web functionality like dashboards, collaborative tools that require simultaneous editing of documents, job progress updates, and real-time forms. It can enable completely new types of web applications that need high-frequency updates from the server. One of the most common examples is real-time gaming.

How do async ap is survive in a rest world

Red Hat

The document discusses how asynchronous APIs can survive in a REST-based world. It describes how Async APIs use an event-driven approach compared to the request-reply model of REST. The AsyncAPI specification aims to describe event-driven APIs similarly to how OpenAPI describes REST APIs. The document demonstrates how Async and REST APIs can work together using an example setup with an API gateway, manager, and message broker to connect an HTML page and Node.js client via published topics.

Twelve-Factor application pattern with Spring Framework

dinkar thakur

The document discusses the twelve-factor application pattern for developing cloud-native applications and how the Spring Framework supports this methodology. It covers the twelve factors which include having a single codebase, explicit and isolated dependencies, storing configuration in the environment, treating backing services as attached resources, separating build and run stages, making applications stateless, binding to ports, scaling out via the process model, maximizing robustness through fast startup and graceful shutdown, keeping development, staging and production environments the same, treating logs as event streams, and running admin/management tasks as one-off processes. The Spring Framework provides features that help implement many of these twelve factors for building cloud-native applications.

Similar to Why reinvent the wheel at Criteo? (20)

DataEngConf: Apache Kafka at Rocana: a scalable, distributed log for machine ...

The Truth About the Service Mesh Data Plane

Http Services in Rust on Containers

Fluent 2018: Tracking Performance of the Web with HTTP Archive

Kubernetes Ingress to Service Mesh (and beyond!)

OpenTSDB for monitoring @ Criteo

SD Times - Docker v2

01282016 Aerospike-Docker webinar

Using Databases and Containers From Development to Deployment

Running Consul on Kubernetes and Beyond

Verifying Apache Kafka-Based Data Pipelines With Subhangi Agarwala | Current ...

What You Missed: Red Hat Summit 2016

Couchbase Connect 2016: Monitoring Production Deployments The Tools – LinkedIn

Kafka Practices @ Uber - Seattle Apache Kafka meetup

Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...

Kubernetes sidecar pattern as a swiss-army knife for microservices.pptx

Leveraging Envoy Proxy and GraphQL to Lower the Risk of Monolith to Microserv...

Brushing skills on SignalR for ASP.NET developers

How do async ap is survive in a rest world

Twelve-Factor application pattern with Spring Framework

Recently uploaded

Software Engineering and Project Management - Introduction to Project Management

Prakhyath Rai

Introduction to Project Management: Introduction, Project and Importance of Project Management, Contract Management, Activities Covered by Software Project Management, Plans, Methods and Methodologies, some ways of categorizing Software Projects, Stakeholders, Setting Objectives, Business Case, Project Success and Failure, Management and Management Control, Project Management life cycle, Traditional versus Modern Project Management Practices.

UNIT I INCEPTION OF INFORMATION DESIGN 20CDE09-ID

GOWSIKRAJA PALANISAMY

Lecture 6 - The effect of Corona effect in Power systems.pdf

peacekipu

Lecture 3 Biomass energy...............ppt

RujanTimsina1

Unblocking The Main Thread - Solving ANRs and Frozen Frames

Sinan KOZAK

In the realm of Android development, the main thread is our stage, but too often, it becomes a battleground where performance issues arise, leading to ANRS, frozen frames, and sluggish Uls. As we strive for excellence in user experience, understanding and optimizing the main thread becomes essential to prevent these common perforrmance bottlenecks. We have strategies and best practices for keeping the main thread uncluttered. We'll examine the root causes of performance issues and techniques for monitoring and improving main thread health as wel as app performance. In this talk, participants will walk away with practical knowledge on enhancing app performance by mastering the main thread. We'll share proven approaches to eliminate real-life ANRS and frozen frames to build apps that deliver butter smooth experience.

IWISS Catalog 2024

Iwiss Tools Co.,Ltd

Unit 1 Information Storage and Retrieval

KishorMahale5

Online music portal management system project report.pdf

Kamal Acharya

The iMMS is a unique application that is synchronizing both user experience and copyrights while providing services like online music management, legal downloads, artists’ management. There are several other applications available in the market that either provides some specific services or large scale integrated solutions. Our product differs from the rest in a way that we give more power to the users remaining within the copyrights circle.

GUIA_LEGAL_CHAPTER_4_FOREIGN TRADE CUSTOMS.pdf

ProexportColombia1

Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...

IJAEMSJORNAL

This study primarily aimed to determine the best practices of clothing businesses to use it as a foundation of strategic business advancements. Moreover, the frequency with which the business's best practices are tracked, which best practices are the most targeted of the apparel firms to be retained, and how does best practices can be used as strategic business advancement. The respondents of the study is the owners of clothing businesses in Talavera, Nueva Ecija. Data were collected and analyzed using a quantitative approach and utilizing a descriptive research design. Unveiling best practices of clothing businesses as a foundation for strategic business advancement through statistical analysis: frequency and percentage, and weighted means analyzing the data in terms of identifying the most to the least important performance indicators of the businesses among all of the variables. Based on the survey conducted on clothing businesses in Talavera, Nueva Ecija, several best practices emerge across different areas of business operations. These practices are categorized into three main sections, section one being the Business Profile and Legal Requirements, followed by the tracking of indicators in terms of Product, Place, Promotion, and Price, and Key Performance Indicators (KPIs) covering finance, marketing, production, technical, and distribution aspects. The research study delved into identifying the core best practices of clothing businesses, serving as a strategic guide for their advancement. Through meticulous analysis, several key findings emerged. Firstly, prioritizing product factors, such as maintaining optimal stock levels and maximizing customer satisfaction, was deemed essential for driving sales and fostering loyalty. Additionally, selecting the right store location was crucial for visibility and accessibility, directly impacting footfall and sales. Vigilance towards competitors and demographic shifts was highlighted as essential for maintaining relevance. Understanding the relationship between marketing spend and customer acquisition proved pivotal for optimizing budgets and achieving a higher ROI. Strategic analysis of profit margins across clothing items emerged as crucial for maximizing profitability and revenue. Creating a positive customer experience, investing in employee training, and implementing effective inventory management practices were also identified as critical success factors. In essence, these findings underscored the holistic approach needed for sustainable growth in the clothing business, emphasizing the importance of product management, marketing strategies, customer experience, and operational efficiency.

Response & Safe AI at Summer School of AI at IIITH

IIIT Hyderabad

21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY

PradeepKumarSK3

Introduction to IP address concept - Computer Networking

Md.Shohel Rana ( M.Sc in CSE Khulna University of Engineering & Technology (KUET))

An Internet Protocol address (IP address) is a logical numeric address that is assigned to every single computer, printer, switch, router, tablets, smartphones or any other device that is part of a TCP/IP-based network. Types of IP address- Dynamic means "constantly changing “ .dynamic IP addresses aren't more powerful, but they can change. Static means staying the same. Static. Stand. Stable. Yes, static IP addresses don't change. Most IP addresses assigned today by Internet Service Providers are dynamic IP addresses. It's more cost effective for the ISP and you.

GUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdf

ProexportColombia1

Evento anual Splunk .conf24 Highlights recap

Rafael Santos

Encontro anual da comunidade Splunk, onde discutimos todas as novidades apresentadas na conferência anual da Spunk, a .conf24 realizada em junho deste ano em Las Vegas. Neste vídeo, trago os pontos chave do encontro, como: - AI Assistant para uso junto com a SPL - SPL2 para uso em Data Pipelines - Ingest Processor - Enterprise Security 8.0 (Maior atualização deste seu release) - Federated Analytics - Integração com Cisco XDR e Cisto Talos - E muito mais. Deixo ainda, alguns links com relatórios e conteúdo interessantes que podem ajudar no esclarecimento dos produtos e funções. https://www.splunk.com/en_us/campaigns/the-hidden-costs-of-downtime.html https://www.splunk.com/en_us/pdfs/gated/ebooks/building-a-leading-observability-practice.pdf https://www.splunk.com/en_us/pdfs/gated/ebooks/building-a-modern-security-program.pdf Nosso grupo oficial da Splunk: https://usergroups.splunk.com/sao-paulo-splunk-user-group/

Rohini @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Yogita Mehra Top Model Safe

binna singh$A17

IS Code SP 23: Handbook on concrete mixes

Mani Krishna Sarkar

Profiling of Cafe Business in Talavera, Nueva Ecija: A Basis for Development ...

IJAEMSJORNAL

This study aimed to profile the coffee shops in Talavera, Nueva Ecija, to develop a standardized checklist for aspiring entrepreneurs. The researchers surveyed 10 coffee shop owners in the municipality of Talavera. Through surveys, the researchers delved into the Owner's Demographic, Business details, Financial Requirements, and other requirements needed to consider starting up a coffee shop. Furthermore, through accurate analysis, the data obtained from the coffee shop owners are arranged to derive key insights. By analyzing this data, the study identifies best practices associated with start-up coffee shops’ profitability in Talavera. These findings were translated into a standardized checklist outlining essential procedures including the lists of equipment needed, financial requirements, and the Traditional and Social Media Marketing techniques. This standardized checklist served as a valuable tool for aspiring and existing coffee shop owners in Talavera, streamlining operations, ensuring consistency, and contributing to business success.

Exploring Deep Learning Models for Image Recognition: A Comparative Review

sipij

Image recognition, which comes under Artificial Intelligence (AI) is a critical aspect of computer vision, enabling computers or other computing devices to identify and categorize objects within images. Among numerous fields of life, food processing is an important area, in which image processing plays a vital role, both for producers and consumers. This study focuses on the binary classification of strawberries, where images are sorted into one of two categories. We Utilized a dataset of strawberry images for this study; we aim to determine the effectiveness of different models in identifying whether an image contains strawberries. This research has practical applications in fields such as agriculture and quality control. We compared various popular deep learning models, including MobileNetV2, Convolutional Neural Networks (CNN), and DenseNet121, for binary classification of strawberry images. The accuracy achieved by MobileNetV2 is 96.7%, CNN is 99.8%, and DenseNet121 is 93.6%. Through rigorous testing and analysis, our results demonstrate that CNN outperforms the other models in this task. In the future, the deep learning models can be evaluated on a richer and larger number of images (datasets) for better/improved results.

Germany Offshore Wind 010724 RE (1) 2 test.pptx

rebecca841358

Recently uploaded (20)

Software Engineering and Project Management - Introduction to Project Management

UNIT I INCEPTION OF INFORMATION DESIGN 20CDE09-ID

Lecture 6 - The effect of Corona effect in Power systems.pdf

Lecture 3 Biomass energy...............ppt

Unblocking The Main Thread - Solving ANRs and Frozen Frames

IWISS Catalog 2024

Unit 1 Information Storage and Retrieval

Online music portal management system project report.pdf

GUIA_LEGAL_CHAPTER_4_FOREIGN TRADE CUSTOMS.pdf

Best Practices of Clothing Businesses in Talavera, Nueva Ecija, A Foundation ...

Response & Safe AI at Summer School of AI at IIITH

21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY

Introduction to IP address concept - Computer Networking

GUIA_LEGAL_CHAPTER-9_COLOMBIAN ELECTRICITY (1).pdf

Evento anual Splunk .conf24 Highlights recap

Rohini @ℂall @Girls ꧁❤ 9873777170 ❤꧂VIP Yogita Mehra Top Model Safe

IS Code SP 23: Handbook on concrete mixes

Profiling of Cafe Business in Talavera, Nueva Ecija: A Basis for Development ...

Exploring Deep Learning Models for Image Recognition: A Comparative Review

Germany Offshore Wind 010724 RE (1) 2 test.pptx

Why reinvent the wheel at Criteo?

1. Cedrick MONTOUT 2016 June 8th A story about C# @ Criteo Why do we reinvent the wheel ?

2. 2 | Copyright © 2016 Criteo • Criteo • Global leader on retargeting • Scalability • One of many groups in the Criteo R&D department • WebScale • We write code to help scale up the real-time Criteo Platform • https://www.linkedin.com/in/kerdrek • https://github.com/kerdrek • @kerdrek Who am I ?

3. Client Side Load Balancing

5. 5 | Copyright © 2016 Criteo The situation then … 2/5 Footer: App A Pool App B Pool HA Proxy Service Pool Random DC Service Pool Traffic 185k QPS Input size ~5K bytes Output size ~3,5K bytes Network traffic ~12Gbits/s

6. 6 | Copyright © 2016 Criteo The situation then … 3/5 Footer: App A Pool App B Pool HA Proxy Service Pool Random DC Service Pool Traffic 185k QPS Input size ~5K bytes Output size ~3,5K bytes Network traffic ~12Gbits/s Compression 30% gain

7. 7 | Copyright © 2016 Criteo The situation then … 4/5 Footer: App A Pool App B Pool HA Proxy Service Pool Random DC Service Pool Traffic 185k QPS Input size ~5K bytes Output size ~3,5K bytes Network traffic ~12Gbits/s Compression 30% gain Bonding 2 ports Combine two physical data links into one logical link, by connecting 2 ports of the switch to 2 network interfaces of the HAProxy

8. 8 | Copyright © 2016 Criteo The situation then … 5/5 Footer: App A Pool App B Pool HA Proxy Service Pool Random DC Service Pool Traffic 185k QPS Input size ~5K bytes Output size ~3,5K bytes Network traffic ~12Gbits/s Compression 30% gain Bonding 2 ports Combine two physical data links into one logical link, by connecting 2 ports of the switch to 2 network interfaces of the HAProxy 4 pairs

9. 9 | Copyright © 2016 Criteo The new wheel : Client Side Load Balancing • Bypass HA Proxy • Implemented inside Twitter/Finagle • Reuse existing health Check • Re-implement monitoring App Pool A App Pool B Service Pool

10. DevHost

11. 11 | Copyright © 2016 Criteo The situation then … Footer: • Harder and harder to release • No available memory for new feature • Fragmented feature in production • Tightly coupled with the HTTP stack

16. Monitoring @ Task level

18. 18 | Copyright © 2016 Criteo The new Wheel : Asynchronous Token Framework • TPL was not a solution at that time • Asynchronous Completion Token • Delegate based • Execution is time boxed • Task underneath • Timing for every thing • Metrics available on the machine • Metrics available aggregated on Graphite

19. Apache Kafka Driver in C#

21. 21 | Copyright © 2016 Criteo The situation then … Footer: • Apache Kafka • Binary • Acknowledged message • Batched messages • Partitioning and replication • Consuming support • Syslog • Text based • Fire and Forget • Single messages • No built-in resiliency • No API for consuming

25. 25 | Copyright © 2016 Criteo Goldilocks Conundrum all over again We looked at several drivers First driver never used in production Second driver was impossible to unit test Third driver was not recently maintained

26. 26 | Copyright © 2016 Criteo The new Wheel: kafka-sharp • Yet another C# driver • Highly tuneable • Written with perf and scale in mind • Battle tested in production • Available here: https://github.com/criteo/kafka-sharp

27. 27 | Copyright © 2016 Criteo The wheel lists • Distributed load balancer between clients. • Lightweight hosting server. • Low level asynchronous execution framework. • Yet another C# driver for Apache Kafka.

29. Click to add text

Editor's Notes

What happens when traditional load balancing is too costly to scale ?
One app server will be partially connected to all or parts of the Service Pool. Because It was already re-invented by twitter in the Finagle we knew it was possible we re-implemented the monitoring on a distributed manner We re-use the health check already in place for HA Proxy benefit
What about migrating monolithic application to service oriented when you have spaghetti code ? The service pool was not always like that, in fact in the past all service were inside the app process.
Some Api were asking for the whole HttpContext when they only needed a few parameters transported in the query string or in the cookies
One Input converted from a byteArray One Output converted to a byteArray On Process that ingest an input and produce an output
You should monitor everything right ? But can you and should you measure and monitor your code down to the task level on your production machines.
Apache Kafka looks good and you want to use it but is there any good enough implementation in C# ?
We will not name names
We will not name names
We will not name names
We will not name names

Why reinvent the wheel at Criteo?

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (16)

Similar to Why reinvent the wheel at Criteo?

Similar to Why reinvent the wheel at Criteo? (20)

Recently uploaded

Recently uploaded (20)

Why reinvent the wheel at Criteo?

Editor's Notes