Unlocking the Power of IoT: A comprehensive approach to real-time insights
•
0 likes•59 views
In today's data-driven world, the Internet of Things (IoT) is revolutionizing industries and unlocking new possibilities. Join Data Reply, Confluent, and Imply as we unveil a comprehensive solution for IoT that harnesses the power of real-time insights.
Report
Share
More Related Content
Similar to Unlocking the Power of IoT: A comprehensive approach to real-time insights
Similar to Unlocking the Power of IoT: A comprehensive approach to real-time insights (20)
More from confluent
More from confluent (20)
Recently uploaded
Recently uploaded (20)
Unlocking the Power of IoT: A comprehensive approach to real-time insights
- 1. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)? Starting soon… STARTING SOOOOON.. Starting sooooon ..
- 2. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)? Starting soon… STARTING SOOOOON..
- 3. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. IoT meets Confluent meets Data Platform MQTT Broker OPC UA gRPC Proxy
- 4. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc. IoT meets Confluent meets Data Platform MQTT Broker OPC UA gRPC Proxy
- 5. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)?
- 6. Goal Partners Tech Talks are webinars where subject matter experts from a Partner talk about a specific use case or project. The goal of Tech Talks is to provide best practices and applications insights, along with inspiration, and help you stay up to date about innovations in confluent ecosystem.
- 7. @yourtwitterhandle | developer.confluent.io Starting soon… STARTING SOOOOON.. Starting sooooon ..
- 8. @yourtwitterhandle | developer.confluent.io Starting soon… STARTING SOOOOON.. Starting sooooon ..
- 9. @yourtwitterhandle | developer.confluent.io Starting soon… STARTING SOOOOON.. Starting sooooon .. Let’s win together!
- 10. Confluent Perspective Real Time Analytics 10
- 11. 11 Business challenges Technical challenges Increasingly complex application environments and mounting pressures to track and respond to every indicator and issue. Data latency and lack of end-to-end, scalable observability for monitoring behavior, performance, and health of complex systems, applications, and infrastructure. Technology failures and security risks that result in disruption to customer-facing services and costly losses for the business. Higher operational costs due to more troubleshooting time, bottlenecks, and suboptimal performance requiring additional resources/infrastructure. INDUSTRY: ALL
- 12. 12 Why Confluent Stream data everywhere, on premises and in every major public cloud. Connect operational data like logs, metrics, and traces from across your entire business including on-prem, cloud, and hybrid environments. Process data streams to feed real-time analytics applications that query and visualize critical metrics at scale including latencies, error rates, overall service health statuses, etc. Govern data to ensure quality, security, and compliance while enabling teams to discover and leverage existing data products. Business impact Enable early detection of system-wide issues to prevent incidents and downtime. Deliver proactive, faster responses to open incidents for quicker resolution. Gain the ability to deeply analyze all systems and make more informed decisions. INDUSTRY: ALL
- 13. Building a custom observability platform for your business delivers long-term benefits Scalability & Latency Ingest, query, and visualize mission-critical log data in real time, at scale and under load. Privacy & Cost Controls Maintain more control over data sensitive to your business and optimize for cost savings. Customization & Flexibility Tailor a solution and all features to the specific needs of your organization.
- 14. Confluent and Imply deliver a simple solution for building a real-time observability platform Confluent Cloud Fully managed data streaming platform Imply Polaris Fully managed Apache Druid® service Connect all of your data in real time with a cloud-native and complete data streaming platform available everywhere you need it. Get all the speed and performance of Druid without having to manage the database or configure infrastructure.
- 15. @yourtwitterhandle | developer.confluent.io What are the best practices to debug client applications (producers/consumers in general but also Kafka Streams applications)? Starting soon… STARTING SOOOOON..
- 16. ©2023 Imply ©2023 imply From Streaming Data to Real-Time Insights: Building with Apache Druid Hellmar Becker, Senior Sales Engineer 16
- 17. ©2023 Imply Agenda 17 ● Introduction to Apache Druid ● K2D Architecture - Kafka to Druid ● K2D Use Cases and Case Studies ● About Imply / Demo
- 18. ©2023 Imply Apache Druid is a real-time analytics database Sub-second queries at any scale Interactive analytics on TB-PBs of data High concurrency at the lowest cost 100s to 1000s QPS via a highly efficient engine Real-time and historical insights True stream ingestion for Kafka and Kinesis Plus, non-stop reliability with automated fault tolerance and continuous backup 1 2 3 For analytics applications that require:
- 19. How It Works: Streaming Analytics with Druid
- 20. ©2023 Imply Files App data Data sources Microservices Database Events Streaming Infrastructure Databases Data Lake Stream ETL Stream Processors Messaging Realtime Analytics Events Analytics Infrastructure ML/AI Dashboards & reports Interactive Data Apps BI tools Machine Driven Queries Sensor data 20 K2D Architecture - Kafka to Druid
- 21. ©2023 Imply Druid is built to analyze Kafka data 21 No connectors needed. Unlike other analytics databases, Druid was purpose-built to ingest data from Kafka, so there’s no need to “connect” the two. Built-in scalability. Druid easily scales real-time and batch ingestion up to millions of events per second / tens of TBs per minute. Event-based ingestion. With Druid, events don’t have to be persisted to storage first—they are instantly available to applications with exactly-once semantics. Fast analytical queries. Druid is built for fast analytical queries on real-time and historical data, enabling contextual analysis and real-time insights.
- 22. ©2023 Imply 22 Real-time use cases for Kafka-to-Druid ALERTING MONITORING DASHBOARDS EXPLORATION DECISIONING State or stateless triggered actions Continuous tracking of KPIs User-facing operational visibility Ad-hoc rapid data exploration On high throughput Kafka streams API-triggered automated workflows if X, then Y
- 24. Chargers IoT Core Kinesis Stream Metrics Consumers Kafka Charger sensor metric produced Medium charger sensor metric Transient charger sensor metric Telemetry filtering Cosmos Service engineering Imply - Druid 2 days 30 days
- 25. ©2022, Imply Confidential. Do not redistribute. © Imply Previous state 25 ● SQL Server was slow & had stability issues, as the portal is forward facing to the end users, this caused the UX to be poor when querying data ● Reporting (and selling) this data, sometimes resulted in the query never completing ● Old UX was clunky and ‘slow’ ● Costs of maintaining SQL Server DW was high ● SQL Server had scalability issues ● Hitting limits of 2M devices (controllers + sensors) ● Plan is to scale to 10M+ devices so SQL already cannot deliver Proprietary Photovoltaic systems Controller
- 26. ©2022, Imply Confidential. Do not redistribute. © Imply New state 26 ● Real Time Visibility & Analysis — Solar PV & Battery ● Removed intermediate steps between source data and the analytics interface by connecting Kafka to Imply ● Ability to scale virtually infinitely ● Visualize Yield & Consumption — live & historical data to ensure yield ● Compare Yield — detect and resolve deviations to save time and reduce loss Photovoltaic systems Controller
- 27. ©2022, Imply Swisscom, the biggest Telecom company in Switzerland (Mobile, Internet, TV). Using Imply since 2016 DEPLOYMENT TYPE & VOLUME: Imply Hybrid & On prem | 235 TB data INDUSTRY / COMPANY SIZE: Telco company / 15,000+ employees USE CASES: Network Monitoring/Troubleshooting, Self-Service analytics for Business teams 2 Use Cases Network Monitoring/Troubleshooting, Self-Service analytics for Business teams 20x Number of users with access to KPIs 20min MTRS Reduce the Mean Time to Restore Service from hours to minutes IMPLY: CUSTOMER STORIES (2022)
- 28. ©2023 Imply Committer Expertise 24/7 Support | 100% of the Original Creators Imply Pivot Custom UI Tableau, etc Effortless Operations Management Tools | Performance Monitoring Cloud Deployment Fully-Managed (Polaris) | Hybrid-Managed Druid Database Commercial Distribution | Enhanced Security Committer-driven support. ✔ Flexible deployment options. ✔ Native Confluent Cloud integration. ✔ Accelerated time-to-value. ✔ Get the scalability, elasticity, and resilience of Druid, plus: Imply completes the Druid experience
- 29. Imply Customers Technology Platform Advertising Visit imply.io/success-stories Comms Gaming Financial
- 30. Confluent and Imply deliver a simple solution for building a real-time observability platform Confluent Cloud Fully managed data streaming platform Imply Polaris Fully managed Apache Druid® service Connect all of your data in real time with a cloud-native and complete data streaming platform available everywhere you need it. Get all the speed and performance of Druid without having to manage the database or configure infrastructure.
- 31. Access Confluent data streams from directly within Imply Polaris Screenshot of Confluent data streams access embedded within Imply Polaris See Imply’s new data streaming integration during today’s demo
- 32. ©2023 Imply ©2023, imply Imply Polaris Confluent Cloud Start your free trials today New signups receive $400 to use during their first 30 days. confluent.io/get-started imply.io/get-started New signups receive $500 to use during their first 30 days.
- 35. ©2022, Imply Real-Time Analytics Applications Real-time Analytics Database + New market requires a new kind of database 35 Analytics Data Warehouses Applications Transactional Databases Read-optimized TB-PBs of Data High Cardinality Sub-Sec Response High Concurrency Real-time Data BI Reporting Monthly Reporting Static Dashboards ACID Compliance Small Data Write-optimized BI Reporting Monthly Reporting Static Dashboards ACID Compliance Small Data Write-optimized ✓ ✓ ✓ ✓ ✓ ✓