SlideShare a Scribd company logo
REAL-TIME
DECISION ENGINES
REACT TO YOUR BUSINESS WHEN IT SPEAKS TO YOU
ALEX PIERMATTEO
SERGIO SPINATELLI
2
Data Reply
Supports you in becoming Data-Driven with Cloud & Open-Source
Data & ML Engineering3
Data Platforms & Cloud
Solutions1
Event-Driven & Streaming
Applications2
Services
§ Enabling Customers to become Data-Driven by providing state of
the art Architectures and Applications focusing on Distributed &
Cloud technologies
§ Architecture and Development of Big Data Applications, Services
and Infrastructure, such as Data Platforms, Real-Time Decision
Engines and Machine Learning Applications
§ Providing highly automated, scalable and customised solutions
matching the client's needs
§ Build and Automate Machine Learning & Analytics Applications
for constantly delivering value
3
Event-Driven & Streaming Applications
We can support you in building efficient and scalable real-time
applications & microservices, based on modern streaming and
event-driven frameworks
§ Real-Time Decision Engines
§ Streaming Data Processing & Analytics
§ Real-Time Visualizations
§ Event-Sourcing for Business Applications
§ Chat-Bots & Cloud Native Applications
§ Technologies:
Apache Kafka, Confluent Platform, Kafka Streams, Apache Flink,
ksqlDB, AWS Kinesis, Azure EventHub, Google PubSub,
Serverless Functions, Redis, Kubernetes
Focus Area
Services
Sergio Spinatelli
Business Unit Manager
Architect
Alex Piermatteo
Business Unit Manager
Architect
REAL-TIME
DECISION ENGINES
Event-Driven Systems
The foundation for a new mindset
Based on Events: “significant changes of state”
§ A customer buys a product, a car’s tire pressure changes, a user clicks on a website button,
a passenger checks in
Events can be streamed from the systems recording or producing them (with Kafka!) to be made
available in real-time
Once events are available, they can drive new Business Value, because being Event-Driven means
being:
§ Actionable:
§ Events can trigger well defined actions (or chain of actions) like upselling, cross-
selling, notifications..
§ Relevant and Meaningful:
§ Reacting to events allows the action to be relevant in the context of the user and more
meaningful for them
§ Real-Time:
§ Actions performed in real-time instead of days later can improve customer satisfaction,
reduce risk and reduce costs
Real-Time Decision Engines
A smart way to use Events
Out in the wild: Organisations make decisions around stale and often inaccurate information
Real-Time Decision Engines turn real-time data into real-time action:
1. Provision of information in context and integrated with the decision work-flow in real-time
2. Ability of an organization to make then operational and business choices based on the most current data
3. Leveraging Situational Awareness to perform the most informed, accurate and fast decision on what action to take
Fast reaction times to real-world business events bring
new use cases:
§ Improving customer experience
§ Recognizing risk quickly
§ Acting on information as soon as it is available
§ Avoid mistakes due to outdated data
Real-Time Decision Engines
Key steps to building one
It's an iterative process based on a Microservice approach where each additional developed service provides a new set of features:
1. Start integrating key real-time data sources to create your Context information
• E.g.: customer information, permissions, last actions of a customer, interactions made with the customer, ..
2. Build services able to create Queryable "materialized views" of this context information
3. Integrate additional real-time Events which will drive your Actions and trigger your Business Logic
4. Deliver a use case able to get your Events as input and based on the most up to date contextual information decide in real-time what
Actions to perform
5. The Real-Time Decision Engine should be highly configurable, reusable and easy to extend
6. Add more contextual information (microservices + additional sources) as you need it for new use cases or to improve the precision of
your current RTDE ecosystem
Real-Time Decision Engines
Pattern 1 – Basic Flow
§ Any part of the architecture can be developed ad hoc based on
custom requirements
§ Due to its Event-Driven nature, the architecture can scale from
low to high volumes of events
§ The System matches every Event with the Real-Time Context
information to understand what to do
§ The Event-Based nature of the system makes the definitions of
the Decision Rules easy to make and understand
§ Each service in the Context layer can be reused for different use
cases
§ The microservice and decoupled approach makes the
architecture easy to extend with new features and tools
Real-Time Decision Engines
Pattern 2 – Batch Flow integration
§ The previous architecture can be adapted and used also for Use
Cases that are not exactly Event-Driven but are driven by some
particular Segmentation on top of the entire Context Dataset
§ In this case the entire Context can be included as a real-time
Golden Record in an OLAP/Data Grid system where the
filtering can run on top to define new Actions
§ The Segmentation is also distributed and can scale accordingly
with the Context Dataset size
Real-Time Decision Engines
Pattern 3 – Feedback loop to fuel further Decisions
§ The final "evolution" of an RTDE is the inclusion
in the Engine of the results coming back from the
generated Actions
§ This information can be processed to extend the
Real-Time Context of the Engine and used again
from the RTDE Layer to make even more precise
decisions
§ The results of the decisions can be also included
in Real-Time Visualization tools
RTDE INDUSTRY
EXAMPLES
RTDEs in the Telecommunications Industry
Common Applications and Use Cases
Example Use CasesTypical Data Sources
§ Network & infrastructure sensor data
§ Subscription data
§ Customer data
§ Consumption data
§ Sales data
§ Sales channel information and privacy constraints/permissions
§ Engagement and interaction data
§ Location data
§ Real-Time marketing campaigns
§ Cross-channel
§ Up-selling
§ Cross-selling
§ Location based
§ Network & infrastructure anomaly detection and alerting
§ Real-Time Customer 360
§ Real-Time Customer satisfaction feedback collection
§ Real-Time credit score check & subscription risk mitigation
13
Real-Time Marketing Campaigning and Next Best Offer
Telecommunication – Event-Driven & Streaming Applications
In order to generate personalized extension Offers to drive
profits, reduce costs and perform more accurate marketing
when the End Users hit their data plan limits, a migration from a
previous ETL-based marketing logic to a Real-Time Decision
Engine able to quickly react to business events was performed
Approach and achieved results
§ Development of a Real-Time Decision Engine able to generate a
personalized offer in seconds based on real-time information of
the End Users
§ Migration of all the previous Throttle Events Campaigns from
batch to a near-real-time fashion: The End Users of the Telco
Customer receive a new offer in seconds after they reach their
data volume threshold
§ Improved accuracy due to the usage of real-time information:
previously the Customer based the offers for its End User on 3-
day old data
Key Facts
§ Next Best offer within 5s of trigger event
§ 20k SMS per day to customers for main Brand
§ 15 Campaigns running after 6 month, 30 after 9 months
§ Errors percentage with the Marketing offers went down from
10% to 0.1% after the migration based on customers real-time
data
§ Cost reduction due to the dismission of some old licenses not
required anymore
Use case visualization
Use Case Overview
Business Goal
RTDEs in the Banking & Insurance Industries
Common Applications and Use Cases
Example Use CasesTypical Data Sources
Banking
§ Transaction data
§ Account data
§ Customer data
§ Loan data and credit score
§ Stock exchange data
§ Mobile phone location (e.g. mobile payments)
Insurance
§ Policy data
§ Customer data
§ Account data
§ Location and sensor data
Banking
§ Real-Time fraud detection and Contextual payment verification
§ Faster Account verification
§ Real-Time security breach detection
§ Shorter Loan decision times
§ Customized trading suggestions
§ Real-Time mobile app operations
§ ChatBot for customers to perform routine actions
Insurance
§ Shorter Policy approval decision times
§ Real-Time upselling and cross-selling
§ Real-Time checks for coverage and notifications
§ Real-Time security breach detection
§ ChatBots for customer support and performing actions on policies
15
Microservice-based Banking Middleware
Banking – Event-Driven & Streaming Applications
Business Goal
In order to process transactions and customer data in a scalable
and resilient way and to replace a legacy system integrating a
diverse ecosystem of internal and external banking systems, an
event-driven middleware was developed
Approach and achieved results
§ Integration and communication with systems of external partner
companies was achieved in a secure manner
§ Integration of internal core banking systems was achieved, and
the end-to-end integration allowed for fast data and transaction
flows
§ End-to-End solution using a microservice and event-driven
approach for scalability and resiliency
Key Facts
§ Greenfield project brought to a production-ready state in 4 months
§ Integrated and harmonized services and systems with substantial
differences in communication interfaces and approaches
§ Orchestrated 40 different workflows, including monetary
transactions
Use case visualization
Use case overview
RTDEs in the Media Industry
Common Applications and Use Cases
Example Use CasesTypical Data Sources
§ Content consumption data: what, when, who, ..
§ Advertisement and tracking data
§ EPG data for live TV
§ Subscription data
§ User demographic data
§ Content metadata (movie title, TV-program name, radio program
type..)
§ Clickstream data (online)
§ Real-Time:
§ Marketing campaigns
§ Content recommendations and cross-selling
§ Advertisement selection and placement
§ Churn analysis and prevention
§ Payment fraud detection
§ Payment method suggestion
§ Customer 360
17
Video Stream Event Analysis & Processing
Media – Event-Driven & Streaming Applications
Business Goal
To improve viewer experience and enforce restrictions in real-
time, a real-time decisioning engine based on video stream
events was needed, able to perform actions and take decisions
according to specific business logic
Approach and achieved results
§ End-to-End solution with frontend device data generation, ingestion,
stream processing applications and data analysis, enabling:
§ Concurrent Streams Block: real-time block of account sharing
§ Trending Content: real-time computation of what content (Video-
on-Demand or Live TV) is currently popular, to be automatically
highlighted to the users in the frontends
§ Analysis of video view sessions and event tracking: real-time
detection, tracking and and analysis of user viewing sessions,
integrated into the tracking and clickstream analytics
§ Resume Position: a more precise and up to date resume
position is computed and can be used to continue playback at a
later point in time
Key Facts
§ Go-Live after 3 months
§ Enforcements of user limitations in seconds
§ Accurate trending content near-real time on the main page for
better content discovery (incl. live events)
§ Improved video view session tracking
§ More accurate resume position after stop, e.g. on turning off the tv
Use case visualization
Use Case Overview
Q&A
SEE YOU AT THE CSE ROUNDTABLE!

More Related Content

Data reply sneak peek: real time decision engines

  • 1. REAL-TIME DECISION ENGINES REACT TO YOUR BUSINESS WHEN IT SPEAKS TO YOU ALEX PIERMATTEO SERGIO SPINATELLI
  • 2. 2 Data Reply Supports you in becoming Data-Driven with Cloud & Open-Source Data & ML Engineering3 Data Platforms & Cloud Solutions1 Event-Driven & Streaming Applications2 Services § Enabling Customers to become Data-Driven by providing state of the art Architectures and Applications focusing on Distributed & Cloud technologies § Architecture and Development of Big Data Applications, Services and Infrastructure, such as Data Platforms, Real-Time Decision Engines and Machine Learning Applications § Providing highly automated, scalable and customised solutions matching the client's needs § Build and Automate Machine Learning & Analytics Applications for constantly delivering value
  • 3. 3 Event-Driven & Streaming Applications We can support you in building efficient and scalable real-time applications & microservices, based on modern streaming and event-driven frameworks § Real-Time Decision Engines § Streaming Data Processing & Analytics § Real-Time Visualizations § Event-Sourcing for Business Applications § Chat-Bots & Cloud Native Applications § Technologies: Apache Kafka, Confluent Platform, Kafka Streams, Apache Flink, ksqlDB, AWS Kinesis, Azure EventHub, Google PubSub, Serverless Functions, Redis, Kubernetes Focus Area Services Sergio Spinatelli Business Unit Manager Architect Alex Piermatteo Business Unit Manager Architect
  • 5. Event-Driven Systems The foundation for a new mindset Based on Events: “significant changes of state” § A customer buys a product, a car’s tire pressure changes, a user clicks on a website button, a passenger checks in Events can be streamed from the systems recording or producing them (with Kafka!) to be made available in real-time Once events are available, they can drive new Business Value, because being Event-Driven means being: § Actionable: § Events can trigger well defined actions (or chain of actions) like upselling, cross- selling, notifications.. § Relevant and Meaningful: § Reacting to events allows the action to be relevant in the context of the user and more meaningful for them § Real-Time: § Actions performed in real-time instead of days later can improve customer satisfaction, reduce risk and reduce costs
  • 6. Real-Time Decision Engines A smart way to use Events Out in the wild: Organisations make decisions around stale and often inaccurate information Real-Time Decision Engines turn real-time data into real-time action: 1. Provision of information in context and integrated with the decision work-flow in real-time 2. Ability of an organization to make then operational and business choices based on the most current data 3. Leveraging Situational Awareness to perform the most informed, accurate and fast decision on what action to take Fast reaction times to real-world business events bring new use cases: § Improving customer experience § Recognizing risk quickly § Acting on information as soon as it is available § Avoid mistakes due to outdated data
  • 7. Real-Time Decision Engines Key steps to building one It's an iterative process based on a Microservice approach where each additional developed service provides a new set of features: 1. Start integrating key real-time data sources to create your Context information • E.g.: customer information, permissions, last actions of a customer, interactions made with the customer, .. 2. Build services able to create Queryable "materialized views" of this context information 3. Integrate additional real-time Events which will drive your Actions and trigger your Business Logic 4. Deliver a use case able to get your Events as input and based on the most up to date contextual information decide in real-time what Actions to perform 5. The Real-Time Decision Engine should be highly configurable, reusable and easy to extend 6. Add more contextual information (microservices + additional sources) as you need it for new use cases or to improve the precision of your current RTDE ecosystem
  • 8. Real-Time Decision Engines Pattern 1 – Basic Flow § Any part of the architecture can be developed ad hoc based on custom requirements § Due to its Event-Driven nature, the architecture can scale from low to high volumes of events § The System matches every Event with the Real-Time Context information to understand what to do § The Event-Based nature of the system makes the definitions of the Decision Rules easy to make and understand § Each service in the Context layer can be reused for different use cases § The microservice and decoupled approach makes the architecture easy to extend with new features and tools
  • 9. Real-Time Decision Engines Pattern 2 – Batch Flow integration § The previous architecture can be adapted and used also for Use Cases that are not exactly Event-Driven but are driven by some particular Segmentation on top of the entire Context Dataset § In this case the entire Context can be included as a real-time Golden Record in an OLAP/Data Grid system where the filtering can run on top to define new Actions § The Segmentation is also distributed and can scale accordingly with the Context Dataset size
  • 10. Real-Time Decision Engines Pattern 3 – Feedback loop to fuel further Decisions § The final "evolution" of an RTDE is the inclusion in the Engine of the results coming back from the generated Actions § This information can be processed to extend the Real-Time Context of the Engine and used again from the RTDE Layer to make even more precise decisions § The results of the decisions can be also included in Real-Time Visualization tools
  • 12. RTDEs in the Telecommunications Industry Common Applications and Use Cases Example Use CasesTypical Data Sources § Network & infrastructure sensor data § Subscription data § Customer data § Consumption data § Sales data § Sales channel information and privacy constraints/permissions § Engagement and interaction data § Location data § Real-Time marketing campaigns § Cross-channel § Up-selling § Cross-selling § Location based § Network & infrastructure anomaly detection and alerting § Real-Time Customer 360 § Real-Time Customer satisfaction feedback collection § Real-Time credit score check & subscription risk mitigation
  • 13. 13 Real-Time Marketing Campaigning and Next Best Offer Telecommunication – Event-Driven & Streaming Applications In order to generate personalized extension Offers to drive profits, reduce costs and perform more accurate marketing when the End Users hit their data plan limits, a migration from a previous ETL-based marketing logic to a Real-Time Decision Engine able to quickly react to business events was performed Approach and achieved results § Development of a Real-Time Decision Engine able to generate a personalized offer in seconds based on real-time information of the End Users § Migration of all the previous Throttle Events Campaigns from batch to a near-real-time fashion: The End Users of the Telco Customer receive a new offer in seconds after they reach their data volume threshold § Improved accuracy due to the usage of real-time information: previously the Customer based the offers for its End User on 3- day old data Key Facts § Next Best offer within 5s of trigger event § 20k SMS per day to customers for main Brand § 15 Campaigns running after 6 month, 30 after 9 months § Errors percentage with the Marketing offers went down from 10% to 0.1% after the migration based on customers real-time data § Cost reduction due to the dismission of some old licenses not required anymore Use case visualization Use Case Overview Business Goal
  • 14. RTDEs in the Banking & Insurance Industries Common Applications and Use Cases Example Use CasesTypical Data Sources Banking § Transaction data § Account data § Customer data § Loan data and credit score § Stock exchange data § Mobile phone location (e.g. mobile payments) Insurance § Policy data § Customer data § Account data § Location and sensor data Banking § Real-Time fraud detection and Contextual payment verification § Faster Account verification § Real-Time security breach detection § Shorter Loan decision times § Customized trading suggestions § Real-Time mobile app operations § ChatBot for customers to perform routine actions Insurance § Shorter Policy approval decision times § Real-Time upselling and cross-selling § Real-Time checks for coverage and notifications § Real-Time security breach detection § ChatBots for customer support and performing actions on policies
  • 15. 15 Microservice-based Banking Middleware Banking – Event-Driven & Streaming Applications Business Goal In order to process transactions and customer data in a scalable and resilient way and to replace a legacy system integrating a diverse ecosystem of internal and external banking systems, an event-driven middleware was developed Approach and achieved results § Integration and communication with systems of external partner companies was achieved in a secure manner § Integration of internal core banking systems was achieved, and the end-to-end integration allowed for fast data and transaction flows § End-to-End solution using a microservice and event-driven approach for scalability and resiliency Key Facts § Greenfield project brought to a production-ready state in 4 months § Integrated and harmonized services and systems with substantial differences in communication interfaces and approaches § Orchestrated 40 different workflows, including monetary transactions Use case visualization Use case overview
  • 16. RTDEs in the Media Industry Common Applications and Use Cases Example Use CasesTypical Data Sources § Content consumption data: what, when, who, .. § Advertisement and tracking data § EPG data for live TV § Subscription data § User demographic data § Content metadata (movie title, TV-program name, radio program type..) § Clickstream data (online) § Real-Time: § Marketing campaigns § Content recommendations and cross-selling § Advertisement selection and placement § Churn analysis and prevention § Payment fraud detection § Payment method suggestion § Customer 360
  • 17. 17 Video Stream Event Analysis & Processing Media – Event-Driven & Streaming Applications Business Goal To improve viewer experience and enforce restrictions in real- time, a real-time decisioning engine based on video stream events was needed, able to perform actions and take decisions according to specific business logic Approach and achieved results § End-to-End solution with frontend device data generation, ingestion, stream processing applications and data analysis, enabling: § Concurrent Streams Block: real-time block of account sharing § Trending Content: real-time computation of what content (Video- on-Demand or Live TV) is currently popular, to be automatically highlighted to the users in the frontends § Analysis of video view sessions and event tracking: real-time detection, tracking and and analysis of user viewing sessions, integrated into the tracking and clickstream analytics § Resume Position: a more precise and up to date resume position is computed and can be used to continue playback at a later point in time Key Facts § Go-Live after 3 months § Enforcements of user limitations in seconds § Accurate trending content near-real time on the main page for better content discovery (incl. live events) § Improved video view session tracking § More accurate resume position after stop, e.g. on turning off the tv Use case visualization Use Case Overview
  • 18. Q&A
  • 19. SEE YOU AT THE CSE ROUNDTABLE!