SlideShare a Scribd company logo
Andrei Lopatenko, VP Eng Zillow
Driving Customer Experience and
Business Revenues Through Search
Engines, Recommendations and
Conversational Interfaces
©CX VIRTUAL SUMMIT
What will you learn from this talk
• How to drive continuous circle of improvements of your
search/recommendation/personalization engines or
conversational system leading to improvement of customer
experience and business revenues
• What investments in this area deliver business value and visible
as investments improving business and customer experience
©CX VIRTUAL SUMMIT
Search, Recommendation, Personalization
• In this talk, we are talking about search, recommendation,
personalization engines driving your business, key gateways
letting your customers to find and discover your business items
to buy, book, rent, etc them
• We do not talk about small secondary search engines (search in
forums of your web site, etc)
©CX VIRTUAL SUMMIT
Engines
• We put Search, Recommendation, Personalization engine into one
category - SRP engine
• Despite very different product functions, under the hood, they
implement similar functionality, letting user to express their interests
(‘query’) explicitly or implicitly, and giving user a set of results relevant
to the user’s interest. But ‘query’ can be a text (search) query, user id
(personalization) , location, item id etc (recommendation) ,
• Examples of SRP engines: Google.com, Yelp (localsearch and local
recommendation), Apple Appstore, Amazon or Walmart online
(search, item recommendation, personalization), Zillow.com,
• Despite they are different, they are a lot of similarities and
recommendations of this talk are valid for all of them

Recommended for you

John Glinski - The B2B Analytics Engine: How Vanguard is Measuring Marketing...
John Glinski - 	The B2B Analytics Engine: How Vanguard is Measuring Marketing...John Glinski - 	The B2B Analytics Engine: How Vanguard is Measuring Marketing...
John Glinski - The B2B Analytics Engine: How Vanguard is Measuring Marketing...

This document presents a visit score recipe developed by John Glinski at Vanguard to measure the success and ROI of a company's B2B web experience. It describes developing an engagement score by tracking key metrics like logins, tool usage, article views, and lead forms to quantify user engagement. The steps include determining key actions and success goals with stakeholders, mapping out the logic, implementing tracking in Adobe, analyzing results, and sharing recommendations. Examples of implementation for different departments like marketing, sales, IT are provided. The process is meant to test solutions, improve based on data, and continuously measure progress.

The Power of a Proper North Star Metric
The Power of a Proper North Star MetricThe Power of a Proper North Star Metric
The Power of a Proper North Star Metric

The document discusses finding and leveraging a North Star Metric, which is a single metric or set of metrics that a company uses to focus on sustainable growth. It recommends that the North Star Metric be easy to understand, reflect customer value, lead to correlated growth, and be measurable. It also discusses frameworks for identifying potential North Star Metrics, such as looking at customer and business experiences, as well as reconciling brand and customer experiences to identify a metric. Finally, it notes that the North Star Metric should start with retention and the back of the funnel in mind.

northstarmetricsnorthstarmetricnorthstar
Effective Lead Generation through Lead Lifecycle Management
Effective Lead Generation through Lead Lifecycle ManagementEffective Lead Generation through Lead Lifecycle Management
Effective Lead Generation through Lead Lifecycle Management

This document outlines steps for creating an effective lead management strategy. It begins by emphasizing the importance of aligning marketing and sales activities and goals. It then discusses understanding the customer's buying cycle, automating lead scoring and routing processes, nurturing leads throughout the buying cycle, and empowering sales. The overall message is that a comprehensive lead management strategy is needed to engage customers at all stages and plug leaks in the sales pipeline.

lead generation
©CX VIRTUAL SUMMIT
SRP Engines
• Typically, recommendation and personalization engines are
smaller siblings of search engines, since they get 10X less
customers, and produce 10X less revenue
• But nevertheless, they are very impactful revenue generators
and affect customer experience in a significant way after search
engines
©CX VIRTUAL SUMMIT
Conversational Interfaces
• Conversational interfaces are voice based or text based
interfaces letting user to interactivity communicate with business
services (voice search-> buy, ask questions -> get answers,
voice based customer support)
• Some of them utilize search engines to answer users’ questions.
• They have a lot tech shared with SRP, but a lot differences in
science and technologies
• Nevertheless, key decisions how to develop and run them are
shared with SRP, so most of focus of this talk is about SRP (due
to time limit), but recommendations are applicable to
Conversational Interfaces
©CX VIRTUAL SUMMIT
Customer experience
• Frequently, we separate customer experience (CX) metrics
(typically measured through various behavior metrics) from
business metrics (such as revenue per sessions, revenue per
month, GMV).
• CX and business are related but not directly.
• Good customer experience leads to growth (new users stick with
your business, existing users do not move to competitors)
• Customer experience metrics must be designed for your
business, few CX metrics are applicable to every SRP engine.
CX metrics reflect how users interact with your business site/app
©CX VIRTUAL SUMMIT
Revenue/Business
• SRP engines drive your business since customers buy, book,
rent through SRP engines
• SRP engines produce cash flow
• And they produce expenses (operational support, development
support, data acquisition, software costs)

Recommended for you

Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...
Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...
Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...

The document discusses the StoryVesting framework for achieving business transformation. It describes how digital disruption has shortened the time between strategic inflection points, making transformation more crucial. The framework guides transformation with data insights and encourages "vested" employees with growth mindsets. It also considers the customer experience framework and how aligning the business and customer experiences can lead to benefits like strong brand association and loyalty. The document argues that without a guiding framework, transformation attempts are like guessing and less likely to succeed.

business transformationtransformation framework
What Factors Determine Contact Center Channels ?
What Factors Determine Contact Center Channels ?What Factors Determine Contact Center Channels ?
What Factors Determine Contact Center Channels ?

A contact center enables agents to communicate with consumers via various communication channels. Businesses should carefully decide the bundle of the channels for the success of operations. Here are the factors to consider to choose the channels for your contact center.

call centercontact centercloud
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěchPokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch

This document discusses market segmentation and how it can help companies better understand their customers. It defines segmentation as dividing a market into subgroups that have similar needs. Effective segmentation requires groups that can be measured, accessed, are substantial in size/profitability, and respond differently to marketing. The document outlines how segmentation allows companies to communicate more directly with customers, increase relevance, and improve conversion rates. It provides examples of common types of segmentation and how segmentation can be used for testing, personalized experiences, and targeted email marketing. Overall, the document promotes the use of segmentation to better understand customers and increase marketing effectiveness and return on investment.

©CX VIRTUAL SUMMIT
Typical architecture of SRP Engine
©CX VIRTUAL SUMMIT
Typical architecture
• There are a lot of variations in that diagram: but overall it’s the
same flow: search assistance -> query understanding ->
retrieval -> ranking -> presentation generator
• Where different technologies can be used at every layer: either
traditional space feature IR or dense representation retrieval or
similarity search etc but still high level architecture is very
similar across systems
©CX VIRTUAL SUMMIT
Typical Entry points of Customer Experience
• Autocomplete/Autosuggest
• Filters/Facets
• Refinements/Query suggestions
• Search box
• Search Results page
• Recommendations on search result pages
• Personalizated lists
©CX VIRTUAL SUMMIT
Customer experience Metrics
• Many way to measure satisfaction of customers
• From offline - questionaries, user studies to live traffic/behavior
based
• Click through rates and shallow engagement metrics
• Conversion rates and deep engagement metrics
• Subscription/unsubscription rates

Recommended for you

Adobe Experience Platform Bootcamp
Adobe Experience Platform BootcampAdobe Experience Platform Bootcamp
Adobe Experience Platform Bootcamp

A day wise presentation of my bootcamp on AEP taken in September 2021. For more information, and to read day wise materials, please head over to my website: https://thelearningproject.in. You can also reach out to me on ritesh@thelearningproject.in

Humanizing Big Data: The Key to Actionable Customer Journey Analytics
Humanizing Big Data: The Key to Actionable Customer Journey AnalyticsHumanizing Big Data: The Key to Actionable Customer Journey Analytics
Humanizing Big Data: The Key to Actionable Customer Journey Analytics

The ability to gather and act on Big Data has changed our world. For companies, this influx of information is an opportunity to understand consumers on an unprecedented level. But there's a big difference between collecting disparate data points and connecting with consumers through journey analytics.

analyticsbig datacustomer journey
Revenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic BlueprintRevenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic Blueprint

The true value in your KPIs is understanding how they complete the bigger picture of the customer journeys that drive the most impact for your business.

b2b marketingb2b advertising
©CX VIRTUAL SUMMIT
Customer Experience Metrics
• Longitudinal studies: when customer come back, behavior shifts
over time
• Longitudinal studies - How frequently customer use SRP engine
/ the number of sessions per week/month/year, why they stay or
leave
• Abandonment rates, queries with no results, session with no
clicks
• Many other metrics, frequently needs to be designed for your
business and your customers
• Many of metrics should be studied by subcategories of services,
query types, customer types
©CX VIRTUAL SUMMIT
Business Metrics
• Revenue based: conversion per query and conversion per
session
• Total Revenue per week/month/year , Gross Merchandise Value
in C2C case
• Conversion per customer
•
©CX VIRTUAL SUMMIT
Business Metrics
• Other business targets
• Revenue/number of engaged customer per certain business
segments
• Cross category basket size
• Depends on business goals
• Should be measured and used to turn models, build systems to
develop business in this way
©CX VIRTUAL SUMMIT
Operational Cost Metrics
• Direct operational costs to support search engine running: cost
of datacenter or cloud costs, cost of data acquisitions
• Sometime hard to compute since many systems are shared with
other software systems: proper attribution models are required
• Salaries of the team developing and fsupporting SRP

Recommended for you

'A Journey to the Centre of Customer-centricity' - Jeff Evans
'A Journey to the Centre of Customer-centricity' - Jeff Evans'A Journey to the Centre of Customer-centricity' - Jeff Evans
'A Journey to the Centre of Customer-centricity' - Jeff Evans

Organisations that thrive are putting their customers in the driving seat, delivering greater value and ensuring an easy, delightful customer experience, leading to an expanding and loyal customer base and improved, continuous profits.

jeff evanscustomer-centricity collectivecustomer-centricity
Wharton Disruptive Decisions Summit SF 2017
Wharton Disruptive Decisions Summit SF 2017Wharton Disruptive Decisions Summit SF 2017
Wharton Disruptive Decisions Summit SF 2017

I had the pleasure to speak at the DD Summit in SF this fall on the topic of building Customer Engagements teams from the ground up leveraging disruptive technologies like Big Data, Machine Learning and the right mix of MarTech platforms.

organizationinnovationmachine learning
How to use LTV predictions in Marketing Campaigns?
How to use LTV predictions in Marketing Campaigns?How to use LTV predictions in Marketing Campaigns?
How to use LTV predictions in Marketing Campaigns?

In today’s customer-focused market, it is very important to know the customer lifetime value (LTV). LTV helps companies focus their business around the most “profitable” customers and predict customer lifetime value through the use of machine learning. Read more more about LTV in marketing campaigns in this white paper. Learn how identify your most productive media channel using predicted LTV. Discover best practices approach to use LTV in marketing campaigns.

campaignscustomerexperiencecustomerinsights
©CX VIRTUAL SUMMIT
Goal: metrics improvement
• Typical goals: improve customer experience and business
• Subgoals: reduce operational costs
• Required metrics prioritization to drive the development in the
right direction. One metrics - too little, many metrics -> too
chaotic, no focus
• Top down system of OKRs is proven as a good way to drive
search engine development
©CX VIRTUAL SUMMIT
Continuous Circle of Improvement
• Ref: Architecture diagram
• Every subsystem has impact on business, customer experience
and operation costs
• Most of them produce very substantial impact
• The continuous circle: each of system must be continuously
improved
©CX VIRTUAL SUMMIT
High level - Circles of improvement
• Design metrics, work on new models, features, return, resample,
to improve
• Analyze user behavior data to find cases when users have bad
experience,
• Analyze root causes of bad CX
• work to resolve these cases (new data, new features, new
software systems, new models, )
©CX VIRTUAL SUMMIT
Examples of improvement
• Search Assistance
• Type ahead helps users to type their queries. 20-30% of queries
are from type ahead.
• Improvements of typeahead may cause double digit %
improvement in business revenues
• Large improvement in customer satisfaction / easy to use the
system

Recommended for you

Webinar: 5 steps to maximize recurring revenue
Webinar: 5 steps to maximize recurring revenue Webinar: 5 steps to maximize recurring revenue
Webinar: 5 steps to maximize recurring revenue

Are you maximizing your recurring revenue performance? Did you know $30 billion in recurring revenue goes unrenewed each year? ServiceSource & Rockwell Automation share 5 proven best practices to help industrial system companies boost and retain more revenue.

industrialrenewal managementmanufacturing
Customer Experience Strategy: Bad News is Good News & Doing the Whole Job
Customer Experience Strategy: Bad News is Good News & Doing the Whole JobCustomer Experience Strategy: Bad News is Good News & Doing the Whole Job
Customer Experience Strategy: Bad News is Good News & Doing the Whole Job

Customer experience management requires embracing of customers' constructive feedback. It is more systematic and broad than typical customer advocacy efforts. See http://ClearActionCX.com

cemcustomer experience managementcustomer experience strategy
Digital Customer Experience Strategy, DocuSign [FutureStack16]
Digital Customer Experience Strategy, DocuSign [FutureStack16]Digital Customer Experience Strategy, DocuSign [FutureStack16]
Digital Customer Experience Strategy, DocuSign [FutureStack16]

Digital Customer Experience Strategy Speaker: Erin Smith, Senior Manager Information Experience, DocuSign

futurestack16new relicdocusign
©CX VIRTUAL SUMMIT
Examples of improvement
• Query Understanding
• Deeper understanding of meaning of queries both in
compositional semantics (natural language search) and other
linguistic features, spell correction (reducing customer errors),
meaning affecting ranking results (strawberry in grocery search -
> fresh produce rather than jams)
©CX VIRTUAL SUMMIT
Examples of improvement
• Ranking
• Despite, presentations about search improvements frequently
focused on query understanding, because it’s easily visible
search improvement, relevance, conversion etc ranking gains
are big factor of search improvements - with big impact on
business and customer satisfaction
©CX VIRTUAL SUMMIT
Examples of improvement
• Search presentation
• How results are presented to customers. What title is shown.
What snippet is shown. Layout of the page. Map. How outputs of
several search engines are mixed
• Double digit % revenue, business impact and customer
experience
©CX VIRTUAL SUMMIT
Key technology stack
• NLP: query understanding, search assistance
• ML: ranking (learning to rank, various techniques from GBTs to
neural methods)
• Data Science, Statistics: analysis of experiments

Recommended for you

Steve Tuppen - Digital Service Management
Steve Tuppen - Digital Service ManagementSteve Tuppen - Digital Service Management
Steve Tuppen - Digital Service Management

In this session, Mozaic described new ways of working based on a cohesive model that incorporated digital and traditional service management methods.

CRM | CRM Software Company in Delhi
CRM | CRM Software Company in DelhiCRM | CRM Software Company in Delhi
CRM | CRM Software Company in Delhi

Vallesoft is an esteemed CRM Software Company in Delhi. We have been serving businesses of all sizes and industry verticals with our cloud-based CRM software in all over India.

crmdevelopmentdelhicrmsoftwarecompanyindelhicrmsoftwarecompaniesinindia
Digital Transformation Road Map for Customer Service with Dynamics 365
Digital Transformation Road Map for Customer Service with Dynamics 365Digital Transformation Road Map for Customer Service with Dynamics 365
Digital Transformation Road Map for Customer Service with Dynamics 365

Dynamics 365 for Customer Service: Digital Transformation Road Map for Customer Service with Dynamics 365. Drive digital transformation with a new approach to customer relationship management (CRM) solutions for a comprehensive view of your customers.

dynamics crmdynamics 365customer service
©CX VIRTUAL SUMMIT
Engineering Quality as investment
• SRP engines are software systems under high load (~ 10^5 qps
, billions items, 100ms latency times)
• Any downtime costs huge money to business due to lost
customers and big loss to customer experience
• Any malfunction - loss of sales, loss of customer experience
• High quality software engineering is crucial part of operations
• Investments into software engineering culture, good SRE and
DevOps - payback well both in short and even more in long
terms
©CX VIRTUAL SUMMIT
Team
• SRP engine team quality
• Important to have a team with good set of diverse talents from
high performance system engineering to machine learning / AI
(multiple functions: NLP, ranking, many others), design and
analysis of experiments/statisticians, SRE, product managers,
•
©CX VIRTUAL SUMMIT
Business Relations
• Productive relationship between business teams and search
engine team
• Product managers which understand business and
understanding engineerings (tech savvy PMs are necessary)
• Analytical and experimental frameworks to evaluate business
ideas, measure their impact key business and customer metrics
• Fast experimentation (engineering quality!) to experiment rapidly
with new search ideas to show their value to business
©CX VIRTUAL SUMMIT
Ops
• Typical SRM engine runs under 24*7 load with hundreds
millions users per month
• Downtime of the engine and any component is not an option
• High quality ops - important investment
• DevOps for canary testing and other scenarios is important
investment
• Observability, - search engine is very complex systems of
hundreds/thousand components

Recommended for you

EzDataMunch_Corporate_Qlikview2015
EzDataMunch_Corporate_Qlikview2015EzDataMunch_Corporate_Qlikview2015
EzDataMunch_Corporate_Qlikview2015

EzDataMunch is a pre-built platform of apps and templates that allows business managers to analyze data quickly without any development costs. It provides predefined dashboards, reports, and KPIs tailored for different business functions and industries. This enables users to gain insights from their data in days rather than months. The platform promises benefits like improved decision making, increased productivity, and savings from not having to develop custom reporting tools. It aims to provide an easy way for business users to discover and understand information across their organization.

EzDataMunch_Corporate_Linkedin
EzDataMunch_Corporate_LinkedinEzDataMunch_Corporate_Linkedin
EzDataMunch_Corporate_Linkedin

EzDataMunch is a pre-built business intelligence platform that allows business managers to analyze data across their organization quickly and comprehensively without needing development skills or resources. The platform contains templates, apps, and playbooks for common business functions and industries that users can configure for their own data and key performance indicators. EzDataMunch provides a low-cost solution to improve data-driven decision making through rapid insights.

Optimizing Innovation- Modular Toolchains that Enable Digital Transformations
Optimizing Innovation-  Modular Toolchains that Enable Digital TransformationsOptimizing Innovation-  Modular Toolchains that Enable Digital Transformations
Optimizing Innovation- Modular Toolchains that Enable Digital Transformations

Lean practices for software delivery are critical to digital transformation and innovation, and the failure to execute on them opens the door to disruption. Software investment and staffing decisions are made anecdotally, using static and stale slivers of data. But what if we could take an fMRI (Functional Magnetic Resonance Imaging) of the organization and see the flow of business value in real-time? See evidence of bottlenecks and use them to prioritize IT investment? Join us as we introduce the concept of Value Stream Networks and explain how to create a modular framework enabling end-to-end business value flow, at any scale.

digital transformationdevopsagile
©CX VIRTUAL SUMMIT
Experimentation platform
• To support improvement circle
• to evaluate changes: ranking, query understanding,
autocomplete, user interface, snippets,
• to measure metrics impact
• How to continuously evaluate quality and impact of already
developed components: (it changes over time, some positive
impact component can become negative impact ones)
©CX VIRTUAL SUMMIT
Summary
• Key investments:
• High quality engineering culture, engineering productivity
• Team (wide and diverse)
• Business relations
• Experimentation platform and culture
• Science culture and tools
• Continuous improvement circle and tools to support it at any part
search engine stack
• Operations

More Related Content

What's hot

The Journey to Revenue Growth
The Journey to Revenue GrowthThe Journey to Revenue Growth
The Journey to Revenue Growth
McKinsey on Marketing & Sales
 
The Use of A.I. in Sales & Marketing Pipeline Build - A Case Study
The Use of A.I. in Sales & Marketing Pipeline Build - A Case StudyThe Use of A.I. in Sales & Marketing Pipeline Build - A Case Study
The Use of A.I. in Sales & Marketing Pipeline Build - A Case Study
Barry Magee
 
Developing a cult of analytics
Developing a cult of analyticsDeveloping a cult of analytics
Developing a cult of analytics
Steve Jackson
 
John Glinski - The B2B Analytics Engine: How Vanguard is Measuring Marketing...
John Glinski - 	The B2B Analytics Engine: How Vanguard is Measuring Marketing...John Glinski - 	The B2B Analytics Engine: How Vanguard is Measuring Marketing...
John Glinski - The B2B Analytics Engine: How Vanguard is Measuring Marketing...
Julia Grosman
 
The Power of a Proper North Star Metric
The Power of a Proper North Star MetricThe Power of a Proper North Star Metric
The Power of a Proper North Star Metric
RocketSource
 
Effective Lead Generation through Lead Lifecycle Management
Effective Lead Generation through Lead Lifecycle ManagementEffective Lead Generation through Lead Lifecycle Management
Effective Lead Generation through Lead Lifecycle Management
edynamic
 
Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...
Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...
Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...
RocketSource
 
What Factors Determine Contact Center Channels ?
What Factors Determine Contact Center Channels ?What Factors Determine Contact Center Channels ?
What Factors Determine Contact Center Channels ?
CallCenterHosting
 
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěchPokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch
Akce Dobrého webu
 
Adobe Experience Platform Bootcamp
Adobe Experience Platform BootcampAdobe Experience Platform Bootcamp
Adobe Experience Platform Bootcamp
Ritesh Gupta, OMCP
 
Humanizing Big Data: The Key to Actionable Customer Journey Analytics
Humanizing Big Data: The Key to Actionable Customer Journey AnalyticsHumanizing Big Data: The Key to Actionable Customer Journey Analytics
Humanizing Big Data: The Key to Actionable Customer Journey Analytics
RocketSource
 
Revenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic BlueprintRevenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic Blueprint
Kwanzoo Inc
 
'A Journey to the Centre of Customer-centricity' - Jeff Evans
'A Journey to the Centre of Customer-centricity' - Jeff Evans'A Journey to the Centre of Customer-centricity' - Jeff Evans
'A Journey to the Centre of Customer-centricity' - Jeff Evans
LemonTree Fundraising
 
Wharton Disruptive Decisions Summit SF 2017
Wharton Disruptive Decisions Summit SF 2017Wharton Disruptive Decisions Summit SF 2017
Wharton Disruptive Decisions Summit SF 2017
Robert Sibo
 
How to use LTV predictions in Marketing Campaigns?
How to use LTV predictions in Marketing Campaigns?How to use LTV predictions in Marketing Campaigns?
How to use LTV predictions in Marketing Campaigns?
Addepto
 
Webinar: 5 steps to maximize recurring revenue
Webinar: 5 steps to maximize recurring revenue Webinar: 5 steps to maximize recurring revenue
Webinar: 5 steps to maximize recurring revenue
ServiceSource
 
Customer Experience Strategy: Bad News is Good News & Doing the Whole Job
Customer Experience Strategy: Bad News is Good News & Doing the Whole JobCustomer Experience Strategy: Bad News is Good News & Doing the Whole Job
Customer Experience Strategy: Bad News is Good News & Doing the Whole Job
ClearAction
 
Digital Customer Experience Strategy, DocuSign [FutureStack16]
Digital Customer Experience Strategy, DocuSign [FutureStack16]Digital Customer Experience Strategy, DocuSign [FutureStack16]
Digital Customer Experience Strategy, DocuSign [FutureStack16]
New Relic
 

What's hot (18)

The Journey to Revenue Growth
The Journey to Revenue GrowthThe Journey to Revenue Growth
The Journey to Revenue Growth
 
The Use of A.I. in Sales & Marketing Pipeline Build - A Case Study
The Use of A.I. in Sales & Marketing Pipeline Build - A Case StudyThe Use of A.I. in Sales & Marketing Pipeline Build - A Case Study
The Use of A.I. in Sales & Marketing Pipeline Build - A Case Study
 
Developing a cult of analytics
Developing a cult of analyticsDeveloping a cult of analytics
Developing a cult of analytics
 
John Glinski - The B2B Analytics Engine: How Vanguard is Measuring Marketing...
John Glinski - 	The B2B Analytics Engine: How Vanguard is Measuring Marketing...John Glinski - 	The B2B Analytics Engine: How Vanguard is Measuring Marketing...
John Glinski - The B2B Analytics Engine: How Vanguard is Measuring Marketing...
 
The Power of a Proper North Star Metric
The Power of a Proper North Star MetricThe Power of a Proper North Star Metric
The Power of a Proper North Star Metric
 
Effective Lead Generation through Lead Lifecycle Management
Effective Lead Generation through Lead Lifecycle ManagementEffective Lead Generation through Lead Lifecycle Management
Effective Lead Generation through Lead Lifecycle Management
 
Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...
Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...
Leveraging the StoryVesting Framework to Achieve Breakthrough Business Transf...
 
What Factors Determine Contact Center Channels ?
What Factors Determine Contact Center Channels ?What Factors Determine Contact Center Channels ?
What Factors Determine Contact Center Channels ?
 
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěchPokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch
Pokročilá segmentace – nezbytný předpoklad pro dlouhodobý úspěch
 
Adobe Experience Platform Bootcamp
Adobe Experience Platform BootcampAdobe Experience Platform Bootcamp
Adobe Experience Platform Bootcamp
 
Humanizing Big Data: The Key to Actionable Customer Journey Analytics
Humanizing Big Data: The Key to Actionable Customer Journey AnalyticsHumanizing Big Data: The Key to Actionable Customer Journey Analytics
Humanizing Big Data: The Key to Actionable Customer Journey Analytics
 
Revenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic BlueprintRevenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic Blueprint
 
'A Journey to the Centre of Customer-centricity' - Jeff Evans
'A Journey to the Centre of Customer-centricity' - Jeff Evans'A Journey to the Centre of Customer-centricity' - Jeff Evans
'A Journey to the Centre of Customer-centricity' - Jeff Evans
 
Wharton Disruptive Decisions Summit SF 2017
Wharton Disruptive Decisions Summit SF 2017Wharton Disruptive Decisions Summit SF 2017
Wharton Disruptive Decisions Summit SF 2017
 
How to use LTV predictions in Marketing Campaigns?
How to use LTV predictions in Marketing Campaigns?How to use LTV predictions in Marketing Campaigns?
How to use LTV predictions in Marketing Campaigns?
 
Webinar: 5 steps to maximize recurring revenue
Webinar: 5 steps to maximize recurring revenue Webinar: 5 steps to maximize recurring revenue
Webinar: 5 steps to maximize recurring revenue
 
Customer Experience Strategy: Bad News is Good News & Doing the Whole Job
Customer Experience Strategy: Bad News is Good News & Doing the Whole JobCustomer Experience Strategy: Bad News is Good News & Doing the Whole Job
Customer Experience Strategy: Bad News is Good News & Doing the Whole Job
 
Digital Customer Experience Strategy, DocuSign [FutureStack16]
Digital Customer Experience Strategy, DocuSign [FutureStack16]Digital Customer Experience Strategy, DocuSign [FutureStack16]
Digital Customer Experience Strategy, DocuSign [FutureStack16]
 

Similar to Driving Customer Experience and Business Revenues Through Search Engines

Steve Tuppen - Digital Service Management
Steve Tuppen - Digital Service ManagementSteve Tuppen - Digital Service Management
Steve Tuppen - Digital Service Management
itSMF UK
 
CRM | CRM Software Company in Delhi
CRM | CRM Software Company in DelhiCRM | CRM Software Company in Delhi
CRM | CRM Software Company in Delhi
marketingvallesoft
 
Digital Transformation Road Map for Customer Service with Dynamics 365
Digital Transformation Road Map for Customer Service with Dynamics 365Digital Transformation Road Map for Customer Service with Dynamics 365
Digital Transformation Road Map for Customer Service with Dynamics 365
Manish Chauhan
 
EzDataMunch_Corporate_Qlikview2015
EzDataMunch_Corporate_Qlikview2015EzDataMunch_Corporate_Qlikview2015
EzDataMunch_Corporate_Qlikview2015
milanmvd
 
EzDataMunch_Corporate_Linkedin
EzDataMunch_Corporate_LinkedinEzDataMunch_Corporate_Linkedin
EzDataMunch_Corporate_Linkedin
Anupama Desai, PMP
 
Optimizing Innovation- Modular Toolchains that Enable Digital Transformations
Optimizing Innovation-  Modular Toolchains that Enable Digital TransformationsOptimizing Innovation-  Modular Toolchains that Enable Digital Transformations
Optimizing Innovation- Modular Toolchains that Enable Digital Transformations
Tasktop
 
Optimizing Innovation: Modular Toolchains that Enable Digital Transformations
Optimizing Innovation: Modular Toolchains that Enable Digital TransformationsOptimizing Innovation: Modular Toolchains that Enable Digital Transformations
Optimizing Innovation: Modular Toolchains that Enable Digital Transformations
DevOps.com
 
Crm implementation in insurance sector
Crm implementation in insurance sectorCrm implementation in insurance sector
Crm implementation in insurance sector
Rahul Gupta
 
The Value of Next Generation Managed Services
The Value of Next Generation Managed ServicesThe Value of Next Generation Managed Services
The Value of Next Generation Managed Services
CloudHesive
 
Converged Systems Sales Playbook
Converged Systems Sales PlaybookConverged Systems Sales Playbook
Converged Systems Sales Playbook
Peter Stone - VSP/VTSP/SCSA
 
Digital Transformation - What it Means & Why it Matters
Digital Transformation - What it Means & Why it MattersDigital Transformation - What it Means & Why it Matters
Digital Transformation - What it Means & Why it Matters
Net at Work
 
GoQueue - Friction Less Customer Experience Management
GoQueue - Friction Less Customer Experience ManagementGoQueue - Friction Less Customer Experience Management
GoQueue - Friction Less Customer Experience Management
Hitesh Thakkar
 
Implementing Advanced Analytics Platform
Implementing Advanced Analytics PlatformImplementing Advanced Analytics Platform
Implementing Advanced Analytics Platform
Arvind Sathi
 
E Commerce: Its role and development
E Commerce: Its role and developmentE Commerce: Its role and development
E Commerce: Its role and development
Anubha Rastogi
 
servicedesk-plus-overview
servicedesk-plus-overviewservicedesk-plus-overview
servicedesk-plus-overview
Mohammad Sadegh Khatibpour
 
ServiceDesk Plus Overview Presentation
ServiceDesk Plus Overview PresentationServiceDesk Plus Overview Presentation
ServiceDesk Plus Overview Presentation
ServiceDesk Plus
 
Ux strategy design digital transformation
Ux strategy design digital transformationUx strategy design digital transformation
Ux strategy design digital transformation
Satyajit Roy
 
BAASS Connect 2013 - Getting CRM to Work For My Business
BAASS Connect 2013 - Getting CRM to Work For My BusinessBAASS Connect 2013 - Getting CRM to Work For My Business
BAASS Connect 2013 - Getting CRM to Work For My Business
BAASS Business Solutions Inc.
 
Integrated Order to Cash (O2C) Automation Software for Global Shared Services...
Integrated Order to Cash (O2C) Automation Software for Global Shared Services...Integrated Order to Cash (O2C) Automation Software for Global Shared Services...
Integrated Order to Cash (O2C) Automation Software for Global Shared Services...
Emagia
 
ESM Webinar Series Part 2 | The Keys to Optimal ESM are Automation and Integr...
ESM Webinar Series Part 2 | The Keys to Optimal ESM are Automation and Integr...ESM Webinar Series Part 2 | The Keys to Optimal ESM are Automation and Integr...
ESM Webinar Series Part 2 | The Keys to Optimal ESM are Automation and Integr...
Cprime
 

Similar to Driving Customer Experience and Business Revenues Through Search Engines (20)

Steve Tuppen - Digital Service Management
Steve Tuppen - Digital Service ManagementSteve Tuppen - Digital Service Management
Steve Tuppen - Digital Service Management
 
CRM | CRM Software Company in Delhi
CRM | CRM Software Company in DelhiCRM | CRM Software Company in Delhi
CRM | CRM Software Company in Delhi
 
Digital Transformation Road Map for Customer Service with Dynamics 365
Digital Transformation Road Map for Customer Service with Dynamics 365Digital Transformation Road Map for Customer Service with Dynamics 365
Digital Transformation Road Map for Customer Service with Dynamics 365
 
EzDataMunch_Corporate_Qlikview2015
EzDataMunch_Corporate_Qlikview2015EzDataMunch_Corporate_Qlikview2015
EzDataMunch_Corporate_Qlikview2015
 
EzDataMunch_Corporate_Linkedin
EzDataMunch_Corporate_LinkedinEzDataMunch_Corporate_Linkedin
EzDataMunch_Corporate_Linkedin
 
Optimizing Innovation- Modular Toolchains that Enable Digital Transformations
Optimizing Innovation-  Modular Toolchains that Enable Digital TransformationsOptimizing Innovation-  Modular Toolchains that Enable Digital Transformations
Optimizing Innovation- Modular Toolchains that Enable Digital Transformations
 
Optimizing Innovation: Modular Toolchains that Enable Digital Transformations
Optimizing Innovation: Modular Toolchains that Enable Digital TransformationsOptimizing Innovation: Modular Toolchains that Enable Digital Transformations
Optimizing Innovation: Modular Toolchains that Enable Digital Transformations
 
Crm implementation in insurance sector
Crm implementation in insurance sectorCrm implementation in insurance sector
Crm implementation in insurance sector
 
The Value of Next Generation Managed Services
The Value of Next Generation Managed ServicesThe Value of Next Generation Managed Services
The Value of Next Generation Managed Services
 
Converged Systems Sales Playbook
Converged Systems Sales PlaybookConverged Systems Sales Playbook
Converged Systems Sales Playbook
 
Digital Transformation - What it Means & Why it Matters
Digital Transformation - What it Means & Why it MattersDigital Transformation - What it Means & Why it Matters
Digital Transformation - What it Means & Why it Matters
 
GoQueue - Friction Less Customer Experience Management
GoQueue - Friction Less Customer Experience ManagementGoQueue - Friction Less Customer Experience Management
GoQueue - Friction Less Customer Experience Management
 
Implementing Advanced Analytics Platform
Implementing Advanced Analytics PlatformImplementing Advanced Analytics Platform
Implementing Advanced Analytics Platform
 
E Commerce: Its role and development
E Commerce: Its role and developmentE Commerce: Its role and development
E Commerce: Its role and development
 
servicedesk-plus-overview
servicedesk-plus-overviewservicedesk-plus-overview
servicedesk-plus-overview
 
ServiceDesk Plus Overview Presentation
ServiceDesk Plus Overview PresentationServiceDesk Plus Overview Presentation
ServiceDesk Plus Overview Presentation
 
Ux strategy design digital transformation
Ux strategy design digital transformationUx strategy design digital transformation
Ux strategy design digital transformation
 
BAASS Connect 2013 - Getting CRM to Work For My Business
BAASS Connect 2013 - Getting CRM to Work For My BusinessBAASS Connect 2013 - Getting CRM to Work For My Business
BAASS Connect 2013 - Getting CRM to Work For My Business
 
Integrated Order to Cash (O2C) Automation Software for Global Shared Services...
Integrated Order to Cash (O2C) Automation Software for Global Shared Services...Integrated Order to Cash (O2C) Automation Software for Global Shared Services...
Integrated Order to Cash (O2C) Automation Software for Global Shared Services...
 
ESM Webinar Series Part 2 | The Keys to Optimal ESM are Automation and Integr...
ESM Webinar Series Part 2 | The Keys to Optimal ESM are Automation and Integr...ESM Webinar Series Part 2 | The Keys to Optimal ESM are Automation and Integr...
ESM Webinar Series Part 2 | The Keys to Optimal ESM are Automation and Integr...
 

More from Andrei Lopatenko

Natural Language Processing at Scale
Natural Language Processing at ScaleNatural Language Processing at Scale
Natural Language Processing at Scale
Andrei Lopatenko
 
AI in multi billion search engines. Building AI and Search teams
AI in multi billion search engines. Building AI and Search teamsAI in multi billion search engines. Building AI and Search teams
AI in multi billion search engines. Building AI and Search teams
Andrei Lopatenko
 
AI in Multi Billion Search Engines. Career building in AI / Search. What make...
AI in Multi Billion Search Engines. Career building in AI / Search. What make...AI in Multi Billion Search Engines. Career building in AI / Search. What make...
AI in Multi Billion Search Engines. Career building in AI / Search. What make...
Andrei Lopatenko
 
AI in Search Engines
AI in Search EnginesAI in Search Engines
AI in Search Engines
Andrei Lopatenko
 
Building multi billion ( dollars, users, documents ) search engines on open ...
Building multi billion ( dollars, users, documents ) search engines  on open ...Building multi billion ( dollars, users, documents ) search engines  on open ...
Building multi billion ( dollars, users, documents ) search engines on open ...
Andrei Lopatenko
 
Data Science and Machine Learning for eCommerce and Retail
Data Science and Machine Learning for eCommerce and RetailData Science and Machine Learning for eCommerce and Retail
Data Science and Machine Learning for eCommerce and Retail
Andrei Lopatenko
 
Machine Learning for retail and ecommerce
Machine Learning for retail and ecommerceMachine Learning for retail and ecommerce
Machine Learning for retail and ecommerce
Andrei Lopatenko
 

More from Andrei Lopatenko (7)

Natural Language Processing at Scale
Natural Language Processing at ScaleNatural Language Processing at Scale
Natural Language Processing at Scale
 
AI in multi billion search engines. Building AI and Search teams
AI in multi billion search engines. Building AI and Search teamsAI in multi billion search engines. Building AI and Search teams
AI in multi billion search engines. Building AI and Search teams
 
AI in Multi Billion Search Engines. Career building in AI / Search. What make...
AI in Multi Billion Search Engines. Career building in AI / Search. What make...AI in Multi Billion Search Engines. Career building in AI / Search. What make...
AI in Multi Billion Search Engines. Career building in AI / Search. What make...
 
AI in Search Engines
AI in Search EnginesAI in Search Engines
AI in Search Engines
 
Building multi billion ( dollars, users, documents ) search engines on open ...
Building multi billion ( dollars, users, documents ) search engines  on open ...Building multi billion ( dollars, users, documents ) search engines  on open ...
Building multi billion ( dollars, users, documents ) search engines on open ...
 
Data Science and Machine Learning for eCommerce and Retail
Data Science and Machine Learning for eCommerce and RetailData Science and Machine Learning for eCommerce and Retail
Data Science and Machine Learning for eCommerce and Retail
 
Machine Learning for retail and ecommerce
Machine Learning for retail and ecommerceMachine Learning for retail and ecommerce
Machine Learning for retail and ecommerce
 

Recently uploaded

Intro to Amazon Web Services (AWS) and Gen AI
Intro to Amazon Web Services (AWS) and Gen AIIntro to Amazon Web Services (AWS) and Gen AI
Intro to Amazon Web Services (AWS) and Gen AI
Ortus Solutions, Corp
 
Top 10 Tips To Get Google AdSense For Your Website
Top 10 Tips To Get Google AdSense For Your WebsiteTop 10 Tips To Get Google AdSense For Your Website
Top 10 Tips To Get Google AdSense For Your Website
e-Definers Technology
 
Development of Chatbot Using AI\ML Technologies
Development of Chatbot Using AI\ML TechnologiesDevelopment of Chatbot Using AI\ML Technologies
Development of Chatbot Using AI\ML Technologies
MaisnamLuwangPibarel
 
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
Hironori Washizaki
 
Attendance Tracking From Paper To Digital
Attendance Tracking From Paper To DigitalAttendance Tracking From Paper To Digital
Attendance Tracking From Paper To Digital
Task Tracker
 
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdfResponsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Trackobit
 
MVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptxMVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptx
Mitchell Marsh
 
introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...
sachin chaurasia
 
Overview of ERP - Mechlin Technologies.pptx
Overview of ERP - Mechlin Technologies.pptxOverview of ERP - Mechlin Technologies.pptx
Overview of ERP - Mechlin Technologies.pptx
Mitchell Marsh
 
Abortion pills in Fujairah *((+971588192166*)☎️)¥) **Effective Abortion Pills...
Abortion pills in Fujairah *((+971588192166*)☎️)¥) **Effective Abortion Pills...Abortion pills in Fujairah *((+971588192166*)☎️)¥) **Effective Abortion Pills...
Abortion pills in Fujairah *((+971588192166*)☎️)¥) **Effective Abortion Pills...
Medical / Health Care (+971588192166) Mifepristone and Misoprostol tablets 200mg
 
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptxAddressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Sparity1
 
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
ssuser2b426d1
 
Cultural Shifts: Embracing DevOps for Organizational Transformation
Cultural Shifts: Embracing DevOps for Organizational TransformationCultural Shifts: Embracing DevOps for Organizational Transformation
Cultural Shifts: Embracing DevOps for Organizational Transformation
Mindfire Solution
 
How we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hoursHow we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hours
Ortus Solutions, Corp
 
Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …
908dutch
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
karim wahed
 
A Comparative Analysis of Functional and Non-Functional Testing.pdf
A Comparative Analysis of Functional and Non-Functional Testing.pdfA Comparative Analysis of Functional and Non-Functional Testing.pdf
A Comparative Analysis of Functional and Non-Functional Testing.pdf
kalichargn70th171
 
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
avufu
 
active-directory-auditing-solution (2).pptx
active-directory-auditing-solution (2).pptxactive-directory-auditing-solution (2).pptx
active-directory-auditing-solution (2).pptx
sudsdeep
 
Migrate your Infrastructure to the AWS Cloud
Migrate your Infrastructure to the AWS CloudMigrate your Infrastructure to the AWS Cloud
Migrate your Infrastructure to the AWS Cloud
Ortus Solutions, Corp
 

Recently uploaded (20)

Intro to Amazon Web Services (AWS) and Gen AI
Intro to Amazon Web Services (AWS) and Gen AIIntro to Amazon Web Services (AWS) and Gen AI
Intro to Amazon Web Services (AWS) and Gen AI
 
Top 10 Tips To Get Google AdSense For Your Website
Top 10 Tips To Get Google AdSense For Your WebsiteTop 10 Tips To Get Google AdSense For Your Website
Top 10 Tips To Get Google AdSense For Your Website
 
Development of Chatbot Using AI\ML Technologies
Development of Chatbot Using AI\ML TechnologiesDevelopment of Chatbot Using AI\ML Technologies
Development of Chatbot Using AI\ML Technologies
 
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
 
Attendance Tracking From Paper To Digital
Attendance Tracking From Paper To DigitalAttendance Tracking From Paper To Digital
Attendance Tracking From Paper To Digital
 
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdfResponsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdf
 
MVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptxMVP Mobile Application - Codearrest.pptx
MVP Mobile Application - Codearrest.pptx
 
introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...
 
Overview of ERP - Mechlin Technologies.pptx
Overview of ERP - Mechlin Technologies.pptxOverview of ERP - Mechlin Technologies.pptx
Overview of ERP - Mechlin Technologies.pptx
 
Abortion pills in Fujairah *((+971588192166*)☎️)¥) **Effective Abortion Pills...
Abortion pills in Fujairah *((+971588192166*)☎️)¥) **Effective Abortion Pills...Abortion pills in Fujairah *((+971588192166*)☎️)¥) **Effective Abortion Pills...
Abortion pills in Fujairah *((+971588192166*)☎️)¥) **Effective Abortion Pills...
 
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptxAddressing the Top 9 User Pain Points with Visual Design Elements.pptx
Addressing the Top 9 User Pain Points with Visual Design Elements.pptx
 
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
 
Cultural Shifts: Embracing DevOps for Organizational Transformation
Cultural Shifts: Embracing DevOps for Organizational TransformationCultural Shifts: Embracing DevOps for Organizational Transformation
Cultural Shifts: Embracing DevOps for Organizational Transformation
 
How we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hoursHow we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hours
 
Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …
 
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
 
A Comparative Analysis of Functional and Non-Functional Testing.pdf
A Comparative Analysis of Functional and Non-Functional Testing.pdfA Comparative Analysis of Functional and Non-Functional Testing.pdf
A Comparative Analysis of Functional and Non-Functional Testing.pdf
 
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
 
active-directory-auditing-solution (2).pptx
active-directory-auditing-solution (2).pptxactive-directory-auditing-solution (2).pptx
active-directory-auditing-solution (2).pptx
 
Migrate your Infrastructure to the AWS Cloud
Migrate your Infrastructure to the AWS CloudMigrate your Infrastructure to the AWS Cloud
Migrate your Infrastructure to the AWS Cloud
 

Driving Customer Experience and Business Revenues Through Search Engines

  • 1. Andrei Lopatenko, VP Eng Zillow Driving Customer Experience and Business Revenues Through Search Engines, Recommendations and Conversational Interfaces
  • 2. ©CX VIRTUAL SUMMIT What will you learn from this talk • How to drive continuous circle of improvements of your search/recommendation/personalization engines or conversational system leading to improvement of customer experience and business revenues • What investments in this area deliver business value and visible as investments improving business and customer experience
  • 3. ©CX VIRTUAL SUMMIT Search, Recommendation, Personalization • In this talk, we are talking about search, recommendation, personalization engines driving your business, key gateways letting your customers to find and discover your business items to buy, book, rent, etc them • We do not talk about small secondary search engines (search in forums of your web site, etc)
  • 4. ©CX VIRTUAL SUMMIT Engines • We put Search, Recommendation, Personalization engine into one category - SRP engine • Despite very different product functions, under the hood, they implement similar functionality, letting user to express their interests (‘query’) explicitly or implicitly, and giving user a set of results relevant to the user’s interest. But ‘query’ can be a text (search) query, user id (personalization) , location, item id etc (recommendation) , • Examples of SRP engines: Google.com, Yelp (localsearch and local recommendation), Apple Appstore, Amazon or Walmart online (search, item recommendation, personalization), Zillow.com, • Despite they are different, they are a lot of similarities and recommendations of this talk are valid for all of them
  • 5. ©CX VIRTUAL SUMMIT SRP Engines • Typically, recommendation and personalization engines are smaller siblings of search engines, since they get 10X less customers, and produce 10X less revenue • But nevertheless, they are very impactful revenue generators and affect customer experience in a significant way after search engines
  • 6. ©CX VIRTUAL SUMMIT Conversational Interfaces • Conversational interfaces are voice based or text based interfaces letting user to interactivity communicate with business services (voice search-> buy, ask questions -> get answers, voice based customer support) • Some of them utilize search engines to answer users’ questions. • They have a lot tech shared with SRP, but a lot differences in science and technologies • Nevertheless, key decisions how to develop and run them are shared with SRP, so most of focus of this talk is about SRP (due to time limit), but recommendations are applicable to Conversational Interfaces
  • 7. ©CX VIRTUAL SUMMIT Customer experience • Frequently, we separate customer experience (CX) metrics (typically measured through various behavior metrics) from business metrics (such as revenue per sessions, revenue per month, GMV). • CX and business are related but not directly. • Good customer experience leads to growth (new users stick with your business, existing users do not move to competitors) • Customer experience metrics must be designed for your business, few CX metrics are applicable to every SRP engine. CX metrics reflect how users interact with your business site/app
  • 8. ©CX VIRTUAL SUMMIT Revenue/Business • SRP engines drive your business since customers buy, book, rent through SRP engines • SRP engines produce cash flow • And they produce expenses (operational support, development support, data acquisition, software costs)
  • 9. ©CX VIRTUAL SUMMIT Typical architecture of SRP Engine
  • 10. ©CX VIRTUAL SUMMIT Typical architecture • There are a lot of variations in that diagram: but overall it’s the same flow: search assistance -> query understanding -> retrieval -> ranking -> presentation generator • Where different technologies can be used at every layer: either traditional space feature IR or dense representation retrieval or similarity search etc but still high level architecture is very similar across systems
  • 11. ©CX VIRTUAL SUMMIT Typical Entry points of Customer Experience • Autocomplete/Autosuggest • Filters/Facets • Refinements/Query suggestions • Search box • Search Results page • Recommendations on search result pages • Personalizated lists
  • 12. ©CX VIRTUAL SUMMIT Customer experience Metrics • Many way to measure satisfaction of customers • From offline - questionaries, user studies to live traffic/behavior based • Click through rates and shallow engagement metrics • Conversion rates and deep engagement metrics • Subscription/unsubscription rates
  • 13. ©CX VIRTUAL SUMMIT Customer Experience Metrics • Longitudinal studies: when customer come back, behavior shifts over time • Longitudinal studies - How frequently customer use SRP engine / the number of sessions per week/month/year, why they stay or leave • Abandonment rates, queries with no results, session with no clicks • Many other metrics, frequently needs to be designed for your business and your customers • Many of metrics should be studied by subcategories of services, query types, customer types
  • 14. ©CX VIRTUAL SUMMIT Business Metrics • Revenue based: conversion per query and conversion per session • Total Revenue per week/month/year , Gross Merchandise Value in C2C case • Conversion per customer •
  • 15. ©CX VIRTUAL SUMMIT Business Metrics • Other business targets • Revenue/number of engaged customer per certain business segments • Cross category basket size • Depends on business goals • Should be measured and used to turn models, build systems to develop business in this way
  • 16. ©CX VIRTUAL SUMMIT Operational Cost Metrics • Direct operational costs to support search engine running: cost of datacenter or cloud costs, cost of data acquisitions • Sometime hard to compute since many systems are shared with other software systems: proper attribution models are required • Salaries of the team developing and fsupporting SRP
  • 17. ©CX VIRTUAL SUMMIT Goal: metrics improvement • Typical goals: improve customer experience and business • Subgoals: reduce operational costs • Required metrics prioritization to drive the development in the right direction. One metrics - too little, many metrics -> too chaotic, no focus • Top down system of OKRs is proven as a good way to drive search engine development
  • 18. ©CX VIRTUAL SUMMIT Continuous Circle of Improvement • Ref: Architecture diagram • Every subsystem has impact on business, customer experience and operation costs • Most of them produce very substantial impact • The continuous circle: each of system must be continuously improved
  • 19. ©CX VIRTUAL SUMMIT High level - Circles of improvement • Design metrics, work on new models, features, return, resample, to improve • Analyze user behavior data to find cases when users have bad experience, • Analyze root causes of bad CX • work to resolve these cases (new data, new features, new software systems, new models, )
  • 20. ©CX VIRTUAL SUMMIT Examples of improvement • Search Assistance • Type ahead helps users to type their queries. 20-30% of queries are from type ahead. • Improvements of typeahead may cause double digit % improvement in business revenues • Large improvement in customer satisfaction / easy to use the system
  • 21. ©CX VIRTUAL SUMMIT Examples of improvement • Query Understanding • Deeper understanding of meaning of queries both in compositional semantics (natural language search) and other linguistic features, spell correction (reducing customer errors), meaning affecting ranking results (strawberry in grocery search - > fresh produce rather than jams)
  • 22. ©CX VIRTUAL SUMMIT Examples of improvement • Ranking • Despite, presentations about search improvements frequently focused on query understanding, because it’s easily visible search improvement, relevance, conversion etc ranking gains are big factor of search improvements - with big impact on business and customer satisfaction
  • 23. ©CX VIRTUAL SUMMIT Examples of improvement • Search presentation • How results are presented to customers. What title is shown. What snippet is shown. Layout of the page. Map. How outputs of several search engines are mixed • Double digit % revenue, business impact and customer experience
  • 24. ©CX VIRTUAL SUMMIT Key technology stack • NLP: query understanding, search assistance • ML: ranking (learning to rank, various techniques from GBTs to neural methods) • Data Science, Statistics: analysis of experiments
  • 25. ©CX VIRTUAL SUMMIT Engineering Quality as investment • SRP engines are software systems under high load (~ 10^5 qps , billions items, 100ms latency times) • Any downtime costs huge money to business due to lost customers and big loss to customer experience • Any malfunction - loss of sales, loss of customer experience • High quality software engineering is crucial part of operations • Investments into software engineering culture, good SRE and DevOps - payback well both in short and even more in long terms
  • 26. ©CX VIRTUAL SUMMIT Team • SRP engine team quality • Important to have a team with good set of diverse talents from high performance system engineering to machine learning / AI (multiple functions: NLP, ranking, many others), design and analysis of experiments/statisticians, SRE, product managers, •
  • 27. ©CX VIRTUAL SUMMIT Business Relations • Productive relationship between business teams and search engine team • Product managers which understand business and understanding engineerings (tech savvy PMs are necessary) • Analytical and experimental frameworks to evaluate business ideas, measure their impact key business and customer metrics • Fast experimentation (engineering quality!) to experiment rapidly with new search ideas to show their value to business
  • 28. ©CX VIRTUAL SUMMIT Ops • Typical SRM engine runs under 24*7 load with hundreds millions users per month • Downtime of the engine and any component is not an option • High quality ops - important investment • DevOps for canary testing and other scenarios is important investment • Observability, - search engine is very complex systems of hundreds/thousand components
  • 29. ©CX VIRTUAL SUMMIT Experimentation platform • To support improvement circle • to evaluate changes: ranking, query understanding, autocomplete, user interface, snippets, • to measure metrics impact • How to continuously evaluate quality and impact of already developed components: (it changes over time, some positive impact component can become negative impact ones)
  • 30. ©CX VIRTUAL SUMMIT Summary • Key investments: • High quality engineering culture, engineering productivity • Team (wide and diverse) • Business relations • Experimentation platform and culture • Science culture and tools • Continuous improvement circle and tools to support it at any part search engine stack • Operations