Driving Insights in the Digital Enterprise
•
1 like•360 views
Digital businesses are reimagining the technologies such as automation, analytics, cloud, and integration. These businesses are efficient, continuously optimizing, proactive, flexible and able to understand customers in detail. A key part of a digital business is analytics. It is the eyes and ears of the system that tracks and analyzes data to provide a detailed view on the past and the present, lets you predict the future for better decision-making and provides automated smart decisions in real-time. This session will explore how WSO2 Smart Analytics: Plays a role in your digital transformation journey Collects and analyzes streaming data in real-time, incrementally and with intelligence Communicates results effectively through dashboards and alerts Builds solutions with WSO2 Data Analytics Server
Report
Share
Driving Insights in the Digital Enterprise
- 1. DERIVING INSIGHTS IN DIGITAL ENTERPRISE WITH WSO2 SMART ANALYTICS S. Suhothayan Architect/Associate Director, WSO2 April 19, 2017
- 2. Smart Analytics Creating realtime, intelligent, actionable business insights, and data products 2
- 3. Smart Analytics 3 Enables ▪ Improve Efficiency ▪ Continuous optimization ▪ Be proactive & flexible ▪ Ensure smooth operation ▪ Help understand customers in detail ▪ Achieve financial gains in Digital Enterprise Source : http://cdn.business2community.com/wp-content/uploads/2012/08/Invest_Money_Photoxpress_2938045.jpg
- 4. ▪ Provide a detailed view on the past and the present ▪ Run predictions to the future ▪ Interactive analytics for investigation ▪ Automated smart decisions in real-time ▪ Business user friendliness ▪ Customizable dashboards and alerts Smart Analytics 4 Core Capabilities Source : https://cdn.pixabay.com/photo/2016/05/03/00/07/analytics-1368293_960_720.png
- 5. WSO2 Data Analytics Server Realtime Incremental Intelligent 5
- 7. Role in Digital Transformation Journey
- 8. Advanced Monitoring and Policy Enforcement ▪ Correlates multiple data streams ▪ Detect usage patterns in realtime ▪ Enforce policy rules while providing alerts Via Realtime Complex Event Processing 8
- 9. ▪ Throttling with various throttling levels API, Application, Resource URL & Subscription ▪ Hierarchical throttling limits ▪ E.g. Each user has a throttling limit is 15 requests/min, and 1000 request/month, while cumulatively all users can process 1000 requests/sec. Advanced Monitoring and Policy Enforcement ... Throttling for WSO2 API Manager 9
- 10. Advanced Monitoring and Policy Enforcement ... Auto Scaling and Dynamic Resource Provisioning for Stratos ▪ Apache Stratos is a highly-extensible Platform-as-a-Service (PaaS) framework use WSO2 DAS for autoscaling. 10
- 11. Advanced Monitoring and Policy Enforcement ... ▪ Uber as Apache Eagle uses WSO2 Siddhi CEP engine (the core of WSO2 DAS) as their Policy Engine for detecting Taxi Fraud and Policy Violations and acting fast on them. As Policy Engine at Uber and Apache Eagle 11
- 12. Predictive Services ▪ Execute machine-learning models in realtime ▪ Automate actions ▪ E.g. : Detect Product Defects With Machine Learning Source : https://img.clipartfest.com/6c86f2f446bd320da5e71e43d0828e01_in-the-pages-that-follow-we-clipart-production-line_557-418.jpeg 12
- 13. Predictive Services ... Identifying Wait Time in Airports Source: http://i.dailymail.co.uk/i/pix/2016/05/16/20/3440003700000578-3593079-CHICAGO_IL_MAY_16_Passengers_at_O_Hare_International_Airport_wa i-a-95_1463426503445.jpg, http://www.waittimealliance.ca/wp-content/uploads/2014/04/clock.png 13
- 14. Threat Detection and Response ▪ Identify security threats and fraud ▪ Detect the class of threats ▪ Respond accordingly ▪ E.g: Value at risk with Monte Carlo Simulation Trends using Regression Analysis ... Via Advanced Analytics Techniques 14
- 15. Threat Detection and Response ... define stream Purchase (price double, cardNo long, place string); from every (a1 = Purchase[price < 10] ) -> a2 = Purchase[ price >10000 and a1.cardNo == a2.cardNo ] within 1 day select a1.cardNo as cardNo, a2.price as price, a2.place as place insert into PotentialFraud ; With Complex Queries 15
- 16. Threat Detection and Response ... ▪ Use Markov models to identify the probabilities of event sequences ▪ Detect anomalous event sequences (eg: ABE) in real-time by comparing sequence probabilities against a threshold Through Activity Sequences C B F A E 0.9 0.9 0.90.1 0.9 0.1 0.1 0.1 16
- 17. Threat Detection and Response ... ▪ Achieve surveillance using Geo Fencing ▪ Detect logins from multiple IPs, from multiple locations, and during abnormal times With Geographical Processing 17
- 18. Threat Detection and Response ... Multiple Response Options 18 + Pluggable Custom Publishers
- 19. Intelligent Decision-Making ▪ Detect and address issues and identify new opportunities. ▪ Remove false positives using Scoring With Scoring & Anomaly Detection Source : https://img.clipartfest.com/6c86f2f446bd320da5e71e43d0828e01_in-the-pages-that-follow-we-clipart-production-line_557-418.jpeg Score = 0.001 * itemPrice + 0.1 * itemQuantity + 2.5 * isFreeEmail + 5 * riskyCountry + 8 * suspicousIPRange + 5 * suspicousUsername + 3 * highTransactionVelocity 19
- 20. New revenue streams ▪ Generate insights from Raw or processed data Or in the form of Artificial Intelligence (AI) ▪ Are valuable new digital products ▪ Provide new business value across all your digital initiatives From Insights 20
- 21. New revenue streams ▪ WSO2 Analytics for API management, Integration, Identity & IoT ▪ And as Smart Analytics Solutions for Several Domains As Solutions 21
- 23. Interoperability with Digital Enterprise Via Out of The Box Data Formats and Transport Protocols + Pluggable Custom Receivers 23
- 24. Respond to events in milliseconds ▪ Correlate data streams over time ▪ Find interesting occurrences ▪ And Notify ▪ All in Realtime ! With State of The Art Siddhi CEP Engine 24
- 25. Respond to events in milliseconds ... With Realtime Processing Patterns ▪ Transformation projection, transformation, enrich, split ▪ Temporal Aggregation basic stats, group by Aggregation, moving averages ▪ Alert and Threshold ▪ Event Correlation ▪ Trends detecting rise, fall, turn, triple bottom ▪ Partitioning ▪ Join Streams ▪ Query Data Stores 25
- 26. Respond to events in milliseconds ... With SQL Like Query Language define stream Sales (region string, brand string, quantity int, price double); from Sales[region == ‘USA’ and quantity > 99] select brand, price, quantity insert into WholeSales ; from Sales#window.time(1 hour) select region, brand, avg(quantity) as avgQuantity group by region, brand insert into LastHourSales ; 26 Filtering Aggregation over sliding 1 hour Other supported window types: timeBatch(), length(), lengthBatch(), etc.
- 27. Facilitate Smart Decisions ▪ Dashboard generation ▪ Gadget generation ▪ Gather data via Websockets Polling ▪ Custom / Personalized Gadget and Dashboard With Insights through Dashboards, Reports, and Alerts 27
- 28. Predictive Decision Making ▪ Author models in an accessible language, or generate models from a variety of popular machine-learning frameworks, such as : R, Python, Spark MLLib ▪ Export them via PMML ▪ Run them in realtime from DataStream#ml:predict(“/home/user/ml.model”, “double”) select * insert into PredictionStream ; With Machine Learning 28
- 29. Business Friendly Interface ▪ Developers create query templates With Trivial Rule Compilation and Informative Dashboard 29
- 30. Business Friendly Interface ... ▪ Business users manage them With Trivial Rule Compilation and Informative Dashboard 30
- 31. Investigative Analysis ▪ Apache Lucene based interactive querying support on historical data ▪ Drilldown search support ▪ Distributed near realtime data indexing With Interactive Analytics 31
- 32. Periodic and Just In Time Analysis ▪ Powered by Apache Spark ▪ Interactive Spark Console for just in time analysis ▪ Scheduled Job execution for Periodic analysis With Batch Analytics 32
- 33. Resource Optimisation ▪ Use realtime incremental reduction technique to calculate summations ▪ Lower CPU and Memory consumption With incremental computation 33
- 34. Grow With Data Scalable Deployment 2 Node Deployment Multi Node Deployment 34
- 35. Extend As You Wish ! Analytics Extension Store Extension for : ▪ Receivers ▪ Publishers ▪ Siddhi Functions https://store.wso2.com/ 35
- 37. Banking and Finance 37 ▪ Fraud Detection & Money Laundering Detection ▪ Risk Management (Value at Risk) ▪ Stock Market Surveillance Sources:http://newbusinessage.com/Articles/view/3960,https://www.moneysmart.gov.au/scams/investment-scams/inflating-the-share -price,https://upload.wikimedia.org/wikipedia/commons/d/d7/Philippine-stock-market-board.jpg
- 38. eCommerce and Digital Marketing 38 ▪ Recommendations Engine ▪ Ad Optimization ▪ Proximity Marketing
- 39. ▪ IBeacons and RFID based object tracking ▪ Geographical Processing Fleet Management 39
- 40. ▪ Catering Smart Grid, Smart Homes and Oil and Gas Industries. Smart Energy Analytics 40 DEBS 2014
- 41. ▪ Sentiment Analysis ▪ Most Popular Tweets ▪ NLP Processing Social Media Analysis 41 Realtime Count Word Cloud
- 42. ▪ System Monitoring ▪ Autoscaling ▪ Throttling Operational Efficiency 42
- 43. ▪ HL7 Support ▪ Patient Motoring ▪ Hospital Management Health Care 43
- 44. Success Stories
- 45. Experian delivers a digital marketing platform, where CEP plays a key role to analyze in real-time customers behavior and offer targeted promotions. CEP was chosen after careful analysis, primarily for its openness, its open source nature, the fact support is driven by engineers and the availability of a complete middleware, integrated with CEP, for additional use cases. Eurecat is the Catalonia innovation center (in Spain) - Using CEP to analyze data from iBeacons deployed within department stores to offer instant rebates to user or send them help if it’s detected that they seem “stuck” in the shop area. They chose WSO2 due to real-time processing, the variety of IoT connectors available as well as the extensible framework and the rich configuration language. They also use WSO2 ESB in conjunction with WSO2 CEP. Pacific Controls is an innovative company delivering an IoT platform of platforms: Galaxy 2021. The platform allows to manage all kinds of devices within a building and take automated decisions such as moving an elevator or starting the air conditioning based on certain conditions. Within Galaxy 2021, CEP is used for monitoring alarms and specific conditions.Pacific Controls also uses other products from the WSO2 platform, such as WSO2 ESB and Identity Server 45 A leading airline uses CEP to enhance customer experience by calculating the average time to reach their boarding gate (going through security, walking, etc.). They also want to track the time it takes to clean a plane, in order to better streamline the boarding process and notify both the airline and customers about potential delays. They evaluated WSO2 CEP first as they were already using our platform and decided to use it as it addressed all their requirements. Success Stories
- 46. 46 a Winning the Data in Motion Hack Week with AWS and Geovation, providing an impressive solution, taking the data from many modes of transport and overlaying passenger flow/train loading and pollution data, and allowing users to plan a route based on how busy their stations/routes are, whilst also taking air quality into account. DEBS (Distributed Event Based Systems) Challenge in Smart Home electricity data: 2000 sensors, 40 houses, 4 billion events. We posted fastest single node solution measured (400K events/sec) and close to one million distributed throughput. WSO2 CEP based solution is one of the four finalists, and the only generic solution to become a finalist. Build solution to search, visualize, analyze healthcare records (HL7) across 20 hospitals in Italy, with the combination of WSO2 ESB. Foods supply company in USA, detects anomalies such as delivery delays and provides personalized notifications, and makes order recommendations based on history. DEBS 2014 Success Stories ...
- 47. Market Recognition ● Named as a Strong Performer in The Forrester Wave™: Big Data Streaming Analytics, Q1 2016. ● Highest score possible in 'Acquisition and Pricing' criteria, and among second-highest scores in 'Ability to execute' criteria ● The Forrester Report notes….. “WSO2 is an open source middleware provider that includes a full spectrum of architected-as-one components such as application servers, message brokers, enterprise service bus, and many others. Its streaming analytics solution follows the complex event processor architectural approach, so it provides very low-latency analytics. Enterprises that already use WSO2 middleware can add CEP seamlessly. Enterprises looking for a full middleware stack that includes streaming analytics will find a place for WSO2 on their shortlist as well.” 47
- 48. Download and Try 48 WSO2 Data Analytics Server http://wso2.com/smart-analytics/ Smart Analytics Solutions http://wso2.com/smart-analytics/solutions/