More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Spirent: Datum User Experience Analytics System
Similar to Spirent: Datum User Experience Analytics System (20)
More from Sailaja Tennati
More from Sailaja Tennati (20)
Recently uploaded
Recently uploaded (20)
Spirent: Datum User Experience Analytics System
- 1. www.spirent.com Datum User Experience Analytics System for Data Services Making 4G Faster: Getting to the Root of Issues Speed Bumps on the 4G Highway. Most users summarize the benefits of 4G LTE and LTE-A services in two words: “It’s faster!” Users expect blazing fast speeds that blow away 3G – but, the fact is, they don’t always get what they expect. With unprecedented network and device complexity, a variety of device, network and application layer issues are reducing the speed of 4G services for many users. All Devices Are NOT Created Equal. Device issues trump the network as a leading cause of impaired data throughput. Our studies across hundreds of device models show that the worst performing devices are 60% slower than the best devices (see Figure 1). The root causes of these issues range from firmware issues to antenna issues to poor design or IP configuration. The Network vs. The Device. With unprecedented complexity and new technologies like carrier aggregation rolling out, many 4G “speed bumps” originate in the network. At the top of the network-based root cause list: improperly configured proxy servers. That’s followed by an array of root causes from other IP/core network configuration issues to RF interference to base station configuration issues. Making 4G Faster with IP Analytics. When we launched Datum in 2007, our customers needed answers to questions like: “how does user experience compare across top devices?” and “is my data service faster than the competition?” Today those same customers are also asking: “why is a device or service not performing?” and “what are the IP-related root causes?” That’s why we created the IP Analytics module for Datum. With this new capability, Datum significantly accelerates root cause analysis and helps carriers and device manufacturers make 4G faster than ever before. Highlights • Measure user experience with a unified approach across all major mobile OS platforms and PCs • Measure in the lab or the live network • Accelerate root cause analysis of web browsing, file transfer and multi-service usage issues • Efficiently measure user experience and manage test cases and data across large numbers of test devices • Pre-test devices to accelerate Spirent Fit4Launch submissions and carrier acceptance All Devices Aren’t Created Equal User Experience Analytics by Spirent Figure 1: A comparison of live network file download speeds for 23 device models from 11 device manufacturers. Each device was compared to the same reference device. -60% -40% -20% 0% 20% 40% Device OEM 11 Device OEM 11 Device OEM 11 Device OEM 10 Device OEM 9 Device OEM 8 Device OEM 1 Device OEM 7 Device OEM 5 Device OEM 5 Device OEM 7 Device OEM 6 Device OEM 3 Device OEM 6 Device OEM 7 Device OEM 6 Device OEM 6 Device OEM 6 Device OEM 5 Device OEM 4 Device OEM 3 Device OEM 2 Device OEM 1 Live Network File Download SpeedLive Network File Download Speed
- 2. SPIRENT Datum | User Experience Analytics System for Data Services 2 | www.spirent.com Datum: Experience Analytics for Data Service and Multi-Service Usage Use a Consistent Test Approach on Any Mobile Device. Datum measures user experience with a consistent approach across any major mobile OS platform or Windows-compatible portable PC. That enables true apples-to-apples comparisons of user experience across devices, service providers and network technologies. Datum’s consistent metrics help device, network and service teams work together to identify and solve user experience issues. Manage the Measurement Process Centrally. Datum’s cloud- based servers provide a central repository for all test cases, allowing these to be centrally managed. Furthermore, all test results are also uploaded to the central repository, enabling seamless management of user experience measurement processes across teams or geographic locations. Define Realistic Use Cases. Datum enables the definition of use cases (referred to as tests sets) which mimic real-world user activities such as uploading a picture, downloading a file attachment, updating social network status or browsing a website. Each test set consists of one or more of the following tasks: web browsing, file transfer, streaming data, multi- service (call and data) and latency test cases. Tasks can be combined in any order and are highly configurable to match specific user profiles or scenarios. For example, the following test set measures the data experience of a young adult user: Troubleshoot Issues with IP Analytics. After tests are completed and results are uploaded to the cloud, user experience metrics may be synchronized with IP logs collected on the mobile device and/or media server. Users may then access detailed analytics reports on the combined IP logs with the following analytics: IP Summary, IP Address Checks, DNS Checks, IP Fragmentation & QoS Detection, Observed Maximum Segment Size, IP Throughput Charts, TCP Window Size Analysis and Application Layer Event Diagram. User Experience Analytics by Spirent Datum is part of a suite of systems that provide an unmatched range of user experience analytics including: Speech Call Web Browsing File Transfer Video Battery Life Android Mobile OS support including Android, Blackberry, Brew, iOS, Java, WindowsPhone iOS WindowsPhone User Activity Datum Test Case Metrics Browsing the web while talking Start a call, then load web pages Web page load time Watching a video on YouTube Download a 10 MB file over HTTP File download speed Posting a photo to Facebook Upload a 1 MB file over HTTP File upload speed
- 3. SPIRENT www.spirent.com | 3 Datum | User Experience Analytics System for Data Services Datum System Architecture Datum Mobile: Data-Only Option. Datum Mobile is an on- device agent which connects to cloud-based media servers to initiate data experience tests including HTTP and FTP file transfers, web page downloads, ping tests and more. The Datum Mobile app may be downloaded and installed directly from the relevant app store (iOS, Android or WindowsPhone). Datum Mobile: Multi-Service Option. For Android devices, Datum Mobile performs multi-service measurements for simultaneous voice and data sessions. Metrics include throughput before, after and during a call and multi-RAB call performance. Datum Mobile also integrates with Spirent’s voice experience analytics system, Nomad, to perform multi- service measurements for all leading mobile OS platforms. Datum PC. Datum PC is a Windows-compatible software agent which initiates data experience tests including HTTP and FTP file transfers, web page downloads, ping tests and more. Datum PC may be used to test USB data dongles or integrated wireless or Ethernet capabilities for Windows-compatible PCs. Datum Cloud. Datum Cloud includes the DatumBase web portal and repository and Call and Media Servers. DatumBase is a web-based application, hosted by Spirent, which acts as a repository for test set definitions and test results and also as a reporting and analytics web-interface (see IP Analytics). Call and media servers connect to the Datum Mobile or Datum PC agent to perform end-to-end data tasks and make calls for multi-service measurements. For lab-based testing, Datum Cloud servers are available in a Datum Lab Appliance which may be deployed within a lab IT network. IP Analytics Module. The IP Analytics module is accessed via the DatumBase web-interface. The module uses a “watermarking” technique to synchronize and combine IP logs from the device and/or media server with user experience metrics. An analytics engine produces key troubleshooting metrics and performs a rule-based analysis of the data to automatically detect problems. Metrics and problem results are available via a web-based IP Analytics report.
- 4. SPIRENT Datum | User Experience Analytics System for Data Services AMERICAS 1-800-SPIRENT | +1-818-676-2683 | sales@spirent.com EUROPE AND THE MIDDLE EAST +44 (0) 1293 767979 | emeainfo@spirent.com ASIA AND THE PACIFIC +86-10-8518-2539 | salesasia@spirent.com © 2014 Spirent Communications, Inc. All of the company names and/or brand names and/or product names and/or logos referred to in this document, in particular the name “Spirent” and its logo device, are either registered trademarks or trademarks pending registration in accordance with relevant national laws. All rights reserved. Specifications subject to change without notice. Rev. A 01/14 How Does Datum Compare? Features Datum In-House Test Tools “Free” Test Apps Network Engineering Tools Commercial solution with global professional support network YES NO NO YES Consistent test capability across PC and all major mobile OS platforms/devices YES NO NO NO Comprehensive range of test cases (Web Browsing, Latency, Streaming, File Transfer) YES NO NO NO Measures call performance with or without concurrent data activity YES NO NO NO Fully transparent test cases and documented KPI’s YES YES NO YES Centralized online management, configuration & reporting YES NO NO NO Centralized real-time access to test results YES NO NO NO Global network of dedicated test servers YES NO YES NO Full access to server-side IP logs for troubleshooting/analysis YES YES NO NO Access to detailed troubleshooting parameters (location, RAT, IP addresses, etc.) YES NO NO YES Access to data performance troubleshooting expertise YES NO NO NO Unified management of field and lab testing YES NO NO NO Datum IP Analysis Report Version 1.0.0.58 Generated (UTC) 2014-01-23 20:20:57 Summary Information Datum Configuration Summary Project Name: Example 1 - PC TCP Scaling Test-Set Name: 25MB-Upload Datum Task Name: FTP_UPLOAD Datum Device ID/IMEI: NoRegistrySettings Stream ID: 1bfbdd69-067a-4eb5-9e77-6db821ae43b3 Test Start Time (UTC): 2013-12-17T18:20:59.057 Reported Device: Not Found Not Found Datum Mobile Test Results Measured Throughput (kbps) 4723.53 kbps — PASS ( > 200.0 kbps) IP Summary IP Summary (Media Server IP Log) Media Server IP Address: 216.157.111.6 IP Lookup Details: Host: 216.157.111.6 Country: United States (US) Device/Network Gateway/Proxy IP Address: 173.8.5.209 IP Lookup Details: Host: 173.8.5.209 Country: United States (US) IP Summary (Device IP Log) Media Server IP Address: 216.157.111.6 Device IP Address: 10.33.100.125 Source Data Download Download Media Server PCAP File for Test (MS_SAVE_216.157.111.6_1bfbdd69-067a-4eb5-9e77-6db821ae43b3.pcap) Download Device PCAP File for Test (MD_SAVE_216.157.111.6_1bfbdd69-067a-4eb5-9e77-6db821ae43b3.pcap) Download Datum Result File (https://trial-datum.metricowireless.com/AlphaDemo/Download.aspx?type=result&id=4e331f20-4867-e311-b53a- 003048fb86fb) Download Datum Test-Set File (https://trial-datum.metricowireless.com/AlphaDemo/Download.aspx?type=testset&id=f8d85c63-4e63-e311- b53a-003048fb86fb) IP Address Checks Network Address Translation (NAT) Detected: YES Source IP Address From Media Server Log: 173.8.5.209 Source IP Address From Device Log: 10.33.100.125 Device IP Address Routable: NO Local Device IP Address: 10.33.100.125 DNS Checks DNS Lookup Observed: NO FTP User-Level Throughput FTP User-Level Throughput From Media Server Log (Mbps): 4.696 FTP User-Level Throughput From Device Log (Mbps): 4.695 Time to generate report: 56.97 second(s). IP Analyses IP Fragmentation Detected: NO IP QoS Detected: YES IP QoS Not Found in Media Server IP Log IP QoS Found in Device IP Log Observed Maximum Segment Size Maximum Segment Size from Media Server (bytes): 1460 Maximum Segment Size from Mobile Device (bytes): 1380 IP Throughput Charts Data Flow Summary Media Server IP Log Data Flow Table Device Server Type Bytes TX Bytes RX Bytes bps TX bps RX bps start duration 173.8.5.209:64599 216.157.111.6:21 TCP 1,977 961 1,016 352 171 181 0.000 44.8998 173.8.5.209:64603 216.157.111.6:2150 TCP 27,913,792 27,347,134 566,658 5,027,270 4,925,215 102,055 0.256 44.4198 Device IP Log Data Flow Table Device Server Type Bytes TX Bytes RX Bytes bps TX bps RX bps start duration 10.33.100.125:64599 216.157.111.6:21 TCP 1,899 931 968 338 166 172 0.000 44.9031 10.33.100.125:64603 216.157.111.6:2150 TCP 27,983,046 27,353,646 629,400 5,040,015 4,926,654 113,361 0.259 44.4174 Example Outputs