Building an A/B Testing Analytics System with R and Shiny

User story mapping involves organizing user stories and tasks on a board to help plan and prioritize work. It is not the same as defining tasks, use cases, documents, or big stories that describe large amounts of work. Effective user story mapping divides stories into logical groups called "user activities" and smaller detailed tasks, then arranges them on a board from left to right in order of time. The mapped stories are then broken into iterative releases to guide incremental development.

Join us as we hear Ramkumar Ravichandran, the Director of A/B Testing at Visa Checkout, explain how he created a high impact experimentation program. Ram will take us through the growth of Visa’s program: from selling the value, to laying down the vision, the roadmap and success criteria, to creating the right team and driving engagement with the program. Attend this webinar to learn: -How an experimentation program drives business impact. -A model to drive continuous stakeholder engagement with the program. -How to build a roadmap that goes above and beyond simple UX optimization.

ChatGPT is a conversational AI model developed by OpenAI that is capable of generating human-like text responses. It is trained on a large dataset of human conversations which allows it to have natural, coherent conversations on a wide range of topics. Key features include generating long-form paragraphs, handling diverse inputs, and maintaining context across multiple conversation turns. While not designed for SEO, ChatGPT could potentially be used to generate website content, but may not optimize for search engines and could harm a site's rankings.

The document discusses a presentation on using user story mapping to build better products. The presentation aims to teach how to use a user story backlog to describe a user's experience with a product. It covers mapping user stories based on user experience, planning valuable incremental releases from the story map, and iteratively constructing software. The presentation discusses starting with user stories, mapping them based on tasks and activities, and slicing the story map into valuable product releases.

Learn to use the user story backlog as a way to describe user’s experience with your product. Section 1: Importance of Product Owners Roll.  Identifying Scrum’s Product Owners roll.  Diagrammatic representation of PO Activities.  Diagrammatic representation of Product Feature Development tracks. Section 2: User stories & Product Backlog Management.  Agile User Stories overview .  Acceptance Criteria.  Backlog Management. Section 3: Project Scope, Product Backlog and Story Mapping.  User Story Mapping Steps.  Story Mapping example with valuable releases.  Benefits of User Story Mapping.

This document summarizes Matt Philip's presentation on NoEstimates forecasting with less effort and more accuracy. Some key points from the presentation include: reducing sources of variation can improve forecasting more than improving estimating skills; data and probabilistic forecasts are preferable to intuition and deterministic estimates; and focusing on delivering value quickly can reduce the need for detailed estimates. The presentation provided examples of techniques to reduce variation, biases to consider, and recommendations for incorporating probabilistic forecasting into projects.

This document discusses best practices for effective product ownership in an agile development process. It covers the primary responsibilities of a product owner including managing the product backlog, prioritizing features, and communicating with stakeholders. The document emphasizes techniques for writing user stories and breaking down features into independently deliverable increments. It also provides methods for assessing the relative importance of different features and themes to guide prioritization of development work.

The document discusses strategies for startups to build on top of large platforms to gain users and grow rapidly. It notes that while platforms are not there to specifically help startups, they can be beneficial for acceleration if used correctly. The key strategies are to 1) build products that provide clear value to users and incentives for the platform, 2) design products to work independently of platforms since platforms will change, and 3) use platforms for user acquisition and acceleration only, not as a primary distribution method or business model. Successful examples include Instagram, YouTube, and Dubsmash who leveraged platforms initially but became independent destinations.

The document provides an overview of agile product management and scrum. It discusses key concepts like lean, agile, scrum roles and artifacts, ceremonies like sprints and planning, and topics like minimum viable products, user stories, prioritization techniques, and product backlog refinement. The document is a training guide or presentation on agile product management best practices.

This document discusses methodologies for defining features, epics, and user stories to estimate the cost of an IT project. It defines key terms: Feature refers to a system's capabilities, like capturing data or booking a hotel. Epics are large user stories. User stories describe features used in a specific context, like using loyalty points to book a hotel. The document recommends using a Story Map to visualize features, epics, and user stories. Features represent macro processes, while epics and user stories are represented as post-its within those processes. To estimate costs, the main features and architectural constraints should be identified, not the entire backlog of epics and user stories, which provides too much

As a product manager, how do you balance dozens of feature requests from countless teams in your organization? Without a mechanism in place to keep track of the noise, prioritization is nearly impossible. But fear not! Here are 7 time-tested prioritization techniques for product managers.

A brief overview of the concept of personas, storymaps and answering a key question of when can I get my Minimum Viable Product

User Story Mapping (USM) helps teams get a common understanding of requirements from the user's perspective to facilitate backlog creation. It improves backlog quality and team communication. USM creates a map with user stories arranged in a usage flow. Each story follows the "As a <user>, I want <goal> so that <benefit>" format. Together, the mapped stories provide an overview of a product from the user experience while maintaining granular stories for planning and testing.

This presentation features the POKER-Scorecard which is a shared language and platform for presenting and applying any business tool especially those used in Silicon Valley.

This document discusses A/B testing at large internet companies. It describes how companies like Amazon, Microsoft, Google, and LinkedIn use A/B testing to evaluate new ideas, measure their impact, and gain customer feedback. It outlines best practices for A/B testing, such as running one experiment at a time, choosing appropriate metrics and statistical significance, properly powering experiments, and addressing issues like multiple testing. The document also describes the key components of a scalable A/B testing system, including experiment management, online infrastructure for traffic routing and data logging, and automated offline analysis.

User Story Mapping is an approach for organizing and prioritizing user stories to plan valuable incremental product releases. It addresses problems with flat backlogs by showing features from the user's perspective and understanding the big picture. The process involves brainstorming major user tasks, grouping them from left to right in order of completion, then adding detailed user stories which are prioritized and broken into releases. Combining Impact Mapping with User Story Mapping allows prioritizing deliverables to achieve user goals through understanding individual scenarios and planning iterative delivery.

This document summarizes key points from a Lean Analytics conference presentation. It discusses lean startup principles like iterating based on data and customer feedback rather than following a predefined plan. It provides examples of how startup ideas and business models can change based on learning. Metrics that matter at different stages are discussed, like activation rate for stickiness and viral coefficient for growth. The importance of focusing on one key metric at a time is emphasized. Baselines for growth rates, engagement, and churn are provided as guidelines for startups.

Everyone fails - but not everyone uses failed attempts as a source of learning and improvement. This talk outlines a framework to turn failure into gaining knowledge by understanding IF, HOW and WHY something fails.

1. The document discusses testing solutions to improve a software product by defining success criteria, building a list of solutions, setting up tests, and gathering/interpreting results. 2. It emphasizes defining measurable success criteria, avoiding subjective terms. Multiple criteria from different angles are recommended. 3. Tests should target relevant user groups to avoid results pollution and confirm the proposed solution provides value before significant investment.

This document provides guidance on conducting a usability study. It outlines the schedule, expectations for participants, and guidelines for running test sessions. Participants will be divided into groups of 3, with one person acting as the user and two observing. As a user, they will complete tasks while thinking aloud, and observers will take notes on successes, obstacles, and areas of confusion. After each task the group will switch roles. The goal is to evaluate the design and identify ways to improve the user experience.

This document provides guidelines for A/B testing, including prioritizing test ideas based on estimated new conversions per day, creating tests by running a power analysis and having incremental tests, analyzing tests by monitoring health metrics, and making decisions carefully based on analysis results. It recommends calculating potential impact, having a data scientist involved, and not launching on neutral results to avoid technical debt.

You’ve built a shiny, new WordPress site. You asked your co-worker and your boss if they like it and they both do. However, you’re lying awake at night wondering if you’re missing something—because you know you’re not the end user. You yearn for actionable feedback. In this talk, I’ll distill my background in usability research into a how-to framework for taking your site and conducting your first unmoderated usability test. I’ll cover why and when you should be running usability tests; how to set research goals and draft a script for them; setting up your lab environment and capturing feedback; and best practices for facilitation, minimizing bias, keeping users on task and gleaning the most from each brief test.

This document provides an overview of data science. It discusses that data science involves taking large amounts of information and turning it into something valuable. It then discusses the main components of data science: data wrangling, analytics, and predictions. It also profiles four common types of data scientists: researchers, AI gurus, statisticians, and super analysts. It describes their typical work environments, job responsibilities, and education/skills. Finally, it discusses options for learning data science, including a free two-week trial program from Thinkful.

You’ve built a shiny, new Drupal site. You asked your grandma and your client if they like it and they both do. However, you’re lying awake at night wondering if you’re missing something—because you know you’re not the end user. You yearn for actionable feedback. In this talk, I’ll distill my background in usability research into a how-to framework for taking your site and conducting your first unmoderated usability test. I’ll cover what to look for, best practices in facilitation, tools on the cheap, and how to glean the most from a brief window of time.

This document provides information about adding usability testing to your skill set. It discusses why usability testing is important, what rapid usability testing entails, different testing methods, and the preparation and process for conducting an unmoderated usability study. Key steps include defining goals, preparing materials like tasks and recruitment, setting up the test environment, observing and analyzing participant videos, and reporting findings to stakeholders. Conducting even a small number of tests can uncover major usability issues.

Usability testing involves planning studies to test a digital product. Key steps in planning include defining goals and participants, designing tasks, scheduling tests, and determining testing methods. Tests can be conducted remotely or in-person. Moderated tests involve a moderator guiding participants through tasks while they think aloud. Unmoderated tests use automated tools to gather metrics from participants remotely. Findings are analyzed to identify usability issues and improve the product's design. Mobile testing requires adaptations for its form factor. Fitting research into agile development requires parallel or staggered sprints.

1. The document outlines the steps to conduct a usability test, including deciding what to test, designing the test, conducting the test, and writing up the test findings. 2. Key steps in designing the test involve identifying users and test tasks, developing metrics to evaluate usability, and creating a data collection form. 3. Conducting the test involves playing the role of a new user, completing each task while collecting data, and taking screenshots to support findings.

This document provides guidance on how to conduct rapid usability testing with minimal resources and quick turnaround times. It discusses preparing for a study by determining goals, participants, tasks and questions. Methods covered include moderated tests, unmoderated tests using tools like Treejack and UserTesting.com. The document also outlines conducting tests, analyzing results, and debriefing stakeholders. The goal is to identify usability issues and improve interfaces through inexpensive, lightweight testing.

The document provides guidance on developing an effective presentation submission for a conference. It recommends choosing a topic that you are knowledgeable about, such as something you recently learned, have been discussing, or wish you had known earlier. The document also suggests researching the conference audience and call for proposals to determine the appropriate level of detail. When crafting the proposal, the document advises including an outline that identifies the problem, solution, and key takeaways. Structuring the presentation in this way helps the audience understand how they can apply what they learn.

Introduction to usability and usability testing as a discipline, followed by how to do guerilla usability testing. Presented at Duke Tech Expo April 13, 2018 with co-author Lauren Hirsh, with content from a prior collaborative presentation of hers.