Product Experiments

Editor's Notes

1. So the more I thought about, the more I thought about Marty Cagan's post "The inconvenient truth about product" which basically says that every feature you build, even the stuff you launch is still an experiment and might fail.  Thinking that all product is an experiment got me thinking about framing my talk around that concept and the tools we use at KISSmetrics to try to succeed more than we fail on our "experiments". Let me know if that's off track from what you're getting from other panelists.
2. This is great when you’re just starting out. Sell the value proposition. It’s less valuable once you have a product, but can often turn into your marketing site. This is great when you don't have a site yet. Once you have a product, less valuable. We're launching a mobile app (see myanalyticsapp.com) and used this and have thousands of sign ups
3. Copy, color, flow, information on the page Good for: Marketing Site, on boarding.
4. We don't do much of this because we love doing the next one...
5. Validate what they need We ask our customers questions all the time
6. We'll search parts of our user base to ask questions of them (use our own product's People Search for this) We always like to have already done some CustDev interviews to better inform the questions 10 open ended questions is a total fail. Have direct discussions if you don't know what to ask
7. We don't like using these to share with customers bc most don't know how to imagine something will work in our experience
8. Share with customers or potential ones a prototype so they can see how it actually works. This helps ourselves too. We test it on ourselves
9. We have a feedback box on every page on the site. We get 50+ emails every week filled with passionate, frustrated, excited, confused users. The perfect group to experiment with. I'll go back to old ones to reach out about experiments or if they're asking about stuff we're currently thinking about I'll send it right away...they may be just the early adopter we want for such a feature.
10. Turn on a feature for a customer with the check of a box, that way you only do it for a few people to get key feedback. Allows you to start getting feedback sooner as you manage the expectations of those seeing it
11. We've been trying not to overbuild things lately and so that means often taking away as much as we can and focusing on releasing only the absolutely core problem. We've already found how that changes our priorities, while getting something out the door faster.
12. I saw this post over the weekend an thought it hammered home what I hope you take away from today. There’s a lot of great tactics you can apply, but they’re all there just to deliver on on goal.