Big Data, Big Changes: Data-Driven Product Development at Etsy
•
2 likes•1,664 views
This document summarizes a presentation by Dr. Jason Davis from Etsy about how they use big data and data-driven experiments to improve their products. It discusses how Etsy analyzes over 1.4 billion pageviews and $525 million in sales to conduct dozens of experiments monthly across search, advertising, and analytics. It emphasizes investing in tools for analysis, understanding tradeoffs, and appreciating the complexity of building big data products and algorithms at scale.
Report
Share
Big Data, Big Changes: Data-Driven Product Development at Etsy
- 1. Big Data, Big Changes: Data-Driven Product Development at Etsy Strata London, 2012 Dr. Jason Davis Director of Search & Data jdavis@etsy.com / @jasondavis Monday, October 1, 12
- 2. (The change) Monday, October 1, 12
- 3. 6 months from inception to launch 3 engineering teams 3 product launches Dozens of experiments Lots and lots of data analysis Monday, October 1, 12
- 4. The switch to relevancy Search, advertising, analytics, and more Building big data products at Etsy Invest in tooling Understand tradeoffs Appreciate complexity Monday, October 1, 12
- 5. Monday, October 1, 12
- 6. By the Numbers 1.4B pageviews/month $525MM sales in 2011 40MM unique visitors/month 800k shops / 150 countries Monday, October 1, 12
- 7. SELECT * FROM listings WHERE title LIKE '%silver%' ORDER BY creation_tsz DESC; Monday, October 1, 12
- 8. Solving the recency problem Monday, October 1, 12
- 9. Better buyer experience Improved search relevancy Monday, October 1, 12
- 10. Improved advertising Search Ads Monday, October 1, 12
- 11. More search insight for sellers Shop Stats Monday, October 1, 12
- 12. Building big data products Monday, October 1, 12
- 13. Build lots of tools Monday, October 1, 12
- 14. Go to the source Raw data visit logs raw query logs Descriptive statistics search depth query conversion rates Monday, October 1, 12
- 15. Invest in tooling Lunr Monday, October 1, 12
- 16. Side by side testing Monday, October 1, 12
- 17. A/B Analyzer Monday, October 1, 12
- 18. Shop Stats with Referrals Monday, October 1, 12
- 19. Understand tradeoffs (most tradeoffs don’t look like this) Monday, October 1, 12
- 20. The bike is 20% lighter! Monday, October 1, 12
- 21. Queries a, b, & c are better. Queries x & y are not. Monday, October 1, 12
- 22. ¡Hola! வணக்கம்! Guten Tag 你好 !"#$% أ*)'م ّ Hello नम#$ 今日は Monday, October 1, 12
- 23. Thinking short-term vs long-term How will the change affect behavior? Affect expectations? User incentives? Improved data? Monday, October 1, 12
- 24. Appreciate complexity Monday, October 1, 12
- 25. Vs Algorithmic Complexity Monday, October 1, 12
- 26. Success / Complexity Tradeoff 4 months 30 experiments 11 wins Monday, October 1, 12
- 27. UI June 2005 Complexity November 2006 May 2008 Today Monday, October 1, 12
- 28. Communications & Support Complexity Monday, October 1, 12
- 29. (The change) Monday, October 1, 12
- 30. Etsy - Strata NYC Continous Experimentation with Continuous Deplyoment Steve Mardenfeld Scala + Cascading = Scalding Avi Bryant Case Study: What’s a Customer Worth? Roberto Medri October 23-25 Monday, October 1, 12
- 31. Resources Code as Craft blog http://codeascraft.etsy.com/ Slides (including these) http://www.slideshare.net/etsy Github http://github.com/etsy/ Join in the fun http://www.etsy.com/careers/ Monday, October 1, 12
- 32. Monday, October 1, 12