![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220216_72ppi_03_TheMagicOfMerlin-ShopifysMachineLearningPlatform_3e07a5ed-cd94-44e2-b4ca-c6db1ec81bd6.jpg?v=1678451850&width=600)
Data Science Engineering
We enable easy access to insights to make the lives of our merchants, partners and all of Shopify better.
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220216_72ppi_03_TheMagicOfMerlin-ShopifysMachineLearningPlatform_3e07a5ed-cd94-44e2-b4ca-c6db1ec81bd6.jpg?v=1678451850&width=600)
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210719_216ppi_05_FiveStepsforBuildingMachineLearning_2.jpg?v=1676298056&width=600)
ShopifyQL Notebooks: Simplifying Querying with Commerce Data Models
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_ComplexDataModelsShopifyTax.jpg?v=1675449874&width=600)
The Complex Data Models Behind Shopify's Tax Insights Feature
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_221107_72ppi_05_MonteCarlo.jpg?v=1670585039&width=600)
Monte Carlo Simulations: Separating Signal from Noise in Sampled Success Metrics
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220216_72ppi_03_7LessonsForOptimizingApacheFlinkApplications_8648a0cf-025d-4da2-bc29-424913527a6b.jpg?v=1670333665&width=600)
3 (More) Tips for Optimizing Apache Flink Applications
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_211209_72ppi_02_SHPBP3BFCMLiveMap_0329bbfa-1e7b-4b6c-9d7e-e10c5b3de6eb.jpg?v=1669749588&width=600)
Using Server Sent Events to Simplify Real-time Streaming at Scale
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210913_72ppi_04_LinNet_1.jpg?v=1669576329&width=600)
How to Export Datadog Metrics for Exploration in Jupyter Notebooks
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_221107_216ppi_02_ReducingBigQueryCostsHowAQueryAlmostCostUs1Million.jpg?v=1669215663&width=600)
Reducing BigQuery Costs: How We Fixed A $1 Million Query
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_221018_72ppi_04_HowToStructureYourDataTeamEffectively.jpg?v=1667612081&width=600)
How to Structure Your Data Team for Maximum Influence
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210719_72ppi_05_FiveStepsforBuildingMachineLearning_2474f27c-89a9-4215-98e2-69d32acd0f02.jpg?v=1662997350&width=600)
What is a Full Stack Data Scientist?
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220111_72ppi_03_ShopifysVicePresidentsPlaybookForScalingMachineLearning_83950eb5-46b1-4b9f-a0fe-cca95d12dd78.jpg?v=1661280375&width=600)
Shopify Data’s Guide To Opportunity Sizing
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220111_72ppi_04_ClassifyingBuyerMessagesWithHighAccuracyInInbox.jpg?v=1668631066&width=600)
Data-Centric Machine Learning: Building Shopify Inbox’s Message Classification Model
![Introducing ShopifyQL: Our New Commerce Data Querying Language](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210719_72ppi_05_FiveStepsforBuildingMachineLearning_2.jpg?v=1655841313&width=600)
Introducing ShopifyQL: Our New Commerce Data Querying Language
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210719_72ppi_05_FiveStepsforBuildingMachineLearning_e08b8bc6-8bab-40ff-b1a6-41035b998c5b.jpg?v=1654807134&width=600)
8 Data Conferences Shopify Data Thinks You Should Attend
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220411_72ppi_07_GenericShopifyEngineeringHeaderB.jpg?v=1653311527&width=600)
Lessons Learned From Running Apache Airflow at Scale
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220411_72ppi_06_GenericShopifyEngineeringHeaderA.jpg?v=1652905709&width=600)
Double Entry Transition Tables: How We Track State Changes At Shopify
![Data Is An Art, Not Just A Science—And Storytelling Is The Key](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220428_72ppi_01_DataIsAnArtNotJustAScienceAndStorytellingIsTheKey.jpg?v=1653421736&width=600)
Data Is An Art, Not Just A Science—And Storytelling Is The Key
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220216_72ppi_03_TheMagicOfMerlin-ShopifysMachineLearningPlatform.jpg?v=1649709642&width=600)
The Magic of Merlin: Shopify's New Machine Learning Platform
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220216_72ppi_06_ADataScientistsGuideToMeasuringProductSuccess.jpg?v=1649706738&width=600)
A Data Scientist’s Guide To Measuring Product Success
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220216_72ppi_03_7LessonsForOptimizingApacheFlinkApplications.jpg?v=1647287310&width=600)
7 Tips For Optimizing Apache Flink Applications
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_220111_72ppi_03_ShopifysVicePresidentsPlaybookForScalingMachineLearning.jpg?v=1645033295&width=600)
Shopify's Playbook for Scaling Machine Learning
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_211209_72ppi_06_HowShopifysUniqueDataScienceHierarchyOfNeedsHelpedOurMerchantsThriveThroughCovid19.jpg?v=1642531693&width=600)
Shopify’s Unique Data Science Hierarchy Of Needs
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_211026_72ppi_06_SurfacingRealTimeHighIntentBuyerSignalsEventsInShopifyInboxUsingApacheBeamKafkaAndGoogleDataflow.jpg?v=1639153879&width=600)
Building a Real-time Buyer Signal Data Pipeline for Shopify Inbox
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_211209_72ppi_02_SHPBP3BFCMLiveMap.jpg?v=1643755114&width=600)
Scaling Shopify's BFCM Live Map: An Apache Flink Redesign
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_211026_72ppi_01_UsingPropensityScoreMatchingToUncoverShopifyCapitalsEffectOnBusinessGrowth.jpg?v=1639078901&width=600)
Using Propensity Score Matching to Uncover Shopify Capital’s Effect on Business Growth
![Shopify's Path to a Faster Trino Query Execution: Custom Verification, Benchmarking, and Profiling Tooling](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_211026_72ppi_01_ShopifysPathToAFasterTrinoQueryExecutionCustomVerificationBenchmarkingAndProfilingTooling.jpg?v=1637706738&width=600)
Shopify's Path to a Faster Trino Query Execution: Custom Verification, Benchmarking, and Profiling Tooling
![Winning AI4TSP: Solving the Travelling Salesperson Problem with Self-programming Machines](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210913_72ppi_06_TeamShopifyAndFriendsWinsInternationalAICompetition.jpg?v=1634153835&width=600)
Winning AI4TSP: Solving the Travelling Salesperson Problem with Self-programming Machines
![Introducing LinNét: Using Rich Image and Text Data to Categorize Products at Scale](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210913_72ppi_04_ContinuousJustInTimeGrowthForYourCareer.jpg?v=1634153999&width=600)
Using Rich Image and Text Data to Categorize Products at Scale
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210719_72ppi_05_FiveStepsforBuildingMachineLearning.jpg?v=1631637937&width=600)
5 Steps for Building Machine Learning Models for Business
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyEng_BlogIllustrations_210719_72ppi_06_ShopifysPathtoaFasterTrinoQueryExecutionInfrastructure.jpg?v=1631638033&width=600)
Shopify's Path to a Faster Trino Query Execution: Infrastructure
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/science-beaker-and-pipette.jpg?v=1623948709&width=600)
10 Lessons Learned From Online Experiments
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/In-Context_Orders_Blogpost_Header_-_1024x538px-1.png?format=jpg&quality=90&v=1621968017&width=600)
How Shopify Built An In-Context Analytics Experience
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/Shop_Eng_Blog_Header_5.png?format=jpg&quality=90&v=1619632390&width=600)
A Five-Step Guide for Conducting Exploratory Data Analysis
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ShopifyHelpCenter.png?format=jpg&quality=90&v=1617296393&width=600)
Building Smarter Search Products: 3 Steps for Evaluating Search Algorithms
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/Untitled_design_13.png?format=jpg&quality=90&v=1615570739&width=600)
Capturing Every Change From Shopify’s Sharded Monolith
![4 Tips for Shipping Data Products Fast](https://cdn.shopify.com/s/files/1/0779/4361/articles/subway-tunnel-speed-vortex.jpg?v=1614706084&width=600)
4 Tips for Shipping Data Products Fast
![How to Make Dashboards Using a Product Thinking Approach](https://cdn.shopify.com/s/files/1/0779/4361/articles/startup-desktop.jpg?v=1611857167&width=600)
How to Make Dashboards Using a Product Thinking Approach
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/fingers-turning-dial-on-dj-board.jpg?v=1607444348&width=600)
How to Reliably Scale Your Data Platform for High Volumes
![How to Build a Production Grade Workflow with SQL Modelling](https://cdn.shopify.com/s/files/1/0779/4361/articles/modern-light-with-pipe-base.jpg?v=1605814629&width=600)
How to Build a Production Grade Workflow with SQL Modelling
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/woman-codes_44b31a7c-c584-4fdf-8b1e-081ffa798fbc.jpg?v=1602776286&width=600)
How to Build an Experiment Pipeline from Scratch
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/fferencesInDifferences.jpg?v=1601306168&width=600)
How to Use Quasi-experiments and Counterfactuals to Build Great Products
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/DatabaseLogging.jpg?v=1598479077&width=600)
How to Track State with Type 2 Dimensional Models
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/ArtifactHighLevelArchitecture.jpeg?v=1595354400&width=600)
How We’re Solving Data Discovery Challenges at Shopify
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/team-sales-business-meeting.jpg?v=1592492556&width=600)
Shopify's Data Science & Engineering Foundations
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/blog_-_warehousing_pic.jpg?v=1590083343&width=600)
7 Ways to Make Your SQL Workshop Beginner-friendly
![Categorizing Products at Scale](https://cdn.shopify.com/s/files/1/0779/4361/articles/Figure_5-_Sample_traversal_of_taxonomy_at_inference_time.jpg?v=1588269390&width=600)
Categorizing Products at Scale
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/facebook_ad_manager.png?format=jpg&quality=90&v=1583271543&width=600)
The Evolution of Kit: Automating Marketing Using Machine Learning
![](https://cdn.shopify.com/s/files/1/0779/4361/articles/woman-relaxed-work.jpg?v=1581320285&width=600)
Great Code Reviews—The Superpower Your Team Needs
![How Shopify Uses Recommender Systems to Empower Entrepreneurs](https://cdn.shopify.com/s/files/1/0779/4361/articles/startup-desktop_4460x4460_2.jpg?v=1548438144&width=600)
How Shopify Uses Recommender Systems to Empower Entrepreneurs
![How Shopify Merchants can Measure Retention](https://cdn.shopify.com/s/files/1/0779/4361/articles/business-team-meeting-boardroom_4460x4460_b6455100-218a-4058-bad1-ecbed64c6fcb.jpg?v=1510680011&width=600)