SearchLove Boston 2016 | Paul Shapiro | How to Automate Your Keyword Research
•Download as PPTX, PDF•
47 likes•72,544 views
Are you tapping into automation for keyword research? If not, why not? When it comes to SEO, automation is awesome. For starters, it can help free up a lot of time that is normally spent on menial tasks. What’s more, it can also aid deep analysis, and even facilitate innovation. If you are still doing keyword research manually, this is a must-attend session. Paul will show you how to get started with automated keyword research, using some easy-to-use tools. You’ll see first-hand how they can help you uncover valuable insights automatically. Overall, you will walk away with an immediately actionable plan to start automating your keyword research today.
Report
Share
More Related Content
What's hot
What's hot (20)
Similar to SearchLove Boston 2016 | Paul Shapiro | How to Automate Your Keyword Research
Similar to SearchLove Boston 2016 | Paul Shapiro | How to Automate Your Keyword Research (20)
More from Distilled
More from Distilled (20)
Recently uploaded
Recently uploaded (20)
SearchLove Boston 2016 | Paul Shapiro | How to Automate Your Keyword Research
- 1. HOW TO AUTOMATE YOUR KEYWORD RESEARCH Paul Shapiro – Organic Search Director at Catalyst SearchLove Boston – May 4th, 2016
- 5. @fighto 5 Tool Data Sources Visualization + +
- 6. @fighto WHAT YOU WILL LEARN • The problems with the standard keyword research process • How you can save time on keyword research with automation • How to automate keyword research with a powerful, but easy-to-use tool • How APIs work and how to leverage them • What data sources are useful for keyword research automation • How to use data visualization to make keyword research decisions 6
- 7. MANUAL KEYWORD RESEARCH IS OFTEN BLAND & TIME CONSUMING 7
- 8. @fighto 8
- 9. @fighto 9 Most Keyword Research Looks Like This
- 10. @fighto 10 That’s just a Google Keyword Planner Export
- 11. @fighto TYPICAL TIME INVESTMENT FOR KEYWORD RESEARCH 5 8 12 21 21 6 10 18 22 28 0 5 10 15 20 25 30 Micro (0-49 pages) Small (50-99 pages) Medium (100-249 pages) Large (250-499 pages) Extra Large (>500 pages) Hours to Complete Keyword Research by Site Size Average (Low End) Average (High End) 11
- 12. @fighto 12 AIN’T NOBODY GOT TIME FOR MANUAL KEYWORD RESEARCH!
- 13. @fighto SIZE OF THE DATA SET 13 vs.
- 14. @fighto 14 We can do better!
- 15. WE CAN SAVE TIME WITH AUTOMATION
- 16. @fighto TYPICAL TIME INVESTMENT FOR KEYWORD RESEARCH 5 8 12 21 21 6 10 18 22 28 0 5 10 15 20 25 30 Micro (0-49 pages) Small (50-99 pages) Medium (100-249 pages) Large (250-499 pages) Extra Large (>500 pages) Hours to Complete Keyword Research by Site Size Average (Low End) Average (High End) 16
- 17. @fighto 17 If we do keyword research once per month, we will save around 24 to 45 full days over a 5-year period using automation. http://xkcd.com/1205/
- 18. WE CAN DO BETTER THAN AN EXCEL SPREADSHEET
- 19. @fighto THAT’S MORE LIKE IT! 19 h/t @victorpan
- 20. @fighto HOW TO USE FOR KEYWORD SELECTION / TARGETING 20 This top-right quadrant contains keywords with: • Low competition • Good growth Larger bubbles show higher search volumes. You can alternatively use current rank on the x-axis to signal organic market share like a traditional growth-share matrix.
- 21. @fighto 21 That’s much better! actionability.
- 22. MY KEYWORD RESEARCH AUTOMATION PROCESS
- 23. @fighto 23 https://www.knime.org It starts with a tool…
- 24. @fighto WHY KNIME? • Singular tool for data collection, manipulation, analysis, and reporting • Easy to use and no development skills necessary • If you are a developer, it provides faster prototyping • Large library of built-in “nodes” • Free/Open Source • Run on Windows/Mac/Linux • Infinitely expandable 24
- 25. THE BASICS OF KNIME
- 26. @fighto WHAT’S A NODE? • Nodes are prebuilt, drag and drop modules designed perform a singular task • Nodes are strung together like a chain to accomplish larger, more complex tasks • Nodes can be grouped together into “meta-nodes”, which can be configured in unison 26
- 27. @fighto HOW DO YOU ADD NODES & HOW DO THEY CONNECT? How do you add nodes to your “workflow”? How do you string nodes together? 27
- 28. @fighto HOW DO YOU CONFIGURE & RUN NODES? Configuring Nodes Running Workflows 28 OR
- 29. APIs 101 29
- 30. @fighto EASY & APPLICABLE API: GOOGLE AUTOSUGGEST http://suggestqueries.google.com/complete/search?output=toolbar&hl=en&q=ninja%20turtles 30 Simple API using “GET Request” This is a variable that changes
- 31. @fighto 31 Output = XML
- 33. @fighto STEP 1: LIST OF KEYWORDS 33
- 34. @fighto STEP 2: ENCODE SPACES IN KEYWORD PHRASES 34 Space Character
- 35. @fighto STEP 3: CONCATENATE API URL + KEYWORDS 35
- 36. @fighto STEP 4: DOWNLOAD API OUTPUT 36
- 37. @fighto STEP 6: LOOP THROUGH EACH KEYWORD 37
- 38. @fighto STEP 7: PARSE THE XML OUTPUT FORM THE API 38
- 40. @fighto 40
- 41. @fighto FYI: I’M NOT A KNIME SALESMAN 41
- 42. DATA INPUTS / SOURCES
- 43. @fighto 43 # of primary data sources to aggregate
- 44. @fighto 44 http://searchwilderness.com/gwmt-data-python/ Google Search Analytics – Historical MySQL Database via Python Script
- 45. @fighto 45 AdWords - Search Query Report API or CSV
- 46. @fighto 46 SEMRush – API (if <100,000) or Domain vs. Domain: Competitor Keywords (Advanced Mode) https://www.semrush.com/info/domain_vs_domain
- 47. @fighto 47 GrepWords API – For Google Keyword Planner related query suggestions & search volumes http://grepwords.com/
- 48. @fighto 48 Google Autosuggest API or Emulation with Selenium
- 49. @fighto 49 Rank Checking/Full SERP API - GetSTAT https://getstat.com/
- 50. @fighto 50 MarketMuse – Semantic Keyword Recommendations https://www.marketmuse.com/ Alternative: http://pshapi.ro/SemanticKWR
- 51. @fighto 51 Moz API + Moz Nodes https://tech.knime.org/book/mmi-labs-nodes https://moz.com/products/mozscape/access
- 52. @fighto 52 Google Trends – Unofficial API https://github.com/GeneralMills/pytrends Script soon to follow: http://pshapi.ro/slopetrends
- 53. @fighto 53 More data sources: http://pshapi.ro/kwmetrics
- 55. @fighto Filtering + Data Manipula- tions ONE BIG KEYWORD LIST Seed Keywords - List - GWMT - SEMRush Comp. KWs - SQR Keywords Keyword Planner Suggestions (via GrepWords) Google Autocomplete Semantic Keyword Recommendations (via MarketMuse) Google Autocomplete SEMRush Domain vs. Domain Keywords Google Autocomplete 55
- 56. @fighto STRING ‘EM ALL TOGETHER AND THEN… 56
- 57. @fighto DATA MANIPULATIONS / CALCULATIONS • Get top 10 results from rank checking API (i.e., GetSTAT) • Use Moz API nodes and find average PA to assess competiveness. • Optionally, use SEMRush’s Keyword Difficulty API Organic Competition Search Volume Keyword Trends 57
- 58. @fighto DATA MANIPULATIONS / CALCULATIONS • Get top 10 results from rank checking API (i.e., GetSTAT) • Use Moz API nodes and find average PA to assess competiveness. • Optionally, use SEMRush’s Keyword Difficulty API Organic Competition • Get Search Volumes via SEMRush API or via GrepWords APISearch Volume Keyword Trends 58
- 59. @fighto DATA MANIPULATIONS / CALCULATIONS • Get top 10 results from rank checking API (i.e., GetSTAT) • Use Moz API nodes and find average PA to assess competiveness. • Optionally, use SEMRush’s Keyword Difficulty API Organic Competition • Get Search Volumes via SEMRush API or via GrepWords APISearch Volume • Use 2 Years of Google Trends data to calculate slope and determine growing/declining keywordsKeyword Trends 59
- 60. @fighto 60
- 62. @fighto 62 You can use this data to filter keywords and reduce massive list... OR, you can leverage data visualization!
- 64. @fighto 64 We have a few options…
- 65. @fighto OPTION: EXCEL + BUBBLE CHART 65
- 67. @fighto OPTION: PYTHON + BOKEH CHARTS IN KNIME 67
- 69. @fighto 69 Sample Python Code: http://pshapi.ro/SLBokeh
- 70. @fighto SO WHAT… • Free up time spend doing mundane, repetitive work and spend more doing impactful, strategic work • Free up staffing resources and utilize them in other ways • Extract and analyze a larger pool of data for greater data pool 70 From this… To this…
- 71. @fighto SUMMARY • You should rethink how your currently do keyword research. • Keyword research is often bland, time consuming, and not as actionable as it can be. • KNIME provides an easy-to-use interface for automating your keyword research • APIs are easier to use than they sound; Use them • There are great data sources you can leverage via APIs/built-in KNIME nodes • Visualize your keywords using a growth-share matrix like-graph to make keyword usage decisions • Emojis are awesome 71
- 72. @fighto RESOURCES: DATA SOURCES > KNIME 72 Data Source KNIME Node(s) Google Search Analytics Database Table Connector/Reader AdWords Search Query Report XLS Reader SEMRush Domain vs. Domain (or API if less than 100,000 keyword) XLS/CSV Reader or GET Resource & Read REST Representation Google Keyword Planner/GrepWords GET Resource & Read REST Representation Google Autosuggest GET Resource & Read REST Representation (or Selenium nodes) Rank Checking (via GetSTAT) GET Resource & Read REST Representation Semantic Keywords/MarketMuse POST Resource MOZ Moz API Connector/URL Metrics nodes Google Trends TBD – Either CSV or Python Script nodes
- 73. @fighto ADDITIONAL RESOURCES • KNIME Version to install: “KNIME Analytics Platform + all free extensions” • To install KNIME Add-ons, Choose Help->Install New Software • Moz nodes • Should come with Nightly Builds. If not can add http://update.knime.org/community- contributions/3.1 as a repository. • Add-on Selenium nodes • http://seleniumnodes.com/ • Search Engine Land article addressing additional data inputs: http://pshapi.ro/kwmetrics • Information about Growth-share matrices: https://en.wikipedia.org/wiki/Growth%E2%80%93share_matrix • Sanple Python + Bokeh script for an interactive keyword research visualization in KNIME: http://pshapi.ro/SLBokeh 73
Editor's Notes
- a CAR need GASOLINE before you SEE IT MOVE
- Focus on medium size-sites
- Average for this year vs last, over time
- More time to do more impactful work Make better use of staff More data means you’re more likely to find gems others wouldn’t have uncovered.