MnSearch Snippets April 2019: Screaming Frog Custom Extraction - Griffin Roer
•Download as PPTX, PDF•
0 likes•495 views
Screaming Frog scrapes a lot of critical information by default; page titles, H1 elements, canonical tags, etc. But what if you want to pull other data points into your site crawls? With Custom Extraction, you can program Screaming Frog to scrape just about any information you want. Once you get a handle on how to use it, you can conduct more advanced site crawls and analysis.
Report
Share
MnSearch Snippets April 2019: Screaming Frog Custom Extraction - Griffin Roer
- 1. Presented by SCREAMING FROG CUSTOM EXTRACTION April 24, 2019
- 2. “The industry leading website crawler… trusted by thousands of SEOs and agencies worldwide for technical SEO audits.” Screaming Frog MORE THAN AN SEO AUDIT TOOL ● Custom website scraping ● Advanced reporting ○ GA & GSC integrations 2Screaming Frog Custom Extraction - MnSearch - April 2019
- 3. CUSTOM EXTRACTION 101 Scrape any data from the HTML of a web page. 3
- 4. DEFAULT EXTRACTIONS Screaming Frog extracts a bunch of data from the HTML of web pages by default. 4Screaming Frog Custom Extraction - MnSearch - April 2019 Page Title Meta Description Meta Keywords H1 H2 Meta Robots Meta Refresh Canonical Link Pagination On-page links Anchor text Alt text Hreflang AMP
- 5. CUSTOM EXTRACTIONS Examples: ● Publish date ● Product reviews ● Article comments ● Schema markup ● Pixel ID’s 5Screaming Frog Custom Extraction - MnSearch - April 2019
- 6. HOW THEY HELP Use cases: ● Analyze performance by factors you may normally not have access to ● Diagnose hidden site issues ● Speed up data-gathering 6Screaming Frog Custom Extraction - MnSearch - April 2019
- 7. HOW TO USE CUSTOM EXTRACTION 7
- 8. 8 ACCESS CUSTOM EXTRACTION Go to Configuration > Custom > Extraction.
- 9. 9 SET UP YOUR EXTRACTION RULES More on this in a minute.
- 10. 10 RUN YOUR CRAWL The data you extract is available in the Custom tab. Set the Filter dropdown to Extraction.
- 11. EXTRACTION RULES How to tell Screaming Frog what you want it to extract. 11
- 12. 12 SET UP YOUR EXTRACTION RULES Screaming Frog requires some information to know how and what to extract: ● Extractor Name (optional) ● Extraction Method ● Rule ● Extraction Filter
- 13. ● Use to extract any HTML element of a webpage ○ Anything in a <div>, <p>, <span>, <a>, <meta>, etc. 13 XPATH & REGEX Two syntaxes that you can use to tell Screaming Frog what you want to extract from a web page. XPATH ● Use to extract inline JavaScript ○ Like, schema markup in JSON-LD or a an account ID from a tracking pixel REGEX
- 14. bit.ly/custom-extraction 14 EXTRACTION RULES YOU CAN COPY! Check out this guide for extraction rules you can copy + paste into Screaming Frog, plus learn to write your own.
- 15. 15 EXAMPLES A quick example showing how to extract the date from articles on the MnSearch blog. Chrome: Right-Click > Inspect Date is in a <span> element with the class “meta-date date updated”
- 16. 16 EXAMPLES A quick example showing how to extract the date from articles on the MnSearch blog. Custom Extraction Results XPath Rule: //span[@class='meta-date date updated']
- 17. 17 EXAMPLES Extract information about each article on a blog.
- 18. 18 EXAMPLES Extract product information from the schema markup.
- 19. 19 EXAMPLES See what types of schema markup are being used on each page. ● This example shows one rule using XPath and another using regex
- 20. bit.ly/custom-extraction 20 ONE MORE TIME... Check out this guide for extraction rules you can copy + paste into Screaming Frog, plus learn to write your own.
- 21. THANK YOU