All Questions
Tagged with xpath web-scraping
2,160
questions
0
votes
0
answers
21
views
Selenium XPATH for Google general search - how to improve results
I wrote a Python program to run a Google search for each company name in an Excel sheet. However, I get a lot more search results when manually searching the company names up on google.
I suspect it's ...
- 27
0
votes
1
answer
37
views
Extracting the text between span tags in a Javascript-rendered page using Selenium in Python
I am trying to scrape all instances of text between tags with a particular class on a web page that dynamically updates. I am using selenium with a chrome WebDriver in Python.
In a normal browser, ...
- 5
0
votes
1
answer
82
views
click() method from element object (HtmlUnit) is doing nothing
I am trying to crawl on this website: https://www.softpedia.com/get/Programming/Other-Programming-Files/Apidog.shtml using html unit.
I want to "click" on "Download now" button, it ...
-1
votes
1
answer
53
views
XPath Python Error: The 'list' object has no attribute 'xpath'
I'm brand new to Python and web scraping and cannot figure out what is wrong with my code for the life of me. Is it because I'm scraping just one element and not a list? I've checked my XPaths so many ...
2
votes
1
answer
172
views
XPath Selector in Puppeteer 22.x
I have read the newest Puppeteer v22.x documentation about XPath, still don't know how to use XPath in Puppeteer 22.x.
I want to click an element containing the text 'Next'. Here the HTML that has the ...
- 157
1
vote
1
answer
54
views
Google sheets IMPORTXML and XPath href and title not returned
I need to extract some URLS and their respective titles using Google Sheets ImportXML with Xpath.
I found these previous answers and tested them but they don't seem to be working for my case:
Extract ...
- 751
1
vote
1
answer
38
views
Selenium returns looked up element unreliably
I got stuck.. so many hours.. looked up hundreds of questions & answers here..
I want to grep data from a banks product website, e.g. "Delta" from:
https://wertpapiere.ing.de/Investieren/...
- 13
1
vote
2
answers
43
views
XPath for all text within same div tag?
<div class="o-text99">
"No "
"messages"
" found"
</div>
I want to identify all the text from the div, and class attribute value is dynamic, so ...
-1
votes
1
answer
43
views
xpath to select text preceded by specific element
I've got the following html:
<body>
<h1 id = 'example'>text</h1>
"My car is a "
<abbr>
<a href = 'exampleRef'>
Ferrari
&...
- 310
1
vote
1
answer
42
views
Xpath HTML Scraping doesn't return text / numerical - useful score
I am scraping the usefulness scores of reviews using xpath and lxml.
#%% Step 1: Import all of the extensions and packages.
from lxml import html
from urllib import request
import requests
from ...
- 25
1
vote
1
answer
44
views
Facing difficulties with the XPATH text() function
My goal is to get the price text from the page.
When I use the XPATH
//*[@id="module_product_price_1"]/div/div/span
, it finds just one item. However, when I include the text() function
//*[...
- 317
0
votes
1
answer
63
views
Selenium scraping same titles, subtitles and links from The Sun Football webpage
I'm encountering a challenge while scraping news headlines, subtitles and links from The Sun Football website using Selenium. Despite implementing seemingly correct XPaths to target the desired ...
- 15
0
votes
1
answer
299
views
Wait using Path (Puppeteer)
I'm using Puppeteer 22.6.0 with NodeJS for web scraping, I'm trying to pause the script until a specific h1 element is visible, the issue is there are multiple h1 elements on the page and the only ...
- 123
-1
votes
1
answer
24
views
Problem to get into the next page, Selenium
I'm new in using Selenium, I wanted scrap 2Gis it is analogue of GoogleMaps but I faced the problem with the clicking to the next page. What is the problem in the provided code, especially why ...
- 11
0
votes
2
answers
64
views
xPath to extract values from a specific table?
Some time ago I successfully made a database with number of cars sold in specific countries. The website I extracted data from has changed and data for multiple countries is now mixed on same page. ...
- 111