Button Clicking Help: Python, Selenium

Question

I'm trying to web scrape an HVAC website. I want to put in a zipcode, hit the submit button, and then scrape the resulting information on the closest dealers. Here is the website if necessary.

My problem is that the zipcode I want is typed fine, but it won't submit. After the zipcode submits, I want to check a div for the text on the number of dealers, and based on that choose what to scrape into a csv.

Simplified HTML of the button:

<div class="dl-zipcode-search__block">
    <div class="zipcode-miles__wrap">
        <div class="dl-searchbtn__wrap">
            <button type="button" onclick="Search();">SEARCH</button>
        </div>
    </div>
</div>

Simplified HTML of the dealer count div with example zipcode:

<div class="dl-dealer-list__block" id="dealer-list-pagination">
        <div class="dl-dealer-count__block">
                <span>6 Dealers near <span id="searchedZip">35005</span></span>
        </div> 
</div>

I tried:

#modules here
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
import codecs
import re
from time import sleep
import csv
#importing zipcode array
from array import array
#zipcode list is 2nd file with zipcode array
from zipcode_list import zip_arr


#function that checks the dealer count div
def is_multiple_dealers(driver):
        dealer = driver.find_elements(By.CSS_SELECTOR, 'div.dl-dealer-count__block:nth-child(1) > span:nth-child(1)')
        #checking if I am finding the dealer
        print(dealer)
        #boolean that returns true or false depending on if there are 0 results, and the text
        for d in dealer:
                if '0 dealers near' in d.text:
                        return False, d.text
                        break
                else:
                        return True, d.text
                        break
                break 


#main excerpt 
def main():
        #webdriver setup code here

        wait = WebDriverWait(driver, 10)

        #webpage validation code here
        page_source = driver.page_source

        #zipcode array
        a = zip_arr

        for x in range(len(a)):    
                #zipcode form submitting code here
 
                #waiting until submit button is there then submitting
                submit_zipcode = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'div.dl-searchbtn__wrap > button')))
                submit_zipcode.click()

                #scraping with beautiful soup
                soup = BeautifulSoup(page_source,features="html.parser")
        
                dealer_boolean, dealer_quantity_text = is_multiple_dealers(driver)
                if dealer_boolean == True:
                        print(dealer_quantity_text)
                        #find data and add to array
                else
                        #add an array entry as "NAN" for no dealers in the zipcode

        driver.quit()
if __name__ == "__main__":
        main()

Since I have been able to scrape another website with a zipcode entry and submit button fine I expected it to work.

Last three traceback lines:

File "C:\Path_to_file_here\Scrape_project.py", line 80, in main
    dealer_boolean, dealer_quantity_text = is_multiple_dealers(driver)
TypeError: cannot unpack non-iterable NoneType object

The "print(dealer)" in the custom function shows up as [] so I'm guessing either my CSS Selector is wrong, or how I'm going about calling it? I also tried adding 'wait' as a parameter to the function to use 'wait.until' for the button to be visible but that didn't help either.

Answer 1

Based on the info you provided, i think you are having problem in locating the search button element, you can use the xpath //button[contains(., 'SEARCH')] to do that:

from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

timeout = 5 #time to wait for element to appar 

browser = webdriver.Firefox()

browser.get("https://www.goodmanmfg.com/support/find-a-dealer")
input_zip_code_elem = WebDriverWait(browser, timeout).until(EC.presence_of_element_located((By.XPATH, "//input[@name='zipcode']")))
search_button_elem = WebDriverWait(browser, timeout).until(EC.presence_of_element_located((By.XPATH, "//button[contains(., 'SEARCH')]")))

input_zip_code_elem.send_keys("123456")
search_button_elem.click()