When trying to scrape the county data from multiple Politico state web pages, such as this one, I concluded the best method was to first click the button that expands the county list before grabbing the table body's data (when present). However, my attempt at clicking the button had failed:
from bs4 import BeautifulSoup as bs import requests from splinter import Browser state_page_url = "https://www.politico.com/2020-election/results/washington/" executable_path = {'executable_path': 'chrome-driver/chromedriver.exe'} browser = Browser('chrome', **executable_path, headless=False) browser.visit(state_page_url) state_soup = bs(browser.html, 'html.parser') reveal_button = state_soup.find('button', class_='jsx-3713440361') if (reveal_button == None): # Steps to take when the button isn't present # ... else: reveal_button.click()
The error returned when following the else-condition is for my click()
call: "TypeError: NoneType object is not callable". This doesn't make sense to me since I thought that the if-statement implied the reveal_button
was not a NoneType. Am I misinterpeting the error message, how the reveal_button
was set or am I misinterpeting what I'm working with after making state_soup
?
没有评论:
发表评论