Selenium Python3, how to click on a div to fully load the html?

I am trying to use Selenium and BeautifulSoup to extract some information from https://superbet.ro/pariuri-sportive/live. I created the urls for the live matches, and now I'm iterating through them to extract some statistics. But STASTISTICS TAB is not loading when I use this code:

def get_soup(url):      options = Options()      options.add_argument('--headless')      options.add_argument('--disable-gpu')        driver = webdriver.Chrome(options=options)      driver.get(url)        time.sleep(3)           page = driver.execute_script('return document.body.innerHTML')      driver.quit()      soup = BeautifulSoup(page, 'html.parser')      print(soup)      return soup  

So I'm trying to click the Statistics button to find the divs I need, because the html obtained in my script is partially loaded and different than the original one from the chrome developer tools.

Here are the difference between what I get and what I need:

<div class="statistics__content">  <div class="sa-sdk-v5">  <div class="sa-sdk-unknown-tab" eventdetails="[object Object]">  

Here are the divs I need

I don't know exactly how to click on Statistics because I don't have any button tag.

Here are the tabs

Finally, I solved the problem by clicking on that tab.

https://stackoverflow.com/questions/65942295/selenium-python3-how-to-click-on-a-div-to-fully-load-the-html January 29, 2021 at 01:32AM