Extract Contents of iframe for an element using Beautiful Soup

My goal: scrape the data from a table on a webpage

What it looks like when you first inspect the source:

What it looks like when you expand the body under id = "AppContainer":

My code, less the lines I have used to try to get the data:

#################################  #variables  target_url = ['https://redacted_url']  page = ''      #################################  #download the page  page = requests.get(target_url[0])  #check if the download of the page was successful; 200 = success 403 = forbidden  page.status_code      #################################  #parse the page you downloaded  my_soup = BeautifulSoup(page.content,'html.parser')    

What I have tried:

1. Navigating to the network tab in the source to find the URL of the iframe; result: no url with the data
2. my_soup.find_all("iframe"); result: when assigned to a variable the variable value was None
3. my_soup.find("div",{"id": "AppContainer"}); result:

<div id="AppContainer">  <!-- App rendered here -->  </div>  

4. Tried getting to page with table using selenium, but I have a similar situation in that href to the page I want to go to is in iframe:

Result: error message:

Message: no such element: Unable to locate element: {"method":"link text","selector":"#/classic/ML0A1ZN"}    (Session info: chrome=87.0.4280.88)  

The code I am using:

#################################  import requests  from bs4 import BeautifulSoup  from selenium import webdriver  from selenium.webdriver.common.keys import Keys   import time    #################################  #variables  try:      target_url = ['https://app-aba.redacted_appname.com/']  except Exception as e:      print(e)  page = ''      #################################  # initiating the webdriver and logging into application. Parameter includes the path of the webdriver.   try:      driver = webdriver.Chrome("C:\\Users\\name.name\\Desktop\\chrome_driver\\chromedriver_win32\\chromedriver.exe")        driver.get(target_url[0])      driver.find_element_by_id("loginUsername").send_keys("email@company.com")      driver.find_element_by_id("loginPassword").send_keys("password")      driver.find_element_by_id("loginButton").click()      time.sleep(60)      driver.find_element_by_link_text("#/classic/ML0A1ZN").click()  except Exception as e:      print(e)    

Any help would be much appreciated, thanks!

https://stackoverflow.com/questions/65571508/extract-contents-of-iframe-for-an-element-using-beautiful-soup January 05, 2021 at 08:04AM