How to select all table elements inside a div parent node with BeautifulSoup?

I am trying to select all table elements from a div parent node by using a customized function.

This is what I've got so far:

import BeautifulSoup  import requests  import lxml    url = 'https://www.salario.com.br/profissao/abacaxicultor-cbo-612510'    def getTables(url):        url = requests.get(url)      soup=BeautifulSoup(url.text, 'lxml')        div_component = soup.find('div', attrs={'class':'td-post-content'})      tables = div_component.find_all('table', attrs={'class':'listas'})        return tables  

However when applied as getTables(url) the output is an empty list [].

I expect this function to return all html tables elements inside div node given specific his specific attributes.

How could I adjust this function?

Is there any other library I could use to accomplish this task?

https://stackoverflow.com/questions/66523174/how-to-select-all-table-elements-inside-a-div-parent-node-with-beautifulsoup March 08, 2021 at 09:10AM