Appending pandas Dataframe to each other inside loop

I am getting a table on the page. The first page shows 100 rows and the next page the next 100 rows and so on. I have tried this, but this doesn't seem to work, It writes to only first table. What would be the correct way so that the next rows get keep getting added to the df.

tables = driver.find_elements_by_tag_name('table')              table = tables[1].get_attribute('outerHTML')              df_i = pd.read_html(table)              df = pd.concat(df_i)              while True:                  try:                      driver.find_element_by_xpath('//a[@title="Next Page"]').click()                      time.sleep(3)                      tables = driver.find_elements_by_tag_name('table')                      table = tables[1].get_attribute('outerHTML')                      df_x = pd.read_html(table)                      df1 = pd.concat(df_x)                      df1 = df.append(df1)                  except:                      break                  df1.to_excel(f'Handloom/Handloom_{str(lofi)}.xlsx')    Current output     A | B | C  1  2  3  .  .  100  Expected Output  A | B | C  1  2  .  .  100  .  .  200  .  .  300  .  .  

https://stackoverflow.com/questions/66937668/appending-pandas-dataframe-to-each-other-inside-loop April 04, 2021 at 11:03AM