python – How Can I Get The Web-Loaded HTML of The Current Page in Selenium?

Once you invoke get() and before you extract the page_source you need to induce WebDriverWait for the visibility_of_element_located() of any of the visible element and you can use the following Locator Strategy:

WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "css_of_a-visible_element")))
html = driver.page_source

As an alternative, you can also use document.documentElement.outerHTML as follows:

WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "css_of_a-visible_element")))
html = driver.execute_script("return document.documentElement.outerHTML")


You can find a couple of relevant detailed discussions in:

Read more here: Source link