python – selenium webdriver connection problems (not connected to DevTools)
I am running a Selenium webdriver program that needs a lot of time to complete. My problem is that, for some reason, whenever I run this script as a headless browser, I end up having issues. For example, if I use Firefox driver I get this after a short time:
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host="localhost", port=58161): Max retries exceeded with url: /session/1aa9d459-8770-41ec-a25b-a1a9be8f7d2b/elements (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1817268d0>: Failed to establish a new connection: [Errno 61] Connection refused'))
If I use Chrome’s webdriver, the script does run much longer, but I always end up encountering this error:
disconnected: Unable to receive message from renderer
(failed to check if window was closed: disconnected: not connected to DevTools)
As for the second error, I have seen this can happen due to having too many elements to load in a page, which is not the case, as the URLs where the driver stops are pages that in previous iterations have been loaded without any issue. I have also read this may happen because of not having interaction with the page for a long amount of time, but that is not the case either, as I am interacting with the page continuously (by typing on fields or clicking on elements).
I have tried to catch the WebDriverException and restart the driver by doing the following:
print(" ........ Restarting webdriver ...... ")
current_url = driver.current_url
driver.quit()
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--auto-show-cursor')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('chromedriver',options=options)
driver.get(current_url)
Nevertheless, even if I do catch the exception, this does not prevent the script from failing and finishing the execution abruptly. Is there any way in which I can reconnect when something like this happens so that my script continues with the execution? I am using python, by the way.
Read more here: Source link
