How to avoid StaleElementReferenceException error in python selenium

Question

There are tons of questions about this error however none of them could be able to help my case.

First i get the main url's ids that i need

url = 'http://www.mosquedirectory.co.uk/browse/uk/england/london'

browser = webdriver.Chrome()
browser.get(url)

listing = browser.find_elements_by_id('directory_listingBrowse')

and I append them in to list to avoid from the error but it is not even worked

hold = []

for i in listing:
    hold.append(i)

and from these hold list I loop in for loop and this is also the rest of the code

for i in hold:
     
    try: ulclass = i.find_elements_by_css_selector('ul.c')

    except StaleElementReferenceException:
        pass
        
    link = []
    for i in ulclass:
        a = i.find_element_by_tag_name('a')
        link.append(a.get_attribute('href'))
        
    for i in link:
        browser.get(i)

    browser.get(url)
    time.sleep(2)

I even tried to avoid the error by using try method, not worked again. And at the end of the code I say get back to old page, again to avoid the error. It's not even worked again. Which part am I missing.

question from:https://stackoverflow.com/questions/66062555/how-to-avoid-staleelementreferenceexception-error-in-python-selenium

Answer 1

Slight change of approach. Collect all links first, then process them as you wish.

url = 'http://www.mosquedirectory.co.uk/browse/uk/england/london'

browser = webdriver.Chrome()
browser.get(url)

listings = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//div[@id='directory_listingBrowse']//ul//*//a[@href]")))

all_links = [listing.get_attribute('href') for listing in listings]

for link in all_links:
    browser.get(link)
    #do whatever else here

Don't forget to add these imports:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait