python - Bookmakers scraping with selenium

Question

I'm trying do understand how to scrape this betting website https://www.betaland.it/

I'm trying to scrape all the table rows that have inside the information of the 1X2 odds of the italian "Serie A".

The code I have written is this:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.expected_conditions import presence_of_element_located
import time
import sys

url = 'https://www.betaland.it/sport/calcio/italia/serie-a-OIA-scommesse-sportive-online'

# absolute path
chrome_driver_path = '/Users/39340/PycharmProjects/pythonProject/chromedriver'

chrome_options = Options()
chrome_options.add_argument('--headless')

webdriver = webdriver.Chrome(
  executable_path=chrome_driver_path, options=chrome_options
)

with webdriver as driver:
    #timeout
    wait = WebDriverWait(driver, 10)

    #retrieve the data
    driver.get(url)


    #wait
    wait.until(presence_of_element_located((By.ID, 'prematch-container-events-1-33')))

    #results
    results = driver.find_elements_by_class_name('simple-row')

    print(results)

    for quote in results:
        quoteArr = quote.text
        print(quoteArr)
        print()

    driver.close()

And the error that I have is:

Traceback (most recent call last):
  File "C:Users39340PycharmProjectspythonProjectmain.py", line 41, in <module>
    wait.until(presence_of_element_located((By.ID, 'prematch-container-events-1-33')))
  File "C:Users39340PycharmProjectspythonProjectvenvlibsite-packagesseleniumwebdriversupportwait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: 

P.S: if you try to access to the bookmakers you have to set an Italian IP address. Italian bookmakers are avaible only from Italy.

question from:https://stackoverflow.com/questions/65873734/bookmakers-scraping-with-selenium

Answer 1

It's basically a timeout error which means the given time to load the page or find the element(as in this case) is insufficient. So firstly try to increase the wait time from 10 to 15 or 30 even.

Secondly you can use other element identifiers like xpath, css_selector and others instead of id and adjust the wait time like said in point one.