web scraping - How to use python to extract/download and web-scrape a doc.google.com/spreadsheet link found in a websites source code?

Question

Thanks for looking at my question.

When inspecting a pages source information I found a lot of data I want to retrieve. On the website's source I opened network to find a XHR/.js file with useful data, when I went to its header, I see the following information:

Request URL: https://docs.google.com/spreadsheets/d/1GJ6CvZ_mgtjdrUyo3h2dU3YvWOahbYvPHpGLgovyhtI/gviz/tq?usp=sharing&tqx=reqId%3A0
Request Method: GET
Status Code: 200 
Remote Address: 172.217.12.206:443
Referrer Policy: strict-origin-when-cross-origin

Does anyone know of any way to download this doc.google data? preferably using python and one of its libraries?

Thank you

question from:https://stackoverflow.com/questions/65925616/how-to-use-python-to-extract-download-and-web-scrape-a-doc-google-com-spreadshee

Answer 1

import requests

r = requests.get('https://docs.google.com/spreadsheets/d/1GJ6CvZ_mgtjdrUyo3h2dU3YvWOahbYvPHpGLgovyhtI/gviz/tq?usp=sharing&tqx=reqId%3A0')

with open('google_docs.txt', 'wb') as f:
    f.write(r.content)