Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
312 views
in Technique[技术] by (71.8m points)

http - Why requests.post() does not use a provided proxy in python?

I am trying to send a post request with a list of http proxies. For a moment I though that the proxies where working fine, until I add a fake proxy to the list to make sure that everything is working like this:


def func_name(i):
  url = 'https://www.some-url.com/Cart/ajax/page.php'

  # Proxies to connect with
  proxies_list = [
    'http://1.1.1.1:2000',
    'http://2.2.2.2:2000',
    'http://1.2.3.0:2000'       # This is the fake one
  ]
  proxy_index = random.randint(0, len(proxies_list) - 1)
  proxy = {"http": proxies_list[proxy_index]}

  # List of user agents
  headers_list = [
    'Linux Mozilla 5/0',
    'Linux Mozilla 5/0',
    'Linux Mozilla 5/0'
  ]
  headers_index = random.randint(0, len(headers_list) - 1)
  headers = {'user-agent':headers_list[headers_index], 'Accept-Encoding':'none'}

  payload = {'dataToValidate':str(i),'actionName':'nc_signup'}
  answer = requests.post(url=url, headers=headers, proxies=proxy, data=payload).json()
  print(answer)

When proxy_index = random.randint(0, len(proxies_list) - 1) picks the fake one I get an answer anyways, the reason for that may be because the requests.post() function doesn't even use the argument proxies=proxy as expected.

question from:https://stackoverflow.com/questions/65921553/why-requests-post-does-not-use-a-provided-proxy-in-python

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Since you only construct the proxy object with an http option, the HTTPS URL you're trying to load is not proxied. Changing

proxy = {"http": proxies_list[proxy_index]}

to

proxy = {"http": proxies_list[proxy_index], "https": proxies_list[proxy_index]}

should fix the problem.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

2.1m questions

2.1m answers

60 comments

57.0k users

...