[TOC] # 一 反爬思路 部分網站會通過統計某個IP或agent在單位時間內的請求次數來識別人機進行反爬 # 二 對抗措施 1. 找免費或付費的代理 2. 實例化一個代理池(IP或/agent) 3. 通過for循環或隨機數調用代理IP # 三 相關代理網站 * 快代理 * 西祠代理 * www.goubanjia.com # 四 代碼實現 ```python import requests import random # 1. UA和IP代理池 header_list = [ {"user-agent": "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Maxthon 2.0)"}, {"user-agent": "Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1"}, {"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11"} ] proxy_list = [ {"http": "112.115.57.20:3128"}, {'http': '121.41.171.223:3128'} ] # 2. 隨機獲取UA和代理IP header = random.choice(header_list) proxy = random.choice(proxy_list) url = 'http://www.baidu.com/s?ie=UTF-8&wd=ip' # 3. 設置代理 response = requests.get(url=url,headers=header,proxies=proxy) response.encoding = 'utf-8' with open('daili.html', 'wb') as fp: fp.write(response.content) ```