<ruby id="bdb3f"></ruby>

    <p id="bdb3f"><cite id="bdb3f"></cite></p>

      <p id="bdb3f"><cite id="bdb3f"><th id="bdb3f"></th></cite></p><p id="bdb3f"></p>
        <p id="bdb3f"><cite id="bdb3f"></cite></p>

          <pre id="bdb3f"></pre>
          <pre id="bdb3f"><del id="bdb3f"><thead id="bdb3f"></thead></del></pre>

          <ruby id="bdb3f"><mark id="bdb3f"></mark></ruby><ruby id="bdb3f"></ruby>
          <pre id="bdb3f"><pre id="bdb3f"><mark id="bdb3f"></mark></pre></pre><output id="bdb3f"></output><p id="bdb3f"></p><p id="bdb3f"></p>

          <pre id="bdb3f"><del id="bdb3f"><progress id="bdb3f"></progress></del></pre>

                <ruby id="bdb3f"></ruby>

                企業??AI智能體構建引擎,智能編排和調試,一鍵部署,支持知識庫和私有化部署方案 廣告
                **1. 在`settings.py`做如下配置** ```python DOWNLOADER_MIDDLEWARES = { "proxies_pro.middlewares.ProxiesProDownloaderMiddleware": 543, # 啟用自己定義的中間件 "proxies_pro.middlewares.RandomIpPoolMiddleware": 125, } IPPOOL = [ # 免費代理,無密碼和用戶名 {"ipaddr": "101.236.21.22:8866"}, {"ipaddr": "101.126.18.101:8866"}, {"ipaddr": "118.31.220.3:8080"}, {"ipaddr": "35.214.170.66:3128"}, # 付費代理,有密碼和用戶名 {"ipaddr": "56.214.180.69:3990", "user_passwd":"username:password"}, ] ``` <br/> **2. 在`middlewares.py`中自定義 RandomIpPoolMiddleware 中間件** ```python import random import logging import base64 from proxies_pro.settings import IPPOOL myLogger = logging.getLogger(__name__) class RandomIpPoolMiddleware: """ 如果某一個ip無效或者超時,process_request會重復其它ip 在我的測試中重復了4次,如果重復了4次依然沒有一個ip有效 則拋出異常: twisted.internet.error.TCPTimedOutError: TCP connection timed out: 10060: 由于連接方在一段時間后沒有正確答復或連接的主機沒有反應,連接嘗試失敗。. """ def process_request(self, request, spider): proxy = random.choice(IPPOOL) # 付費的ip代理需要進行認證 if 'user_passwd' in proxy: # 對賬號密碼進行編碼 b64_up = base64.b64encode(proxy['user_passwd']) # 設置認證 request.headers['Proxy-Authorization'] = 'Basic '+ b64_up.decode() # 設置代理 request.meta['proxy'] = proxy["ipaddr"] else: # 設置代理 request.meta['proxy'] = proxy['ipaddr'] return None def process_response(self, request, response, spider): myLogger.warning("proxy:{}".format(request.meta["proxy"])) return response ```
                  <ruby id="bdb3f"></ruby>

                  <p id="bdb3f"><cite id="bdb3f"></cite></p>

                    <p id="bdb3f"><cite id="bdb3f"><th id="bdb3f"></th></cite></p><p id="bdb3f"></p>
                      <p id="bdb3f"><cite id="bdb3f"></cite></p>

                        <pre id="bdb3f"></pre>
                        <pre id="bdb3f"><del id="bdb3f"><thead id="bdb3f"></thead></del></pre>

                        <ruby id="bdb3f"><mark id="bdb3f"></mark></ruby><ruby id="bdb3f"></ruby>
                        <pre id="bdb3f"><pre id="bdb3f"><mark id="bdb3f"></mark></pre></pre><output id="bdb3f"></output><p id="bdb3f"></p><p id="bdb3f"></p>

                        <pre id="bdb3f"><del id="bdb3f"><progress id="bdb3f"></progress></del></pre>

                              <ruby id="bdb3f"></ruby>

                              哎呀哎呀视频在线观看