Python爬蟲實戰（4）：模擬登錄淘寶并獲取所有訂單 · Python爬蟲

經過多次嘗試，模擬登錄淘寶終于成功了，實在是不容易，淘寶的登錄加密和驗證太復雜了，煞費苦心，在此寫出來和大家一起分享，希望大家支持。 ## 本篇內容 1\. python模擬登錄淘寶網頁 2\. 獲取登錄用戶的所有訂單詳情 3\. 學會應對出現驗證碼的情況 4\. 體會一下復雜的模擬登錄機制 ## 探索部分成果 1\. 淘寶的密碼用了AES加密算法，最終將密碼轉化為256位，在POST時，傳輸的是256位長度的密碼。 2\. 淘寶在登錄時必須要輸入驗證碼，在經過幾次嘗試失敗后最終獲取了驗證碼圖片讓用戶手動輸入來驗證。 3\. 淘寶另外有復雜且每天在變的 ua 加密算法，在程序中我們需要提前獲取某一 ua 碼才可進行模擬登錄。 4\. 在獲取最后的登錄 st 碼時，歷經了多次請求和正則表達式提取，且 st 碼只可使用一次。 ## 整體思路梳理 1\. 手動到瀏覽器獲取 ua 碼以及加密后的密碼，只獲取一次即可，一勞永逸。 2\. 向登錄界面發送登錄請求，POST 一系列參數，包括 ua 碼以及密碼等等，獲得響應，提取驗證碼圖像。 3\. 用戶輸入手動驗證碼，重新加入驗證碼數據再次用 POST 方式發出請求，獲得響應，提取 J_Htoken。 4\. 利用 J_Htoken 向 alipay 發出請求，獲得響應，提取 st 碼。 5\. 利用 st 碼和用戶名，重新發出登錄請求，獲得響應，提取重定向網址，存儲 cookie。 6\. 利用 cookie 向其他個人頁面如訂單頁面發出請求，獲得響應，提取訂單詳情。是不是沒看懂？沒事，下面我將一點點說明自己模擬登錄的過程，希望大家可以理解。 ## 前期準備由于淘寶的 ua 算法和 aes 密碼加密算法太復雜了，ua 算法在淘寶每天都是在變化的，不過，這個內容你獲取之后一直用即可，經過測試之后沒有問題，一勞永逸。那么 ua 和 aes 密碼怎樣獲取呢？我們就從瀏覽器里面直接獲取吧，打開瀏覽器，找到淘寶的登錄界面，按 F12 或者瀏覽器右鍵審查元素。在這里我用的是火狐瀏覽器，首先記得在瀏覽器中設置一下顯示持續日志，要不然頁面跳轉了你就看不到之前抓取的信息了。在這里截圖如下： [![](https://box.kancloud.cn/2016-05-29_574a95ddb93ef.jpg)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/20150225013600.jpg) 好，那么接下來我們就從瀏覽器中獲取 ua 和 aes 密碼點擊網絡選項卡，這時都是空的，什么數據也沒有截取。這時你就在網頁上登錄一下試試吧，輸入用戶名啊，密碼啊，有必要時需要輸入驗證碼，點擊登錄。 [![](https://box.kancloud.cn/2016-05-29_574a95de02097.jpg)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/QQ%E6%88%AA%E5%9B%BE20150225014124.jpg) 等跳轉成功后，你就可以看到好多日志記錄了，點擊圖中的那一行 login.taobo.com，然后查看參數，你就會發現表單數據了，其中就包括 ua 還有下面的 password2，把這倆復制下來，我們之后要用到的。這就是我們需要的 ua 還有 aes 加密后的密碼。 [![](https://box.kancloud.cn/2016-05-29_574a95de2609f.jpg)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/QQ%E6%88%AA%E5%9B%BE20150225014019.jpg) 恩，讀到這里，你應該獲取到了屬于自己的 ua 和 password2 兩個內容。 ## 輸入驗證碼并獲取J_HToken 經過博主本人親自驗證，有時候，在模擬登錄時你并不需要輸入驗證碼，它直接返回的結果就是前面所說的下一步用到的 J_Token，而有時候你則會需要輸入驗證碼，等你手動輸入驗證碼之后，重新請求登錄一次。博主是邊寫程序邊更新文章的，現在寫完了是否有必要輸入驗證碼的檢驗以及在瀏覽器中呈現驗證碼。代碼如下 ~~~ __author__ = 'CQC' # -*- coding:utf-8 -*- import urllib import urllib2 import cookielib import re import webbrowser #模擬登錄淘寶類 class Taobao: ????#初始化方法 ????def __init__(self): ????????#登錄的URL ????????self.loginURL = "https://login.taobao.com/member/login.jhtml" ????????#代理IP地址，防止自己的IP被封禁 ????????self.proxyURL = 'http://120.193.146.97:843' ????????#登錄POST數據時發送的頭部信息 ????????self.loginHeaders =??{ ????????????'Host':'login.taobao.com', ????????????'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0', ????????????'Referer' : 'https://login.taobao.com/member/login.jhtml', ????????????'Content-Type': 'application/x-www-form-urlencoded', ????????????'Connection' : 'Keep-Alive' ????????} ????????#用戶名 ????????self.username = 'cqcre' ????????#ua字符串，經過淘寶ua算法計算得出，包含了時間戳,瀏覽器,屏幕分辨率,隨機數,鼠標移動,鼠標點擊,其實還有鍵盤輸入記錄,鼠標移動的記錄、點擊的記錄等等的信息 ????????self.ua = '191UW5TcyMNYQwiAiwTR3tCf0J/QnhEcUpkMmQ=|Um5Ockt0TXdPc011TXVKdyE=|U2xMHDJ+H2QJZwBxX39Rb1d5WXcrSixAJ1kjDVsN|VGhXd1llXGNaYFhkWmJaYl1gV2pIdUtyTXRKfkN4Qn1FeEF6R31TBQ==|VWldfS0TMw8xDjYWKhAwHiUdOA9wCDEVaxgkATdcNU8iDFoM|VmNDbUMV|V2NDbUMV|WGRYeCgGZhtmH2VScVI2UT5fORtmD2gCawwuRSJHZAFsCWMOdVYyVTpbPR99HWAFYVMpUDUFORshHiQdJR0jAT0JPQc/BDoFPgooFDZtVBR5Fn9VOwt2EWhCOVQ4WSJPJFkHXhgoSDVIMRgnHyFqQ3xEezceIRkmahRqFDZLIkUvRiEDaA9qQ3xEezcZORc5bzk=|WWdHFy0TMw8vEy0UIQE0ADgYJBohGjoAOw4uEiwXLAw2DThu9a==|WmBAED5+KnIbdRh1GXgFQSZbGFdrUm1UblZqVGxQa1ZiTGxQcEp1I3U=|W2NDEz19KXENZwJjHkY7Ui9OJQsre09zSWlXY1oMLBExHzERLxsuE0UT|XGZGFjh4LHQdcx5zH34DRyBdHlFtVGtSaFBsUmpWbVBkSmpXd05zTnMlcw==|XWdHFzl5LXUJYwZnGkI/VitKIQ8vEzMKNws3YTc=|XmdaZ0d6WmVFeUB8XGJaYEB4TGxWbk5yTndXa0tyT29Ta0t1QGBeZDI=' ????????#密碼，在這里不能輸入真實密碼，淘寶對此密碼進行了加密處理，256位，此處為加密后的密碼 ????????self.password2 = '7511aa68sx629e45de220d29174f1066537a73420ef6dbb5b46f202396703a2d56b0312df8769d886e6ca63d587fdbb99ee73927e8c07d9c88cd02182e1a21edc13fb8e140a4a2a4b5c253bf38484bd0e08199e03eb9bf7b365a5c673c03407d812b91394f0d3c7564042e3f2b11d156aeea37ad6460118914125ab8f8ac466f' ????????self.post = post = { ????????????'ua':self.ua, ????????????'TPL_checkcode':'', ????????????'CtrlVersion': '1,0,0,7', ????????????'TPL_password':'', ????????????'TPL_redirect_url':'http://i.taobao.com/my_taobao.htm?nekot=udm8087E1424147022443', ????????????'TPL_username':self.username, ????????????'loginsite':'0', ????????????'newlogin':'0', ????????????'from':'tb', ????????????'fc':'default', ????????????'style':'default', ????????????'css_style':'', ????????????'tid':'XOR_1_000000000000000000000000000000_625C4720470A0A050976770A', ????????????'support':'000001', ????????????'loginType':'4', ????????????'minititle':'', ????????????'minipara':'', ????????????'umto':'NaN', ????????????'pstrong':'3', ????????????'llnick':'', ????????????'sign':'', ????????????'need_sign':'', ????????????'isIgnore':'', ????????????'full_redirect':'', ????????????'popid':'', ????????????'callback':'', ????????????'guf':'', ????????????'not_duplite_str':'', ????????????'need_user_id':'', ????????????'poy':'', ????????????'gvfdcname':'10', ????????????'gvfdcre':'', ????????????'from_encoding ':'', ????????????'sub':'', ????????????'TPL_password_2':self.password2, ????????????'loginASR':'1', ????????????'loginASRSuc':'1', ????????????'allp':'', ????????????'oslanguage':'zh-CN', ????????????'sr':'1366*768', ????????????'osVer':'windows|6.1', ????????????'naviVer':'firefox|35' ????????} ????????#將POST的數據進行編碼轉換 ????????self.postData = urllib.urlencode(self.post) ????????#設置代理 ????????self.proxy = urllib2.ProxyHandler({'http':self.proxyURL}) ????????#設置cookie ????????self.cookie = cookielib.LWPCookieJar() ????????#設置cookie處理器 ????????self.cookieHandler = urllib2.HTTPCookieProcessor(self.cookie) ????????#設置登錄時用到的opener，它的open方法相當于urllib2.urlopen ????????self.opener = urllib2.build_opener(self.cookieHandler,self.proxy,urllib2.HTTPHandler) ????#得到是否需要輸入驗證碼，這次請求的相應有時會不同，有時需要驗證有時不需要 ????def needIdenCode(self): ????????#第一次登錄獲取驗證碼嘗試，構建request ????????request = urllib2.Request(self.loginURL,self.postData,self.loginHeaders) ????????#得到第一次登錄嘗試的相應 ????????response = self.opener.open(request) ????????#獲取其中的內容 ????????content = response.read().decode('gbk') ????????#獲取狀態嗎 ????????status = response.getcode() ????????#狀態碼為200，獲取成功 ????????if status == 200: ????????????print u"獲取請求成功" ????????????#\u8bf7\u8f93\u5165\u9a8c\u8bc1\u7801這六個字是請輸入驗證碼的utf-8編碼 ????????????pattern = re.compile(u'\u8bf7\u8f93\u5165\u9a8c\u8bc1\u7801',re.S) ????????????result = re.search(pattern,content) ????????????#如果找到該字符，代表需要輸入驗證碼 ????????????if result: ????????????????print u"此次安全驗證異常，您需要輸入驗證碼" ????????????????return content ????????????#否則不需要 ????????????else: ????????????????print u"此次安全驗證通過，您這次不需要輸入驗證碼" ????????????????return False ????????else: ????????????print u"獲取請求失敗" ????#得到驗證碼圖片 ????def getIdenCode(self,page): ????????#得到驗證碼的圖片 ????????pattern = re.compile(',re.S) ????????#匹配的結果 ????????matchResult = re.search(pattern,page) ????????#已經匹配得到內容，并且驗證碼圖片鏈接不為空 ????????if matchResult and matchResult.group(1): ????????????print matchResult.group(1) ????????????return matchResult.group(1) ????????else: ????????????print u"沒有找到驗證碼內容" ????????????return False ????#程序運行主干 ????def main(self): ????????#是否需要驗證碼，是則得到頁面內容，不是則返回False ????????needResult = self.needIdenCode() ????????if not needResult == False: ????????????print u"您需要手動輸入驗證碼" ????????????idenCode = self.getIdenCode(needResult) ????????????#得到了驗證碼的鏈接 ????????????if not idenCode == False: ????????????????print u"驗證碼獲取成功" ????????????????print u"請在瀏覽器中輸入您看到的驗證碼" ????????????????webbrowser.open_new_tab(idenCode) ????????????#驗證碼鏈接為空，無效驗證碼 ????????????else: ????????????????print u"驗證碼獲取失敗，請重試" ????????else: ????????????print u"不需要輸入驗證碼" taobao = Taobao() taobao.main() ~~~ 恩，請把里面的 ua 和 password2 還有用戶名換成自己的進行嘗試，用我的可能會產生錯誤的。運行結果 [![](https://box.kancloud.cn/2016-05-29_574a95de55950.jpg)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/QQ%E6%88%AA%E5%9B%BE20150225015508.jpg) 然后會蹦出瀏覽器，顯示了驗證碼的內容，這個需要你來手動輸入。在這里有小伙伴向我反映有這么個錯誤 [![](https://box.kancloud.cn/2016-05-29_574a95de75198.png)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/QQ%E5%9B%BE%E7%89%8720150227181617.png) 經過查證，竟然是版本問題，博主本人用的是 2.7.7，而小伙伴用的是 2.7.9。后來換成 2.7.7 就好了…，我也是醉了，希望有相同錯誤的小伙伴，可以嘗試換一下版本… 好啦，運行時會彈出瀏覽器，如圖 [![](https://box.kancloud.cn/2016-05-29_574a95de86d9f.jpg)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/QQ%E6%88%AA%E5%9B%BE20150225015717.jpg) 那么，我們現在需要手動輸入驗證碼，重新向登錄界面發出登錄請求，之前的post數據內容加入驗證碼這一項，重新請求一次，如果請求成功，則會返回下一步我們需要的 J_HToken，如果驗證碼輸入錯誤，則會返回驗證碼輸入錯誤的選項。好，下面，我已經寫到了獲取J_HToken的進度，代碼如下，現在運行程序，會蹦出瀏覽器，然后提示你輸入驗證碼，用戶手動輸入之后，則會返回一個頁面，我們提取出 J_Htoken即可。注意，到現在為止，你還沒有登錄成功，只是獲取到了J_HToken的值。目前寫到的代碼如下 ~~~ __author__ = 'CQC' # -*- coding:utf-8 -*- import urllib import urllib2 import cookielib import re import webbrowser #模擬登錄淘寶類 class Taobao: ????#初始化方法 ????def __init__(self): ????????#登錄的URL ????????self.loginURL = "https://login.taobao.com/member/login.jhtml" ????????#代理IP地址，防止自己的IP被封禁 ????????self.proxyURL = 'http://120.193.146.97:843' ????????#登錄POST數據時發送的頭部信息 ????????self.loginHeaders =??{ ????????????'Host':'login.taobao.com', ????????????'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0', ????????????'Referer' : 'https://login.taobao.com/member/login.jhtml', ????????????'Content-Type': 'application/x-www-form-urlencoded', ????????????'Connection' : 'Keep-Alive' ????????} ????????#用戶名 ????????self.username = 'cqcre' ????????#ua字符串，經過淘寶ua算法計算得出，包含了時間戳,瀏覽器,屏幕分辨率,隨機數,鼠標移動,鼠標點擊,其實還有鍵盤輸入記錄,鼠標移動的記錄、點擊的記錄等等的信息 ????????self.ua = '191UW5TcyMNYQwiAiwTR3tCf0J/QnhEcUpkMmQ=|Um5Ockt0TXdPc011TXVKdyE=|U2xMHDJ+H2QJZwBxX39Rb1d5WXcrSixAJ1kjDVsN|VGhXd1llXGNaYFhkWmJaYl1gV2pIdUtyTXRKfkN4Qn1FeEF6R31TBQ==|VWldfS0TMw8xDjYWKhAwHiUdOA9wCDEVaxgkATdcNU8iDFoM|VmNDbUMV|V2NDbUMV|WGRYeCgGZhtmH2VScVI2UT5fORtmD2gCawwuRSJHZAFsCWMOdVYyVTpbPR99HWAFYVMpUDUFORshHiQdJR0jAT0JPQc/BDoFPgooFDZtVBR5Fn9VOwt2EWhCOVQ4WSJPJFkHXhgoSDVIMRgnHyFqQ3xEezceIRkmahRqFDZLIkUvRiEDaA9qQ3xEezcZORc5bzk=|WWdHFy0TMw8vEy0UIQE0ADgYJBohGjoAOw4uEiwXLAw2DThuOA==|WmBAED5+KnIbdRh1GXgFQSZbGFdrUm1UblZqVGxQa1ZiTGxQcEp1I3U=|W2NDEz19KXENZwJjHkY7Ui9OJQsre09zSWlXY1oMLBExHzERLxsuE0UT|XGZGFjh4LHQdcx5zH34DRyBdHlFtVGtSaFBsUmpWbVBkSmpXd05zTnMlcw==|XWdHFzl5LXUJYwZnGkI/VitKIQ8vEzMKNws3YTc=|XmdaZ0d6WmVFeUB8XGJaYEB4TGxWbk5yTndXa0tyT29Ta0t1QGBeZDI=' ????????#密碼，在這里不能輸入真實密碼，淘寶對此密碼進行了加密處理，256位，此處為加密后的密碼 ????????self.password2 = '7511aa6854629e45de220d29174f1066537a73420ef6dbb5b46f202396703a2d56b0312df8769d886e6ca63d587fdbb99ee73927e8c07d9c88cd02182e1a21edc13fb8e0a4a2a4b5c253bf38484bd0e08199e03eb9bf7b365a5c673c03407d812b91394f0d3c7564042e3f2b11d156aeea37ad6460118914125ab8f8ac466f' ????????self.post = post = { ????????????'ua':self.ua, ????????????'TPL_checkcode':'', ????????????'CtrlVersion': '1,0,0,7', ????????????'TPL_password':'', ????????????'TPL_redirect_url':'http://i.taobao.com/my_taobao.htm?nekot=udm8087E1424147022443', ????????????'TPL_username':self.username, ????????????'loginsite':'0', ????????????'newlogin':'0', ????????????'from':'tb', ????????????'fc':'default', ????????????'style':'default', ????????????'css_style':'', ????????????'tid':'XOR_1_000000000000000000000000000000_625C4720470A0A050976770A', ????????????'support':'000001', ????????????'loginType':'4', ????????????'minititle':'', ????????????'minipara':'', ????????????'umto':'NaN', ????????????'pstrong':'3', ????????????'llnick':'', ????????????'sign':'', ????????????'need_sign':'', ????????????'isIgnore':'', ????????????'full_redirect':'', ????????????'popid':'', ????????????'callback':'', ????????????'guf':'', ????????????'not_duplite_str':'', ????????????'need_user_id':'', ????????????'poy':'', ????????????'gvfdcname':'10', ????????????'gvfdcre':'', ????????????'from_encoding ':'', ????????????'sub':'', ????????????'TPL_password_2':self.password2, ????????????'loginASR':'1', ????????????'loginASRSuc':'1', ????????????'allp':'', ????????????'oslanguage':'zh-CN', ????????????'sr':'1366*768', ????????????'osVer':'windows|6.1', ????????????'naviVer':'firefox|35' ????????} ????????#將POST的數據進行編碼轉換 ????????self.postData = urllib.urlencode(self.post) ????????#設置代理 ????????self.proxy = urllib2.ProxyHandler({'http':self.proxyURL}) ????????#設置cookie ????????self.cookie = cookielib.LWPCookieJar() ????????#設置cookie處理器 ????????self.cookieHandler = urllib2.HTTPCookieProcessor(self.cookie) ????????#設置登錄時用到的opener，它的open方法相當于urllib2.urlopen ????????self.opener = urllib2.build_opener(self.cookieHandler,self.proxy,urllib2.HTTPHandler) ????#得到是否需要輸入驗證碼，這次請求的相應有時會不同，有時需要驗證有時不需要 ????def needCheckCode(self): ????????#第一次登錄獲取驗證碼嘗試，構建request ????????request = urllib2.Request(self.loginURL,self.postData,self.loginHeaders) ????????#得到第一次登錄嘗試的相應 ????????response = self.opener.open(request) ????????#獲取其中的內容 ????????content = response.read().decode('gbk') ????????#獲取狀態嗎 ????????status = response.getcode() ????????#狀態碼為200，獲取成功 ????????if status == 200: ????????????print u"獲取請求成功" ????????????#\u8bf7\u8f93\u5165\u9a8c\u8bc1\u7801這六個字是請輸入驗證碼的utf-8編碼 ????????????pattern = re.compile(u'\u8bf7\u8f93\u5165\u9a8c\u8bc1\u7801',re.S) ????????????result = re.search(pattern,content) ????????????print content ????????????#如果找到該字符，代表需要輸入驗證碼 ????????????if result: ????????????????print u"此次安全驗證異常，您需要輸入驗證碼" ????????????????return content ????????????#否則不需要 ????????????else: ????????????????#返回結果直接帶有J_HToken字樣，表明直接驗證通過 ????????????????tokenPattern = re.compile('id="J_HToken"') ????????????????tokenMatch = re.search(tokenPattern,content) ????????????????if tokenMatch: ????????????????????print u"此次安全驗證通過，您這次不需要輸入驗證碼" ????????????????????return False ????????else: ????????????print u"獲取請求失敗" ????????????return None ????#得到驗證碼圖片 ????def getCheckCode(self,page): ????????#得到驗證碼的圖片 ????????pattern = re.compile(',re.S) ????????#匹配的結果 ????????matchResult = re.search(pattern,page) ????????#已經匹配得到內容，并且驗證碼圖片鏈接不為空 ????????if matchResult and matchResult.group(1): ????????????print matchResult.group(1) ????????????return matchResult.group(1) ????????else: ????????????print u"沒有找到驗證碼內容" ????????????return False ????#輸入驗證碼，重新請求，如果驗證成功，則返回J_HToken ????def loginWithCheckCode(self): ????????#提示用戶輸入驗證碼 ????????checkcode = raw_input('請輸入驗證碼:') ????????#將驗證碼重新添加到post的數據中 ????????self.post['TPL_checkcode'] = checkcode ????????#對post數據重新進行編碼 ????????self.postData = urllib.urlencode(self.post) ????????try: ????????????#再次構建請求，加入驗證碼之后的第二次登錄嘗試 ????????????request = urllib2.Request(self.loginURL,self.postData,self.loginHeaders) ????????????#得到第一次登錄嘗試的相應 ????????????response = self.opener.open(request) ????????????#獲取其中的內容 ????????????content = response.read().decode('gbk') ????????????#檢測驗證碼錯誤的正則表達式，\u9a8c\u8bc1\u7801\u9519\u8bef 是驗證碼錯誤五個字的編碼 ????????????pattern = re.compile(u'\u9a8c\u8bc1\u7801\u9519\u8bef',re.S) ????????????result = re.search(pattern,content) ????????????#如果返回頁面包括了，驗證碼錯誤五個字 ????????????if result: ????????????????print u"驗證碼輸入錯誤" ????????????????return False ????????????else: ????????????????#返回結果直接帶有J_HToken字樣，說明驗證碼輸入成功，成功跳轉到了獲取HToken的界面 ????????????????tokenPattern = re.compile('id="J_HToken" value="(.*?)"') ????????????????tokenMatch = re.search(tokenPattern,content) ????????????????#如果匹配成功，找到了J_HToken ????????????????if tokenMatch: ????????????????????print u"驗證碼輸入正確" ????????????????????print tokenMatch.group(1) ????????????????????return tokenMatch.group(1) ????????????????else: ????????????????????#匹配失敗，J_Token獲取失敗 ????????????????????print u"J_Token獲取失敗" ????????????????????return False ????????except urllib2.HTTPError, e: ????????????print u"連接服務器出錯，錯誤原因",e.reason ????????????return False ????#程序運行主干 ????def main(self): ????????#是否需要驗證碼，是則得到頁面內容，不是則返回False ????????needResult = self.needCheckCode() ????????#請求獲取失敗，得到的結果是None ????????if not needResult ==None: ????????????if not needResult == False: ????????????????print u"您需要手動輸入驗證碼" ????????????????idenCode = self.getCheckCode(needResult) ????????????????#得到了驗證碼的鏈接 ????????????????if not idenCode == False: ????????????????????print u"驗證碼獲取成功" ????????????????????print u"請在瀏覽器中輸入您看到的驗證碼" ????????????????????webbrowser.open_new_tab(idenCode) ????????????????????J_HToken = self.loginWithCheckCode() ????????????????????print "J_HToken",J_HToken ????????????????#驗證碼鏈接為空，無效驗證碼 ????????????????else: ????????????????????print u"驗證碼獲取失敗，請重試" ????????????else: ????????????????print u"不需要輸入驗證碼" ????????else: ????????????print u"請求登錄頁面失敗，無法確認是否需要驗證碼" taobao = Taobao() taobao.main() ~~~ 現在的運行結果是這樣的，我們已經可以得到 J_HToken 了，離成功又邁進了一步。 [![](https://box.kancloud.cn/2016-05-29_574a95dea49de.jpg)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/QQ%E6%88%AA%E5%9B%BE20150225200329.jpg) 好，到現在為止，我們應該可以獲取到J_HToken的值啦。 ## 利用J_HToken獲取st st也是一個經計算得到的code，可以這么理解，st是淘寶后臺利用J_HToken以及其他數據經過計算之后得到的，可以利用st和用戶名直接用get方式登錄，所以st可以理解為一個秘鑰。這個st值只會使用一次，如果第二次用get方式登錄則會失效。所以它是一次性使用的。下面J_HToken計算st的方法如下 ~~~ #通過token獲得st def getSTbyToken(self,token): ????tokenURL = 'https://passport.alipay.com/mini_apply_st.js?site=0&token=%s&callback=stCallback6' % token ????request = urllib2.Request(tokenURL) ????response = urllib2.urlopen(request) ????#處理st，獲得用戶淘寶主頁的登錄地址 ????pattern = re.compile('{"st":"(.*?)"}',re.S) ????result = re.search(pattern,response.read()) ????#如果成功匹配 ????if result: ????????print u"成功獲取st碼" ????????#獲取st的值 ????????st = result.group(1) ????????return st ????else: ????????print u"未匹配到st" ????????return False ~~~ ## 直接利用st登錄得到st之后，基本上就大功告成啦，一段辛苦終于沒有白費，你可以直接構建get方式請求的URL，直接訪問這個URL便可以實現登錄。 ~~~ stURL = 'https://login.taobao.com/member/vst.htm?st=%s&TPL_username=%s' % (st,username) ~~~ 比如 ~~~ https://login.taobao.com/member/vst.htm?st=1uynJELa4hKfsfWU3OjPJCw&TPL_username=cqcre ~~~ 直接訪問該鏈接即可實現登錄，不過我這個應該已經失效了吧~ 代碼在這先不貼了，剩下的一起貼了~ ## 獲取已買到的寶貝頁面已買到的寶貝的頁面地址是 ~~~ http://buyer.trade.taobao.com/trade/itemlist/list_bought_items.htm ~~~ 另外還有頁碼的參數。重新構建一個帶有cookie的opener，將上面的帶有st的URL打開，保存它的cookie，然后再利用這個opener打開已買到的寶貝的頁面，你就會得到已買到的寶貝頁面詳情了。 ~~~ #獲得已買到的寶貝頁面 def getGoodsPage(self,pageIndex): ????goodsURL = 'http://buyer.trade.taobao.com/trade/itemlist/listBoughtItems.htm?action=itemlist/QueryAction&event_submit_do_query=1&pageNum=' + str(pageIndex) ????response = self.newOpener.open(goodsURL) ????page =??response.read().decode('gbk') ????return page ~~~ 正則表達式提取信息這是我的已買到的寶貝界面，審查元素可以看到，每一個寶貝都是tbody標簽包圍著。 [![](https://box.kancloud.cn/2016-05-29_574a95dec8c8f.jpg)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/QQ%E6%88%AA%E5%9B%BE20150225223302.jpg)我們現在想獲取訂單時間，訂單號，賣家店鋪名稱，寶貝名稱，原價，購買數量，最后付款多少，交易狀態這幾個量，具體就不再分析啦，正則表達式還不熟悉的同學請參考前面所說的正則表達式的用法，在這里，正則表達式匹配的代碼是 ~~~ #u'\u8ba2\u5355\u53f7'是訂單號的編碼 pattern = re.compile(u'dealtime.*?>(.*?).*?\u8ba2\u5355\u53f7.*?(.*?).*?shopname.*?title="(.*?)".*?baobei-name">.*?(.*?).*?' ???????????????????? u'price.*?title="(.*?)".*?quantity.*?title="(.*?)".*?amount.*?em.*?>(.*?).*?trade-status.*?(.*?)',re.S) result = re.findall(pattern,page) for item in result: ????print '------------------------------------------------------------' ????print "購買日期:",item[0].strip(), '訂單號:',item[1].strip(),'賣家店鋪:',item[2].strip() ????print '寶貝名稱:',item[3].strip() ????print '原價:',item[4].strip(),'購買數量:',item[5].strip(),'實際支付:',item[6].strip(),'交易狀態',item[7].strip() ~~~ ## 最終代碼整理恩，你懂得，最重要的東西來了，經過博主2天多的奮戰，代碼基本就構建完成。寫了兩個類，其中提取頁面信息的方法我單獨放到了一個類中，叫 tool.py，類名為 Tool。先看一下運行結果吧~ [![](https://box.kancloud.cn/2016-05-29_574a95df0c3d0.jpg)](http://qiniu.cuiqingcai.com/wp-content/uploads/2015/02/QQ%E6%88%AA%E5%9B%BE20150225234414.jpg) 最終代碼如下 ~~~ tool.py ~~~ ~~~ __author__ = 'CQC' # -*- coding:utf-8 -*- import re #處理獲得的寶貝頁面 class Tool: ????#初始化 ????def __init__(self): ????????pass ????#獲得頁碼數 ????def getPageNum(self,page): ????????pattern = re.compile(u'.*?\u5171(.*?)\u9875',re.S) ????????result = re.search(pattern,page) ????????if result: ????????????print "找到了共多少頁" ????????????pageNum = result.group(1).strip() ????????????print '共',pageNum,'頁' ????????????return pageNum ????def getGoodsInfo(self,page): ????????#u'\u8ba2\u5355\u53f7'是訂單號的編碼 ????????pattern = re.compile(u'dealtime.*?>(.*?).*?\u8ba2\u5355\u53f7.*?(.*?).*?shopname.*?title="(.*?)".*?baobei-name">.*?(.*?).*?' ???????????????????????????? u'price.*?title="(.*?)".*?quantity.*?title="(.*?)".*?amount.*?em.*?>(.*?).*?trade-status.*?(.*?)',re.S) ????????result = re.findall(pattern,page) ????????for item in result: ????????????print '------------------------------------------------------------' ????????????print "購買日期:",item[0].strip(), '訂單號:',item[1].strip(),'賣家店鋪:',item[2].strip() ????????????print '寶貝名稱:',item[3].strip() ????????????print '原價:',item[4].strip(),'購買數量:',item[5].strip(),'實際支付:',item[6].strip(),'交易狀態',item[7].strip() ~~~ ~~~ taobao.py ~~~ ~~~ __author__ = 'CQC' # -*- coding:utf-8 -*- import urllib import urllib2 import cookielib import re import webbrowser import tool #模擬登錄淘寶類 class Taobao: ????#初始化方法 ????def __init__(self): ????????#登錄的URL ????????self.loginURL = "https://login.taobao.com/member/login.jhtml" ????????#代理IP地址，防止自己的IP被封禁 ????????self.proxyURL = 'http://120.193.146.97:843' ????????#登錄POST數據時發送的頭部信息 ????????self.loginHeaders =??{ ????????????'Host':'login.taobao.com', ????????????'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0', ????????????'Referer' : 'https://login.taobao.com/member/login.jhtml', ????????????'Content-Type': 'application/x-www-form-urlencoded', ????????????'Connection' : 'Keep-Alive' ????????} ????????#用戶名 ????????self.username = 'cqcre' ????????#ua字符串，經過淘寶ua算法計算得出，包含了時間戳,瀏覽器,屏幕分辨率,隨機數,鼠標移動,鼠標點擊,其實還有鍵盤輸入記錄,鼠標移動的記錄、點擊的記錄等等的信息 ????????self.ua = '191UW5TcyMNYQwiAiwTR3tCf0J/QnhEcUpkMmQ=|Um5Ockt0TXdPc011TXVKdyE=|U2xMHDJ+H2QJZwBxX39Rb1d5WXcrSixAJ1kjDVsN|VGhXd1llXGNaYFhkWmJaYl1gV2pIdUtyTXRKfkN4Qn1FeEF6R31TBQ==|VWldfS0TMw8xDjYWKhAwHiUdOA9wCDEVaxgkATdcNU8iDFoM|VmNDbUMV|V2NDbUMV|WGRYeCgGZhtmH2VScVI2UT5fORtmD2gCawwuRSJHZAFsCWMOdVYyVTpbPR99HWAFYVMpUDUFORshHiQdJR0jAT0JPQc/BDoFPgooFDZtVBR5Fn9VOwt2EWhCOVQ4WSJPJFkHXhgoSDVIMRgnHyFqQ3xEezceIRkmahRqFDZLIkUvRiEDaA9qQ3xEezcZORc5bzk=|WWdHFy0TMw8vEy0UIQE0ADgYJBohGjoAOw4uEiwXLAw2DThuOA==|WmBAED5+KnIbdRh1GXgFQSZbGFdrUm1UblZqVGxQa1ZiTGxQcEp1I3U=|W2NDEz19KXENZwJjHkY7Ui9OJQsre09zSWlXY1oMLBExHzERLxsuE0UT|XGZGFjh4LHQdcx5zH34DRyBdHlFtVGtSaFBsUmpWbVBkSmpXd05zTnMlcw==|XWdHFzl5LXUJYwZnGkI/VitKIQ8vEzMKNws3YTc=|XmdaZ0d6WmVFeUB8XGJaYEB4TGxWbk5yTndXa0tyT29Ta0t1QGBeZDI=' ????????#密碼，在這里不能輸入真實密碼，淘寶對此密碼進行了加密處理，256位，此處為加密后的密碼 ????????self.password2 = '7511aa6854629e45de220d29174f1066537a73420ef6dbb5b46f202396703a2d56b0312df8769d886e6ca63d587fdbb99ee73927e8c07d9c88cd02182e1a21edc13fb8e140a4a2a4b53bf38484bd0e08199e03eb9bf7b365a5c673c03407d812b91394f0d3c7564042e3f2b11d156aeea37ad6460118914125ab8f8ac466f' ????????self.post = post = { ????????????'ua':self.ua, ????????????'TPL_checkcode':'', ????????????'CtrlVersion': '1,0,0,7', ????????????'TPL_password':'', ????????????'TPL_redirect_url':'http://i.taobao.com/my_taobao.htm?nekot=udm8087E1424147022443', ????????????'TPL_username':self.username, ????????????'loginsite':'0', ????????????'newlogin':'0', ????????????'from':'tb', ????????????'fc':'default', ????????????'style':'default', ????????????'css_style':'', ????????????'tid':'XOR_1_000000000000000000000000000000_625C4720470A0A050976770A', ????????????'support':'000001', ????????????'loginType':'4', ????????????'minititle':'', ????????????'minipara':'', ????????????'umto':'NaN', ????????????'pstrong':'3', ????????????'llnick':'', ????????????'sign':'', ????????????'need_sign':'', ????????????'isIgnore':'', ????????????'full_redirect':'', ????????????'popid':'', ????????????'callback':'', ????????????'guf':'', ????????????'not_duplite_str':'', ????????????'need_user_id':'', ????????????'poy':'', ????????????'gvfdcname':'10', ????????????'gvfdcre':'', ????????????'from_encoding ':'', ????????????'sub':'', ????????????'TPL_password_2':self.password2, ????????????'loginASR':'1', ????????????'loginASRSuc':'1', ????????????'allp':'', ????????????'oslanguage':'zh-CN', ????????????'sr':'1366*768', ????????????'osVer':'windows|6.1', ????????????'naviVer':'firefox|35' ????????} ????????#將POST的數據進行編碼轉換 ????????self.postData = urllib.urlencode(self.post) ????????#設置代理 ????????self.proxy = urllib2.ProxyHandler({'http':self.proxyURL}) ????????#設置cookie ????????self.cookie = cookielib.LWPCookieJar() ????????#設置cookie處理器 ????????self.cookieHandler = urllib2.HTTPCookieProcessor(self.cookie) ????????#設置登錄時用到的opener，它的open方法相當于urllib2.urlopen ????????self.opener = urllib2.build_opener(self.cookieHandler,self.proxy,urllib2.HTTPHandler) ????????#賦值J_HToken ????????self.J_HToken = '' ????????#登錄成功時，需要的Cookie ????????self.newCookie = cookielib.CookieJar() ????????#登陸成功時，需要的一個新的opener ????????self.newOpener = urllib2.build_opener(urllib2.HTTPCookieProcessor(self.newCookie)) ????????#引入工具類 ????????self.tool = tool.Tool() ????#得到是否需要輸入驗證碼，這次請求的相應有時會不同，有時需要驗證有時不需要 ????def needCheckCode(self): ????????#第一次登錄獲取驗證碼嘗試，構建request ????????request = urllib2.Request(self.loginURL,self.postData,self.loginHeaders) ????????#得到第一次登錄嘗試的相應 ????????response = self.opener.open(request) ????????#獲取其中的內容 ????????content = response.read().decode('gbk') ????????#獲取狀態嗎 ????????status = response.getcode() ????????#狀態碼為200，獲取成功 ????????if status == 200: ????????????print u"獲取請求成功" ????????????#\u8bf7\u8f93\u5165\u9a8c\u8bc1\u7801這六個字是請輸入驗證碼的utf-8編碼 ????????????pattern = re.compile(u'\u8bf7\u8f93\u5165\u9a8c\u8bc1\u7801',re.S) ????????????result = re.search(pattern,content) ????????????#如果找到該字符，代表需要輸入驗證碼 ????????????if result: ????????????????print u"此次安全驗證異常，您需要輸入驗證碼" ????????????????return content ????????????#否則不需要 ????????????else: ????????????????#返回結果直接帶有J_HToken字樣，表明直接驗證通過 ????????????????tokenPattern = re.compile('id="J_HToken" value="(.*?)"') ????????????????tokenMatch = re.search(tokenPattern,content) ????????????????if tokenMatch: ????????????????????self.J_HToken = tokenMatch.group(1) ????????????????????print u"此次安全驗證通過，您這次不需要輸入驗證碼" ????????????????????return False ????????else: ????????????print u"獲取請求失敗" ????????????return None ????#得到驗證碼圖片 ????def getCheckCode(self,page): ????????#得到驗證碼的圖片 ????????pattern = re.compile(',re.S) ????????#匹配的結果 ????????matchResult = re.search(pattern,page) ????????#已經匹配得到內容，并且驗證碼圖片鏈接不為空 ????????if matchResult and matchResult.group(1): ????????????return matchResult.group(1) ????????else: ????????????print u"沒有找到驗證碼內容" ????????????return False ????#輸入驗證碼，重新請求，如果驗證成功，則返回J_HToken ????def loginWithCheckCode(self): ????????#提示用戶輸入驗證碼 ????????checkcode = raw_input('請輸入驗證碼:') ????????#將驗證碼重新添加到post的數據中 ????????self.post['TPL_checkcode'] = checkcode ????????#對post數據重新進行編碼 ????????self.postData = urllib.urlencode(self.post) ????????try: ????????????#再次構建請求，加入驗證碼之后的第二次登錄嘗試 ????????????request = urllib2.Request(self.loginURL,self.postData,self.loginHeaders) ????????????#得到第一次登錄嘗試的相應 ????????????response = self.opener.open(request) ????????????#獲取其中的內容 ????????????content = response.read().decode('gbk') ????????????#檢測驗證碼錯誤的正則表達式，\u9a8c\u8bc1\u7801\u9519\u8bef 是驗證碼錯誤五個字的編碼 ????????????pattern = re.compile(u'\u9a8c\u8bc1\u7801\u9519\u8bef',re.S) ????????????result = re.search(pattern,content) ????????????#如果返回頁面包括了，驗證碼錯誤五個字 ????????????if result: ????????????????print u"驗證碼輸入錯誤" ????????????????return False ????????????else: ????????????????#返回結果直接帶有J_HToken字樣，說明驗證碼輸入成功，成功跳轉到了獲取HToken的界面 ????????????????tokenPattern = re.compile('id="J_HToken" value="(.*?)"') ????????????????tokenMatch = re.search(tokenPattern,content) ????????????????#如果匹配成功，找到了J_HToken ????????????????if tokenMatch: ????????????????????print u"驗證碼輸入正確" ????????????????????self.J_HToken = tokenMatch.group(1) ????????????????????return tokenMatch.group(1) ????????????????else: ????????????????????#匹配失敗，J_Token獲取失敗 ????????????????????print u"J_Token獲取失敗" ????????????????????return False ????????except urllib2.HTTPError, e: ????????????print u"連接服務器出錯，錯誤原因",e.reason ????????????return False ????#通過token獲得st ????def getSTbyToken(self,token): ????????tokenURL = 'https://passport.alipay.com/mini_apply_st.js?site=0&token=%s&callback=stCallback6' % token ????????request = urllib2.Request(tokenURL) ????????response = urllib2.urlopen(request) ????????#處理st，獲得用戶淘寶主頁的登錄地址 ????????pattern = re.compile('{"st":"(.*?)"}',re.S) ????????result = re.search(pattern,response.read()) ????????#如果成功匹配 ????????if result: ????????????print u"成功獲取st碼" ????????????#獲取st的值 ????????????st = result.group(1) ????????????return st ????????else: ????????????print u"未匹配到st" ????????????return False ????#利用st碼進行登錄,獲取重定向網址 ????def loginByST(self,st,username): ????????stURL = 'https://login.taobao.com/member/vst.htm?st=%s&TPL_username=%s' % (st,username) ????????headers = { ????????????'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0', ????????????'Host':'login.taobao.com', ????????????'Connection' : 'Keep-Alive' ????????} ????????request = urllib2.Request(stURL,headers = headers) ????????response = self.newOpener.open(request) ????????content =??response.read().decode('gbk') ????????#檢測結果，看是否登錄成功 ????????pattern = re.compile('top.location = "(.*?)"',re.S) ????????match = re.search(pattern,content) ????????if match: ????????????print u"登錄網址成功" ????????????location = match.group(1) ????????????return True ????????else: ????????????print "登錄失敗" ????????????return False ????#獲得已買到的寶貝頁面 ????def getGoodsPage(self,pageIndex): ????????goodsURL = 'http://buyer.trade.taobao.com/trade/itemlist/listBoughtItems.htm?action=itemlist/QueryAction&event_submit_do_query=1' + '&pageNum=' + str(pageIndex) ????????response = self.newOpener.open(goodsURL) ????????page =??response.read().decode('gbk') ????????return page ????#獲取所有已買到的寶貝信息 ????def getAllGoods(self,pageNum): ????????print u"獲取到的商品列表如下" ????????for x in range(1,int(pageNum)+1): ????????????page = self.getGoodsPage(x) ????????????self.tool.getGoodsInfo(page) ????#程序運行主干 ????def main(self): ????????#是否需要驗證碼，是則得到頁面內容，不是則返回False ????????needResult = self.needCheckCode() ????????#請求獲取失敗，得到的結果是None ????????if not needResult ==None: ????????????if not needResult == False: ????????????????print u"您需要手動輸入驗證碼" ????????????????checkCode = self.getCheckCode(needResult) ????????????????#得到了驗證碼的鏈接 ????????????????if not checkCode == False: ????????????????????print u"驗證碼獲取成功" ????????????????????print u"請在瀏覽器中輸入您看到的驗證碼" ????????????????????webbrowser.open_new_tab(checkCode) ????????????????????self.loginWithCheckCode() ????????????????#驗證碼鏈接為空，無效驗證碼 ????????????????else: ????????????????????print u"驗證碼獲取失敗，請重試" ????????????else: ????????????????print u"不需要輸入驗證碼" ????????else: ????????????print u"請求登錄頁面失敗，無法確認是否需要驗證碼" ????????#判斷token是否正常獲取到 ????????if not self.J_HToken: ????????????print "獲取Token失敗，請重試" ????????????return ????????#獲取st碼 ????????st = self.getSTbyToken(self.J_HToken) ????????#利用st進行登錄 ????????result = self.loginByST(st,self.username) ????????if result: ????????????#獲得所有寶貝的頁面 ????????????page = self.getGoodsPage(1) ????????????pageNum = self.tool.getPageNum(page) ????????????self.getAllGoods(pageNum) ????????else: ????????????print u"登錄失敗" taobao = Taobao() taobao.main() ~~~ 好啦，運行結果就是上面貼的圖片，可以成功獲取到自己的商品列表，前提是把你們的用戶名，ua，password2這三個設置好。以上均為博主親身所敲，代碼寫的不好，謹在此貼出和大家一起分享經驗~ 小伙伴們試一下吧，希望對大家有幫助~