# httpclient4
<div><div><div style="font-family:Verdana, Arial, Tahoma, sans-serif;font-size:large;font-weight:bold;"><span style="font-size:19px;">第四章 HTTP認證</span></div><div style="font-family:Verdana, Arial, Tahoma, sans-serif;font-size:9.5pt;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpClient提供對由HTTP標準規范定義的認證模式的完全支持。HttpClient的認證框架可以擴展支持非標準的認證模式,比如NTLM和SPNEGO。</div><h3>4.1 用戶憑證</h3><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">任
何用戶身份驗證的過程都需要一組可以用于建立用戶身份的憑據。用戶憑證的最簡單的形式可以僅僅是用戶名/密碼對。
UsernamePasswordCredentials代表了一組包含安全規則和明文密碼的憑據。這個實現對由HTTP標準規范中定義的標準認證模式是
足夠的</p><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">UsernamePasswordCredentials creds = new UsernamePasswordCredentials("user", "pwd");</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println(creds.getUserPrincipal().getName());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println(creds.getPassword());</div></blockquote><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">輸出內容為:</p><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">user</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">pwd</div></blockquote><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">NTCredentials是微軟Windows指定的實現,它包含了除了用戶名/密碼對外,一組額外的Windows指定的屬性,比如用戶域名的名字,比如在微軟的Windows網絡中,相同的用戶使用不同設置的認證可以屬于不同的域。</p><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">NTCredentials creds = new NTCredentials("user", "pwd", "workstation", "domain");</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println(creds.getUserPrincipal().getName());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println(creds.getPassword());</div></blockquote><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">輸出內容為:</p><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">DOMAIN/user</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">pwd</div></blockquote><h3>4.2 認證模式</h3><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">AuthScheme接口代表了抽象的,面向挑戰-響應的認證模式。一個認證模式期望支持如下的功能:</div><ul><li>解析和處理由目標服務器在對受保護資源請求的響應中發回的挑戰。</li><li>提供處理挑戰的屬性:認證模式類型和它的參數,如果可用,比如這個認證模型可應用的領域。</li><li>對給定的憑證組和HTTP請求對響應真實認證挑戰生成認證字符串。</li></ul><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">要注意認證模式可能是有狀態的,涉及一系列的挑戰-響應交流。HttpClient附帶了一些AuthScheme實現:</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;"><ul><li>Basic(基本):Basic認證模式定義在RFC 2617中。這個認證模式是不安全的,因為憑據以明文形式傳送。盡管它不安全,如果用在和TLS/SSL加密的組合中,Basic認證模式是完全夠用的。</li><li>Digest(摘要):Digest認證模式定義在RFC 2617中。Digest認證模式比Basic有顯著的安全提升,對不想通過TLS/SL加密在完全運輸安全上開銷的應用程序來說也是很好的選擇。</li><li>NTLM:NTLM是一個由微軟開發的優化Windows平臺的專有認證模式。NTLM被認為是比Digest更安全的模式。這個模式需要外部的NTLM引擎來工作。要獲取更多詳情請參考包含在HttpClient發布包中的NTLM_SUPPORT.txt文檔。</li></ul><h3>4.3 HTTP認證參數</h3><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">有一些可以用于定制HTTP認證過程和獨立認證模式行為的參數:</div><ul><li>'http.protocol.handle-authentication':定義了是否認證應該被自動處理。這個參數期望的得到一個java.lang.Boolean類型的值。如果這個參數沒有被設置,HttpClient將會自動處理認證。</li><li>'http.auth.credential-charset':定義了當編碼用戶憑證時使用的字符集。這個參數期望得到一個java.lang.String類型的值。如果這個參數沒有被設置,那么就會使用US-ASCII。</li></ul><h3>4.4 認證模式注冊表</h3><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpClient使用AuthSchemeRegistry類維護一個可用的認證模式的注冊表。對于每個默認的下面的模式是注冊過的:</div><ul><li>Basic:基本認證模式</li><li>Digest:摘要認證模式</li></ul><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">請注意NTLM模式沒有對每個默認的進行注冊。NTLM不能對每個默認開啟是應為許可和法律上的原因。要獲取更詳細的關于如何開啟NTLM支持的內容請看這部分。</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;"><h3>4.5 憑據提供器</h3><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">憑
據提供器意來維護一組用戶憑據,還有能夠對特定認證范圍生產用戶憑據。認證范圍包括主機名,端口號,領域名稱和認證模式名稱。當使用憑據提供器來注冊憑據
時,我們可以提供一個通配符(任意主機,任意端口,任意領域,任意模式)來替代確定的屬性值。如果直接匹配沒有發現,憑據提供器期望被用來發現最匹配的特
定范圍。</p><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpClient
可以和任意實現了CredentialsProvider接口的憑據提供器的物理代表一同工作。默認的CredentialsProvider實現被稱為
BasicCredentialsProvider,它是簡單的憑借java.util.HashMap的實現。</div><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">CredentialsProvider credsProvider = new BasicCredentialsProvider();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">credsProvider.setCredentials(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new AuthScope("somehost", AuthScope.ANY_PORT),</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new UsernamePasswordCredentials("u1", "p1"));</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">credsProvider.setCredentials(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new AuthScope("somehost", 8080),</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new UsernamePasswordCredentials("u2", "p2"));</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">credsProvider.setCredentials(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new AuthScope("otherhost", 8080, AuthScope.ANY_REALM, "ntlm"),</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new UsernamePasswordCredentials("u3", "p3"));</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println(credsProvider.getCredentials(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new AuthScope("somehost", 80, "realm", "basic")));</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println(credsProvider.getCredentials(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new AuthScope("somehost", 8080, "realm", "basic")));</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println(credsProvider.getCredentials(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new AuthScope("otherhost", 8080, "realm", "basic")));</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println(credsProvider.getCredentials(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">new AuthScope("otherhost", 8080, null, "ntlm")));</div></blockquote><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">輸出內容為:</p><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">[principal: u1]</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">[principal: u2]</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">null</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">[principal: u3]</div></blockquote><h3>4.6 HTTP認證和執行上下文</h3><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpClient
依賴于AuthState類來跟蹤關于認證過程狀態的詳細信息。在HTTP請求執行過程中,HttpClient創建2個AuthState的實例:一個
對于目標主機認證,另外一個對于代理認證。如果目標服務器或代理需要用戶認證,那么各自的AuthState實例將會被在認證處理過程中使用的
AuthScope,AuthScheme和Crednetials來填充。AuthState可以被檢查來找出請求的認證是什么類型的,是否匹配
AuthScheme的實現,是否憑據提供器對給定的認證范圍去找用戶憑據。</p><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">在HTTP請求執行的過程中,HttpClient添加了下列和認證相關的對象到執行上下文中:</p><ul><li>'http.authscheme-registry':AuthSchemeRegistry實例代表真實的認證模式注冊表。在本地內容中設置的這個屬性的值優先于默認的。</li><li>'http.auth.credentials-provider':CookieSpec實例代表了真實的憑據提供器。在本地內容中設置的這個屬性的值優先于默認的。</li><li>'http.auth.target-scope':AuthState實例代表了真實的目標認證狀態。在本地內容中設置的這個屬性的值優先于默認的。</li><li>'http.auth.proxy-scope':AuthState實例代表了真實的代理認證狀態。在本地內容中設置的這個屬性的值優先于默認的。</li></ul><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">本地的HttpContext對象可以用于定制HTTP認證內容,并先于請求執行或在請求被執行之后檢查它的狀態:</p><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpClient httpclient = new DefaultHttpClient();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpContext localContext = new BasicHttpContext();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpGet httpget = new HttpGet("http://localhost:8080/");</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpResponse response = httpclient.execute(httpget, localContext);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">AuthState proxyAuthState = (AuthState) localContext.getAttribute(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">ClientContext.PROXY_AUTH_STATE);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println("Proxy auth scope: " + proxyAuthState.getAuthScope());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println("Proxy auth scheme: " + proxyAuthState.getAuthScheme());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println("Proxy auth credentials: " + proxyAuthState.getCredentials());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">AuthState targetAuthState = (AuthState) localContext.getAttribute(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">ClientContext.TARGET_AUTH_STATE);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println("Target auth scope: " + targetAuthState.getAuthScope());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println("Target auth scheme: " + targetAuthState.getAuthScheme());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">System.out.println("Target auth credentials: " + targetAuthState.getCredentials());</div></blockquote><h3>4.7 搶占認證</h3><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpClient
不支持開箱的搶占認證,因為濫用或重用不正確的搶占認證可能會導致嚴重的安全問題,比如將用戶憑據以明文形式發送給未認證的第三方。因此,用戶期望評估搶
占認證和在它們只能應用程序環境內容安全風險潛在的好處,而且要求使用如協議攔截器的標準HttpClient擴展機制添加對搶占認證的支持。</p><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">這是一個簡單的協議攔截器,如果沒有企圖認證,來搶先引入BasicScheme的實例到執行上下文中。請注意攔截器必須在標準認證攔截器之前加入到協議處理鏈中。</p><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpRequestInterceptor preemptiveAuth = new HttpRequestInterceptor() {</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">public void process(final HttpRequest request,</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">final HttpContext context) throws HttpException, IOException {</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">AuthState authState = (AuthState) context.getAttribute(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">ClientContext.TARGET_AUTH_STATE);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">CredentialsProvider credsProvider = (CredentialsProvider) context.getAttribute(ClientContext.CREDS_PROVIDER);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpHost targetHost = (HttpHost) context.getAttribute(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">ExecutionContext.HTTP_TARGET_HOST);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">// 如果沒有初始化auth模式</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">if (authState.getAuthScheme() == null) {</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">AuthScope authScope = new AuthScope(</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">targetHost.getHostName(),</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">targetHost.getPort());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">// 獲得匹配目標主機的憑據</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">Credentials creds = credsProvider.getCredentials(authScope);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">// 如果發現了,搶先生成BasicScheme</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">if (creds != null) {</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">authState.setAuthScheme(new BasicScheme());</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">authState.setCredentials(creds);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">}</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">}</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">}</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">};</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">DefaultHttpClient httpclient = new DefaultHttpClient();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">// 作為第一個攔截器加入到協議鏈中</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">httpclient.addRequestInterceptor(preemptiveAuth, 0);</div></blockquote></div></div><h3>4.8 NTLM 認證</h3><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">當
前HttpClient沒有提對開箱的NTLM認證模式的支持也可能永遠也不會。這個原因是法律上的而不是技術上的。然而,NTLM認證可以使用外部的
NTLM引擎比如JCIFS[http://jcifs.samba.org/]來開啟,類庫由Samba[http://www.samba.org
/]項目開發,作為它們Windows的交互操作程序套裝的一部分。要獲取詳細內容請參考HttpClient發行包中包含的
NTLM_SUPPORT.txt文檔。</p><h4>4.8.1 NTLM連接持久化</h4><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">NTLM
認證模式是在計算開銷方面昂貴的多的,而且對標準的Basic和Digest模式的性能影響也很大。這很可能是為什么微軟選擇NTLM認證模式為有狀態的
主要原因之一。也就是說,一旦認證通過,用戶標識是和連接的整個生命周期相關聯的。NTLM連接的狀態特性使得連接持久化非常復雜,對于明顯的原因,持久
化NTLM連接不能被使用不同用戶標識的用戶重用。標準的連接管理器附帶HttpClient是完全能夠管理狀態連接的。而邏輯相關的,使用同一
session和執行上下文為了讓它們了解到當前的用戶標識的請求也是極為重要的。否則,HttpClient將會終止對每個基于NTLM保護資源的
HTTP請求創建新的HTTP連接。要獲取關于有狀態的HTTP連接的詳細討論,請參考這個部分。</p><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">因為NTLM連接是有狀態的,通常建議使用相對簡單的方法觸發NTLM認證,比如GET或HEAD,而重用相同的連接來執行代價更大的方法,特別是它們包含請求實體,比如POST或PUT。</p><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;"><blockquote style="background-color:#F9F9FF;"><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">DefaultHttpClient httpclient = new DefaultHttpClient();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">NTCredentials creds = new NTCredentials("user", "pwd", "myworkstation", "microsoft.com");</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">httpclient.getCredentialsProvider().setCredentials(AuthScope.ANY, creds);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpHost target = new HttpHost("www.microsoft.com", 80, "http");</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">// 保證相同的內容來用于執行邏輯相關的請求</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpContext localContext = new BasicHttpContext();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">// 首先執行簡便的方法。這會觸發NTLM認證</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpGet httpget = new HttpGet("/ntlm-protected/info");</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpResponse response1 = httpclient.execute(target, httpget, localContext);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpEntity entity1 = response1.getEntity();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">if (entity1 != null) {</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">entity1.consumeContent();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">}</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">//之后使用相同的內容(和連接)執行開銷大的方法。</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpPost httppost = new HttpPost("/ntlm-protected/form");</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">httppost.setEntity(new StringEntity("lots and lots of data"));</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpResponse response2 = httpclient.execute(target, httppost, localContext);</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">HttpEntity entity2 = response2.getEntity();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">if (entity2 != null) {</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">entity2.consumeContent();</div><div style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;">}</div></blockquote></div><p style="color:inherit;font-family:inherit;font-size:inherit;font-style:inherit;"> </p></div></div></div>
> 資源參考 http://www.cnblogs.com/loveyakamoz/archive/2011/07/21/2112804.html
- Introduction
- 爬蟲相關技能介紹
- 爬蟲簡單介紹
- 爬蟲涉及到的知識點
- 爬蟲用途
- 爬蟲流程介紹
- 需求描述
- Http請求處理
- http基礎知識介紹
- http狀態碼
- httpheader
- java原生態處理http
- URL類
- 獲取URL請求狀態
- 模擬Http請求
- apache httpclient
- Httpclient1
- httpclient2
- httpclient3
- httpclient4
- httpclient5
- httpclient6
- okhttp
- OKhttp使用教程
- 技術使用
- java執行javascript
- 網頁解析
- Xpath介紹
- HtmlCleaner
- HtmlCleaner介紹
- HtmlCleaner使用
- HtmlParser
- HtmlParser介紹
- Jsoup
- 解析和遍歷一個HTML文檔
- 解析一個HTML字符串
- 解析一個body片斷
- 從一個URL加載一個Document
- 從一個文件加載一個文檔
- 使用DOM方法來遍歷一個文檔
- 使用選擇器語法來查找元素
- 從元素抽取屬性,文本和HTML
- 處理URLs
- 示例程序 獲取所有鏈接
- 設置屬性的值
- 設置一個元素的HTML內容
- 消除不受信任的HTML (來防止XSS攻擊)
- 正則表達式
- elasticsearch筆記
- 下載安裝elasticsearch
- 檢查es服務健康