處理瀏覽器中的媒體 · PHP/Python/前端/Linux 等等學習筆記

[TOC] > [參考](https://a-wing.github.io/webrtc-book-cn/02_handling-media-in-the-browser.html#webrtc-%E7%9A%84-10-%E4%B8%AA%E6%AD%A5%E9%AA%A4) ## 三個邏輯組： 1. 本地和遠程音頻和視頻的獲取和管理： `MediaStream` 界面（以及 HTML5 `<audio>` 和 `<video>` 標簽的相關用法） 2. 連接管理： `RTCPeerConnection` 接口 3. 管理任意數據： `RTCDataChannel` 接口 ## WebRTC 的 10 個步驟 1. 從本地設備（如麥克風、網絡攝像頭）創建一個`MediaStream`對象。 2. 從本地`MediaStream`獲取*URL Blob* 3. 使用獲取的*URL Blob*進行本地預覽 4. 創建一個`RTCPeerConnection`對象 5. 將本地流添加到新創建的連接 6. 將你自己的會話描述發送到遠程對等點 Send your own session description to the remote peer. 7. 從您的對等方接收遠程會話描述 Receive the remote session description from your peer. 8. 處理收到的會話描述，并將遠程流添加到您的`RTCPeerConnection` 9. 從遠程流獲取*URL Blob* 10. 使用獲取的*URL Blob*播放遠程對等方的音頻和/或視頻 ## 媒體捕獲及數據流 ![](https://a-wing.github.io/webrtc-book-cn/assets/img/rcwr_0201.9228fb61.png) 由一個視頻軌道和兩個音頻軌道組成的`MediaStream` ### 獲取本地多媒體內容 `getUserMedia(constraints, successCallback, errorCallback)` ### URL `createObjectUrl()`方法指示瀏覽器創建和管理與本地文件或二進制對象（blob）關聯的唯一URL： ``` createObjectURL(stream) ``` 它在 WebRTC 中的典型用法是從`MediaStream`對象開始創建*Blob URL*。然后，將在 HTML 頁面內使用*Blob URL*。實際上，本地和遠程流都需要此過程 ## 使用 getUserMedia() API <details> <summary>index.html</summary> ``` <!DOCTYPE html> <html> <head> <title>getUserMedia very simple demo</title> </head> <body> <div id="mainDiv"> <h1><code>getUserMedia()</code> very simple demo</h1> <p>With this example, we simply call <code>getUserMedia()</code> and display the received stream inside an HTML5 <video> element</p> <p>View page source to access both HTML and JavaScript code...</p> <video autoplay></video> </div> <script language="JavaScript"> // API method: // Opera --> getUserMedia // Chrome --> webkitGetUserMedia // Firefox --> mozGetUserMedia navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia; var constraints = {audio: false, video: true}; var video = document.querySelector("video"); function successCallback(stream) { // 賦值給window,可供瀏覽器在console 輸入 steam 檢查 window.stream = stream; video.srcObject=stream; video.play(); } function errorCallback(error) { console.log("navigator.getUserMedia error: ", error); } navigator.getUserMedia(constraints, successCallback, errorCallback); </script> </body> </html> ``` </details> <br /> ## 媒體模型瀏覽器提供了從源(sources)到接收器(sinks)的媒體管道(pipeline)。在瀏覽器中，**接收器**是`<img>`，`<video>`和`<audio>`標記。 **來源**可以是物理網絡攝像頭，麥克風，來自用戶硬盤驅動器的本地視頻或音頻文件，網絡資源或靜態圖像 ### 媒體約束約束是一項可選功能，用于限制`MediaStream`軌道源上允許的可變性范圍。約束通過`Constrainable`接口在軌道上公開，該接口包括用于動態更改約束的 API getUserMedia() 調用還允許在首次獲取軌道時應用一組初始約束（例如，設置視頻分辨率的值） ### 使用約束 >[warning] WebRTC 瀏覽器中的 `getUserMedia()` 約束支持 <details> <summary>調節分辨率 index.html</summary> ``` <!DOCTYPE html> <html> <head> <title>getUserMedia very simple demo</title> </head> <body> <div id="mainDiv"> <h1><code>getUserMedia()</code>: playing with video constraints</h1> <p>Click one of the below buttons to change video resolution...</p> <div id="buttons"> <button id="qvga">320x240</button> <button id="vga">640x480</button> <button id="hd">1280x960</button> </div> <p id="dimensions"></p> <video autoplay></video> </div> <script lang="JavaScript"> // API method: // Opera --> getUserMedia // Chrome --> webkitGetUserMedia // Firefox --> mozGetUserMedia navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia; var constraints = {audio: false, video: true}; var video = document.querySelector("video"); var vgaButton = document.querySelector("button#vga"); var qvgaButton = document.querySelector("button#qvga"); var hdButton = document.querySelector("button#hd"); var qvgaConstraints = { video: { mandatory: { maxWidth: 100, maxHeight: 100 } } }; // Constraints object for standard resolution video var vgaConstraints = { video: { mandatory: { maxWidth: 200, maxHeight: 200 } }, }; // Constraints object for high resolution video var hdConstraints = { video: { mandatory: { maxWidth: 300, maxHeight: 300 } }, }; // Associate actions with buttons: qvgaButton.onclick = function () { getMedia(qvgaConstraints) }; vgaButton.onclick = function () { getMedia(vgaConstraints) }; hdButton.onclick = function () { getMedia(hdConstraints) }; function getMedia(constraints) { if (!!stream) { video.src = null; } navigator.getUserMedia(constraints, successCallback, errorCallback); } function successCallback(stream) { // 賦值給window,可供瀏覽器在console 輸入 steam 檢查 window.stream = stream; video.srcObject = stream; video.play(); } function errorCallback(error) { console.log("navigator.getUserMedia error: ", error); } navigator.getUserMedia(constraints, successCallback, errorCallback); </script> </body> </html> ``` </details> <br />