WebRTC,名称源自网页即时通信(Web Real-Time Communication)的缩写,是一个支持网页浏览器]进行实时语音对话或视频对话的API。它于2011年6月1日开源并在Google、Mozilla、Opera支持下被纳入万维网联盟的W3C推荐标准。
WebRTC实现了基于网页的视频会议,标准是WHATWG 协议,目的是通过浏览器提供简单的javascript就可以达到实时通讯(Real-Time Communications (RTC))能力。WebRTC项目的最终目的主要是让Web开发者能够基于浏览器轻易快捷开发出丰富的实时多媒体应用,而无需下载安装任何插件,Web开发者也无需关注多媒体的数字信号处理过程,只需编写简单的Javascript程序即可实现,W3C等组织正在制定Javascript 标准API,目前是WebRTC 1.0版本,Draft状态;另外WebRTC还希望能够建立一个多互联网浏览器间健壮的实时通信的平台,形成开发者与浏览器厂商良好的生态环境。同时,Google也希望和致力于让WebRTC的技术成为HTML5标准之一,可见Google布局之深远。
WebRTC提供了视频会议的核心技术,包括音视频的采集、编解码、网络传输、显示等功能,并且还支持跨平台:windows,linux,mac,android。
客户端
创建WebRTC主要分为以下几个步骤
创建RTCPeerConnection实例 -> 发送candidate -> 创建offer -> 交换offer -> 添加媒体流
客户端主要工作简单来看就是三个交换,交换ice、交换offer、交换流。通过代码来看:
创建WebRTC实例
去除浏览器前缀转换通用变量
const RTCPeerConnection = window.mozRTCPeerConnection || window.webkitRTCPeerConnection;
const IceCandidate = window.mozRTCIceCandidate || window.RTCIceCandidate;
const SessionDescription = window.mozRTCSessionDescription || window.RTCSessionDescription;
navigator.getUserMedia = navigator.getUserMedia || navigator.mozGetUserMedia || navigator.webkitGetUserMedia;
创建WebRTC peer实例,它接收一个配置选项configuration
,其中配置了用来创建连接的 ICE 服务器信息。
const configuration = {
// 内网穿透使用sturn和turn服务信息,下面的服务器国内无法使用,需要自行搭建,👉见后文
iceServers: [
{urls: "stun:23.21.150.121"},
{urls: "stun:stun.l.google.com:19302"},
{urls: "turn:numb.viagenie.ca", credential: "webrtcdemo", username: "louis%40mozilla.com"}
]
}
const pc = new RTCPeerConnection(configuration);
// 当 ICE 框架找到可以使用 Peer 创建连接的 “候选者”,会立即触发一个事件(onicecandidate)。ICE候选者信息通过信令服务器转发
pc.onicecandidate = function (e) {
if (!e.candidate) return;
// 通过信令服务器转发candidate 接收方通过addIceCandidate方法添加
send("icecandidate", JSON.stringify(e.candidate));
};
// ...
function addIce(candidate){
// 信令服务器响应后 将远端的candidate加入本地ice
pc.addIceCandidate(new RTCIceCandidate(candidate));
}
ICECandidate
ICE全称Interactive Connectivity Establishment:交互式连通建立方式。ICE参照RFC5245建议实现,是一组基于offer/answer模式解决NAT穿越的协议集合。它综合利用现有的STUN,TURN等协议,以更有效的方式来建立会话。客户端侧无需关心所处网络的位置以及NAT类型,并且能够动态的发现最优的传输路径。简单来说 发起方创建RTCPeerConnection实例后要将自己的candidate发送给接收方,双方互换candidate才可建立链接。
// 添加ice
pc.addIceCandidate(new RTCIceCandidate(candidate))
交换offer SDP
Offer SDP 是用来向另一端描述期望格式(视频, 格式, 解编码, 加密, 解析度, 尺寸 等等)的元数据。一次信息的交换需要从一端拿到 offer,然后另外一端接受这个 offer 然后返回一个 answer。
// 创建offer
await pc.createOffer({
// 是否接收音频
offerToReceiveAudio: true,
// 是否接收视频
offerToReceiveVideo: true
});
// 添加至本地描述
await pc.setLocalDescription(offer);
// ... 将本地的offer通过信令服务器转发给接收方
// 接收方收到远端的offer 创建自己answer 并发送给主叫
await recvPc.setRemoteDescription(offer);
await recvPc.setLocalDescription(await recvPc.createAnswer());
// ...将answer发送给主叫
// ...接收到被叫方返回的answer信息添加到setRemoteDescription
pc.setRemoteDescription(answer);
添加流发送给对方
在接收到offer时我们就可以获取本地的媒体流,并添加到RTCPeerConnection实例中
//获取媒体流 如果只想使用默认配置video直接赋值为true
window.navigator.mediaDevices.getUserMedia({video: {
// 帧数
frameRate: { ideal: 10, max: 15 },
// 宽高 最大1280 最小800 ideal(应用最理想的)值时,这个值有着更高的权重,意味着浏览器会先尝试找到最接近指定的理想值的设定或者摄像头(如果设备拥有不止一个摄像头)。此外还要facingMode 前置或者后置摄像头
width: { max: 1280,ideal: 1024,min:800 },
height: { max: 720,ideal: 960,min:600 }
},audio:true})
.then(mediaStream => {
console.log('addStream');
// 添加流到RTCPeerConnection实例中
this.pc.addStream(mediaStream);
// 把本地的流添加到video标签中播放
$('.my-video').srcObject = mediaStream;
});
// 监听onaddstream
pc.onaddstream=function(e){
// e.stream
$('.your-best-friend-video').srcObject = e.stream;
}
信令服务器
为客户端交换offer交换ice的中转站,使用SpringBoot创建一个简单的WebSocket服务,如果想要在公网上必须要用wss,因为客户端用的是https协议无法与ws协议连接
STURN/TURN中继服务器
客户端在局域网环境下想要与外网的机器通讯,需要NAT网络穿越。
- coturn是谷歌开源的中继服务器应用,集合了sturn 和 turn。
- NAT两种类型,对称NAT与非对称NAT:
非对称NAT,每次请求对应的IP端口是不对应的(海王) 使用sturn就可以完成打洞穿越。
对称NAT,每次请求对应的IP端口是对应的(5201314),打洞穿越难度大 需要使用turn服务器中继。
通过一个场景解释:客户端向sturn服务器发请求,获得自己的对外ip和端口,此时客户端是对称NAT,这个对外暴露的端口只能给服务器使用,被叫客户端无法使用主叫暴露的端口导致无法连接。如果是非对称的,sturn服务器获得主叫对外暴露的IP和端口,告诉其他客户端,其他客户端就可以拿着这个ip和端口进行连接。当然非对称也有限制,具体分为三种。
- sturn和turn区别:
sturn帮助客户端寻址,客户端请求sturn服务器,返回给客户端对外暴露的ip和端口,使客户端可以相互寻址,并p2p连接turn 采用中继的方式,占用资源大
完整代码
信令服务器SpringBoot
/**
* WebSocket业务类
* 此类用来作为RTC的信令服务器
*/
@Service
@ServerEndpoint("/websocketRTC")
public class WebSocketRTC {
private static Vector<Session> sessions = new Vector<>();
private static Vector<JSONObject> sessionProduce = new Vector<>();
private static TreeMap<String,Session> sessionTreeMap = new TreeMap<>();
private static int loginNumber = 0;
private Session session ;
private static final org.slf4j.Logger log = org.slf4j.LoggerFactory.getLogger(WebSocketRTC.class);
/**
* 响应一个客户端WebSocket连接
* @param session
* @throws IOException
*/
@OnOpen
public void onopenproc(Session session) throws IOException {
System.out.println("hava a client connected");
this.session = session;
JSONObject open = new JSONObject();
open.put("status", "success");
sendMessageToClient(open.toJSONString(), session);
}
/**
* 响应一个客户端的连接关闭
* @param session
*/
@OnClose
public void oncloseproc(Session session){
System.out.println("had a client is disconneted");
// sessionTreeMap.remove(data);
}
/**
* 对于客户端消息的处理和响应
* @param message 客户端发送的消息
* @param session 客户端的WebSocket会话对象
* @throws IOException
*/
@OnMessage
public void onmessageproc(String message , Session session) throws IOException {
/**
* 信令服务器与客户端之间的消息传递采用JSON
*/
if(message!=null) {
JSONObject msgJSON = JSON.parseObject(message);
/**
* 消息中的type字段表示此次消息的类型
* 服务器根据消息的type针对性的处理
*/
switch (msgJSON.getString("type")) {
case "login" :{
/**
* 处理客户端登录
*/
log.info("session : "+session + "is login .. "+new Date());
log.info("user login in as "+msgJSON.getString("name"));
if (sessionTreeMap.containsKey(msgJSON.getString("name"))) {
JSONObject login = new JSONObject();
login.put("type", "login");
login.put("success", false);
sendMessageToClient(login.toJSONString() , session);
}else {
sessionTreeMap.put(msgJSON.getString("name"), session);
JSONObject login = new JSONObject();
login.put("type", "login");
login.put("success", true);
login.put("myName", msgJSON.getString("name"));
sendMessageToClient(login.toJSONString() , session);
}
}break;
case "offer": {
/**
* 处理offer消息
* offer是一个peer to peer 连接中的 第一步
* 这个是响应通话发起者的消息
* 这里主要是找到 通话发起者要通话的对方的会话
*/
// onOffer(data.offer, data.name);\
log.info("Sending offer to " + msgJSON.getString("name")+" from "+msgJSON.getString("myName"));
Session conn = sessionTreeMap.get(msgJSON.getString("name"));
if (conn != null) {
JSONObject offer = new JSONObject();
offer.put("type", "offer");
offer.put("offer", msgJSON.getString("offer"));
offer.put("name", msgJSON.getString("name"));
sendMessageToClient(offer.toJSONString(), conn);
/**
* 保存会话状态
*/
JSONObject offerAnswer = new JSONObject();
offerAnswer.put("offerName", msgJSON.getString("myName"));
offerAnswer.put("answerName", msgJSON.getString("name"));
JSONObject sessionTemp = new JSONObject();
sessionTemp.put("session", offerAnswer);
sessionTemp.put("type", "offer");
sessionProduce.add(sessionTemp);
}
}
break;
case "answer": {
/**
* 响应answer消息
* answer是 被通话客户端 对 发起通话者的回复
*/
log.info("answer ..." + sessionProduce.size());
for (int i = 0; i < sessionProduce.size(); i++) {
log.info(sessionProduce.get(i).toJSONString());
}
if (true) {
Session conn = null;
/**
* 保存会话状态
* 查询谁是应该接受Anser消息的人
*/
for (int ii = 0; ii < sessionProduce.size(); ii++) {
JSONObject i = sessionProduce.get(ii);
JSONObject sessionJson = i.getJSONObject("session");
log.info(msgJSON.toJSONString());
log.info(sessionJson.toJSONString());
log.info("myName is " + msgJSON.getString("myName") + " , answer to name " + sessionJson.getString("answerName"));
if (/*i.getString("offerName").equals(msgJSON.getString("name")) && */sessionJson.getString("answerName").equals(msgJSON.getString("myName"))) {
conn = sessionTreeMap.get(sessionJson.getString("offerName"));
log.info("Sending answer to " + sessionJson.getString("offerName") + " from " + msgJSON.getString("myName"));
sessionProduce.remove(ii);
}
}
JSONObject answer = new JSONObject();
answer.put("type", "answer");
answer.put("answer", msgJSON.getString("answer"));
sendMessageToClient(answer.toJSONString(),conn);
}
}
break;
case "candidate": {
/**
* 这个是对候选连接的处理
* 这个消息处理在一次通话中可能发生多次
*/
log.info("Sending candidate to "+msgJSON.getString("name"));
Session conn = sessionTreeMap.get(msgJSON.getString("name"));
if (conn != null) {
JSONObject candidate = new JSONObject();
candidate.put("type", "candidate");
candidate.put("candidate", msgJSON.getString("candidate"));
sendMessageToClient(candidate.toJSONString(),conn );
}
}
break;
case "leave":{
/**
* 此消息是处理结束通话的事件
*/
log.info("Disconnectiong user from " + msgJSON.getString(" name"));
Session conn = sessionTreeMap.get(msgJSON.getString("name"));
if (conn != null) {
JSONObject leave = new JSONObject();
leave.put("type", "leave");
sendMessageToClient(leave.toJSONString(),conn);
}
}
break;
default:
JSONObject defaultMsg = new JSONObject();
defaultMsg.put("type", "error");
defaultMsg.put("message", "Unreconfized command : "+ msgJSON.getString("type") );
sendMessageToClient(defaultMsg.toJSONString(),session);
break;
}
System.out.println(message);
}
}
/**
* 发送消息
* @param msg
* @throws IOException
*/
public void sendMessage(String msg) throws IOException {
if(this.session!=null)
this.session.getBasicRemote().sendText("hello everyone!");
this.session.getBasicRemote().sendText(msg);
}
public void sendMessageForAllClient(String msg){
if(!sessions.isEmpty()){
sessions.forEach(i->{
try {
if(i.isOpen()) {
i.getBasicRemote().sendText(msg+" : "+new Date().toString());
}
} catch (IOException e) {
e.printStackTrace();
}
});
}
}
/**
* 向指定客户端发送消息
* @param msg
* @param session
* @throws IOException
*/
public void sendMessageToClient(String msg , Session session) throws IOException {
if(session.isOpen())
session.getBasicRemote().sendText(msg);
}
}
网页
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>WebRTC 系列文章 一对一视频通话和文字聊天</title>
<style>
body {
background-color: #3D6DF2;
margin-top: 15px;
font-family: sans-serif;
color: white;
}
video {
background: black;
border: 1px solid gray;
}
.page {
position: relative;
display: block;
margin: 0 auto;
width: 500px;
height: 500px;
}
#yours {
width: 150px;
height: 150px;
position: absolute;
top: 15px;
right: 15px;
}
#theirs {
width: 500px;
height: 500px;
}
#received {
display: block;
width: 480px;
height: 100px;
background: white;
padding: 10px;
margin-top: 10px;
color: black;
overflow: scroll;
}
</style>
</head>
<body>
<div id="login-page" class="page">
<h2>Login As</h2>
<input type="text" id="username" />
<button id="login">Login</button>
</div>
<div id="call-page" class="page">
<video id="yours" muted="muted" autoplay ></video>
<video id="theirs" muted="muted" autoplay></video>
<input type="text" id="their-username" />
<button id="call">Call</button>
<button id="hang-up">Hang Up</button>
<input type="text" id="message"></input>
<button id="send">Send</button>
<div id="received"></div>
</div>
<script src="client.js"></script>
</body>
</html>
客户端JavaScript
// 核心的javascript
// 声明 变量 : 记录自己的登录名 , 对方的登录名
var name,
connectedUser;
var myName;
// 建立WebSocket连接 信令服务器
var connection = new WebSocket("wss://119.3.239.168:9443/websocketRTC");
// var connection = new WebSocket("wss://localhost:9443/websocketRTC");
// 自己的RTCPeerConnection
// RTC 最重要的对象
var yourConnection;
// 打开连接事件响应
connection.onopen = function () {
console.log("Connected");
};
// Handle all messages through this callback
connection.onmessage = function (message) {
console.log("Got message", message.data);
var data = JSON.parse(message.data);
switch (data.type) {
case "login":
onLogin(data.success);
break;
case "offer":
onOffer(data.offer, data.name);
break;
case "answer":
onAnswer(data.answer);
break;
case "candidate":
onCandidate(data.candidate);
break;
case "leave":
onLeave();
break;
default:
console.log("default message");
console.log(data);
break;
}
};
connection.onerror = function (err) {
console.log("Got error", err);
};
// 发送消息的方法 向信令服务器
// Alias for sending messages in JSON format
function send(message) {
if (connectedUser) {
message.name = connectedUser;
message.myName = name;
}
connection.send(JSON.stringify(message));
};
// 绑定HTML上的一些标签
var loginPage = document.querySelector('#login-page'),
usernameInput = document.querySelector('#username'),
loginButton = document.querySelector('#login'),
callPage = document.querySelector('#call-page'),
theirUsernameInput = document.querySelector('#their-username'),
callButton = document.querySelector('#call'),
hangUpButton = document.querySelector('#hang-up'),
messageInput = document.querySelector('#message'),
sendButton = document.querySelector('#send'),
received = document.querySelector('#received');
callPage.style.display = "none";
// 登录按钮click事件响应
// Login when the user clicks the button
// 记录登录名,向信令服务器发送登录信息
loginButton.addEventListener("click", function (event) {
name = usernameInput.value;
myName = usernameInput.value;
if (name.length > 0) {
send({
type: "login",
name: name
});
}
});
// 响应信令服务器反馈的登录信息
function onLogin(success) {
if (success === false) {
alert("Login unsuccessful, please try a different name.");
} else {
loginPage.style.display = "none";
callPage.style.display = "block";
// Get the plumbing ready for a call
// 准备开始一个连接
startConnection();
}
};
var yourVideo = document.querySelector('#yours'),
theirVideo = document.querySelector('#theirs'),
// yourConnection, connectedUser, stream, dataChannel;
connectedUser, stream, dataChannel;
// 打开自己的摄像头
// 准备开始一次peer to peer 连接
function startConnection() {
// 想要获取一个最接近 1280x720 的相机分辨率
var constraints = {audio: false, video: {width: 320, height: 480}};
navigator.mediaDevices.getUserMedia(constraints)
.then(function (mediaStream) {
// var video = document.querySelector('video');
yourVideo.srcObject = mediaStream;
if (hasRTCPeerConnection()) {
console.log("setupPeerConnection .. ")
setupPeerConnection(mediaStream);
} else {
alert("Sorry, your browser does not support WebRTC.");
}
yourVideo.onloadedmetadata = function (e) {
yourVideo.play();
};
})
.catch(function (err) {
console.log(err.name + " -- : " + err.message);
});
}
// 创建RTCPeerConnection对象 ,绑定ICE服务器,绑定多媒体数据流
function setupPeerConnection(stream) {
if (yourConnection == null) {
var configuration = {
// "iceServers": [{ "url": "stun:127.0.0.1:9876" }]
"iceServers": [{"url": "stun:119.3.239.168:3478"}, {
"url": "turn:119.3.239.168:3478",
"username": "codeboy",
"credential": "helloworld"
}]
};
yourConnection = new RTCPeerConnection(configuration, {optional: [{RtpDataChannels: true}]});
}
if (yourConnection == null) {
console.log("yourConneion is null");
} else {
console.log("yourConnection is a object")
}
console.log("========================= setupPeerConnection stream ====================================")
// console.log(stream);
// Setup stream listening
yourConnection.addStream(stream);
yourConnection.onaddstream = function (e) {
console.log(e);
// theirVideo.src = window.URL.createObjectURL(e.stream);
theirVideo.srcObject = e.stream;
theirVideo.play();
};
// Setup ice handling
yourConnection.onicecandidate = function (event) {
if (event.candidate) {
send({
type: "candidate",
candidate: event.candidate
});
}
};
// 打开数据通道 (这个是用于 文字交流用)
openDataChannel();
}
function openDataChannel() {
var dataChannelOptions = {
reliable: true
};
dataChannel = yourConnection.createDataChannel("myLabel", dataChannelOptions);
dataChannel.onerror = function (error) {
console.log("Data Channel Error:", error);
};
dataChannel.onmessage = function (event) {
console.log("Got Data Channel Message:", event.data);
received.innerHTML += event.data + "<br />";
received.scrollTop = received.scrollHeight;
};
dataChannel.onopen = function () {
dataChannel.send(name + " has connected.");
};
dataChannel.onclose = function () {
console.log("The Data Channel is Closed");
};
}
// Bind our text input and received area
sendButton.addEventListener("click", function (event) {
var val = messageInput.value;
received.innerHTML += val + "<br />";
received.scrollTop = received.scrollHeight;
dataChannel.send(val);
});
/* function hasUserMedia() {
navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
return !!navigator.getUserMedia;
}*/
function hasRTCPeerConnection() {
window.RTCPeerConnection = window.RTCPeerConnection || window.webkitRTCPeerConnection || window.mozRTCPeerConnection;
window.RTCSessionDescription = window.RTCSessionDescription || window.webkitRTCSessionDescription || window.mozRTCSessionDescription;
window.RTCIceCandidate = window.RTCIceCandidate || window.webkitRTCIceCandidate || window.mozRTCIceCandidate;
return !!window.RTCPeerConnection;
}
callButton.addEventListener("click", function () {
var theirUsername = theirUsernameInput.value;
console.log("call " + theirUsername)
if (theirUsername.length > 0) {
startPeerConnection(theirUsername);
}
});
// 开始peer to peer 连接
function startPeerConnection(user) {
connectedUser = user;
// yourConnection
// Begin the offer
// 发送通话请求 1
yourConnection.createOffer(function (offer) {
console.log(" yourConnection.createOffer");
send({
type: "offer",
offer: offer
});
console.log(" yourConnection.setLocalDescription(offer);");
yourConnection.setLocalDescription(offer);
}, function (error) {
alert("An error has occurred.");
});
};
// 接受通话者 响应 通话请求 2
function onOffer(offer, name) {
connectedUser = name;
console.log("============================================================");
console.log("=============== onOffer (===================");
console.log("connector user name is "+connectedUser);
console.log("============================================================");
var offerJson = JSON.parse(offer);
var sdp = offerJson.sdp;
// 设置对方的会话描述
try {
console.log(" yourConnection.setRemoteDescription ");
yourConnection.setRemoteDescription(new window.RTCSessionDescription(offerJson), function () {
console.log("success");
}
,
function () {
console.log("fail")
});
} catch (e) {
alert(e)
}
// 向通话请求者 发送回复消息 3
yourConnection.createAnswer(function (answer) {
yourConnection.setLocalDescription(answer);
console.log(" yourConnection.createAnswer ");
send({
type: "answer",
answer: answer
});
}, function (error) {
alert("An error has occurred");
});
console.log("onOffer is success");
};
// 通话请求者 处理 回复 4
function onAnswer(answer) {
if (yourConnection == null) {
alert("yourconnection is null in onAnswer");
}
console.log("============================================================");
console.log("================ OnAnswer ============================");
console.log("============================================================");
console.log(answer);
if (answer != null) {
console.log(typeof answer);
}
var answerJson = JSON.parse(answer);
console.log(answerJson);
try {
// 设置本次会话的描述
yourConnection.setRemoteDescription(new RTCSessionDescription(answerJson));
} catch (e) {
alert(e);
}
console.log("onAnswer is success");
};
// 对ICE候选连接的事情响应
function onCandidate(candidate) {
console.log("============================================================");
console.log("================ OnCandidate ============================");
console.log("============================================================");
console.log(candidate);
if (candidate != null) {
console.log(typeof candidate);
}
var iceCandidate;
// try {
var candidateJson = JSON.parse(candidate);
console.log(candidateJson);
iceCandidate = new RTCIceCandidate(candidateJson);
// }catch(e){
// console.log("exception is ")
// console.log(e);
// }
if (yourConnection == null) {
alert("yourconnection is null in onCandidate");
}
// yourConnection.addIceCandidate(new RTCIceCandidate(candidate));
yourConnection.addIceCandidate(iceCandidate);
};
hangUpButton.addEventListener("click", function () {
send({
type: "leave"
});
onLeave();
});
function onLeave() {
connectedUser = null;
theirVideo.src = null;
yourConnection.close();
yourConnection.onicecandidate = null;
yourConnection.onaddstream = null;
setupPeerConnection(stream);
};