Zoom (zoom. us) is a widely used online conference software. I believe you must have experienced or used it in various scenarios such as office, conference and chat. As a mature commercial software, zoom provides stable real-time audio and video call quality, as well as common functions such as whiteboard, chat, screen sharing and PPT projection. But how can real-time audio and video lag behind in the era when browsers become mainstream? Compared with Zoom, which needs to install the package, developing a similar conference software directly on the web page will certainly attract more attention. When a meeting is needed, people can access and start the meeting through a link. Now, we can easily turn your idea into reality by using the Qiniu real-time audio and video Web SDK.
First of all, let's sort out what key points need to be broken for a Web online conference product
- Browser compatibility is good, and most mainstream desktop browsers need to be supported.
Qiniu real-time audio and video is based on the WebRTC protocol promoted by Google on Chrome. At present, this protocol has been officially written into the Web standard, and all modern browsers have good compatibility with it.
- Good call quality, low latency, high definition
Unlike traditional WebRTC, which uses the form of user to user P2P for communication, we use nodes deployed worldwide as real-time interactive networks with low latency to communicate with each end through P2P, which ensures both the latency and the quality of calls.
- The conference function should be rich, including ppt demonstration, whiteboard, screen sharing, etc
Our SDK provides a rich list of functions to meet the needs of most conference scenarios. In theory, we can use the SDK to completely reproduce a Web version of the zoo.
- Having said so, is it difficult to access? Are there examples and documentation?
of course! Now you can experience the Demo of our Web (open it in the desktop browser). https://demo-rtc.qnsdk.com 。 Demo The source code of is also open source on Github for your reference https://github.com/pili-engineering/QNRTC-Web
This Demo implements the functions directly provided by most SDKs. The Demo integrating whiteboard/PPT sharing/chat and other scenarios is still preparing to go online. Please wait. We will briefly introduce the specific process of access below. Detailed instructions and references can be moved to our document station https://developer.qiniu.com/rtn/sdk/4412/description-of-web-sdk
Development process
A simple conference product usually goes through the following process:
- User registration/login (developers integrate by themselves, and the SDK only needs to distinguish user userIDs)
- Create a conference room/join a conference room
- Collect your own camera/microphone data
- Publish the collected media data to the room
- Subscribe to other people's media data in the room and play it in real time
- Processing user joining/leaving, publishing/unpublishing
Here we simplify to the various functions of the SDK. In fact, it is adding rooms - collecting local media streams - publishing media streams - subscribing to media streams - event processing. The SDK simply encapsulates each step and can be completed with a few lines of code.
Introducing SDK
It is recommended to use npm to import our SDK directly npm i pili-rtc-web You can, or you can choose to directly import the packaged js file https://github.com/pili-engineering/QNRTC-Web/blob/master/Release/pili-rtc-web.js
Asynchronous processing
Real time audio and video is a strongly asynchronous scene. All kinds of operations are asynchronous related to the network, so that developers can better control the asynchronous logic in the code writing process. The SDK does not use the cumbersome callback mode, but uses the async/await or promise features of modern Javascript to write asynchronous code, so as to avoid callback hell during development (all the await code below is assumed to be wrapped in an async function).
Join Room
After preparation, the first step is to join the room. It is said to join a room. The abstract expression is "what user joins what room in what identity". There are three unknowns: user ID, identity ID (permission), and room ID. In fact, there are still many unknowns in the whole process of adding rooms, such as which APP the room belongs to (the application is independent of rooms in different applications), which Qiniu account the APP belongs to, and so on. Here we uniformly code and sign these values, and turn them into a roomToken to provide to the front end. The end can join the room only through this token. (Token generation can be completed on the Qiniu console, or it can be dynamically generated using the service SDK as needed)
const myRTC = new QNRTC.QNRTCSession(); // initialization await myRTC.joinRoomWithToken(ROOM_TOKEN); // Join Room
Collect local media stream
Generally, audio and video will be collected at the same time, that is, microphone and camera. However, the SDK also supports pure audio collection or pure video collection mode according to requirements. Calling methods is also very simple, just change the options.
Const DOM_ELEMENT=...//The dom element on the page is prepared to play the stream //Collect local stream const localStream = await QNRTC.deviceManager.getLocalStream({ video: { enabled: true }, audio: { enabled: true }, }); //Play the collected stream localStream.play(DOM_ELEMENT)
Publish Media Stream
Directly take the stream object just obtained as the contest, and call the publish method
await myRTC.publish(localStream);
Subscribe to media streams
After joining the room successfully, you can access the user status of the current room at any time by visiting the users member. If there are users in the room who are publishing but not themselves, then we can initiate a subscription
const users = myRTC.users; users.forEach(async (user) => { if (user.published && user.userId !== myRTC.userId) { //Stream data returned by other users in the subscription room const remoteStream = await myRTC.subscribe(user.userId); //Similarly, calling play directly can play the stream remoteStream.play(DOM_ELEMENT); } });
event processing
The SDK exposes a rich event list to meet the needs of most scenarios. The event processing is also very simple. Take the event "published by other users" as an example
//Listen for events myRTC.on('user-publish', handleUserPublish); //Listen only once myRTC.once('user-publish', handleUserPublish); //Cancel a listening function myRTC.off('user-publish', handleUserPublish); //Cancel all listening functions myRTC.removeAllListeners('user-publish');
The specific event list can be referred to https://developer.qiniu.com/rtn/sdk/4423/the-event-list-web
Features
In addition to these basic functions, the SDK also provides many powerful advanced functions to further meet the needs of all walks of life.
Screen sharing
In addition to collecting cameras directly, the SDK also supports screen acquisition (or window acquisition) to share screens for meetings. It also supports seamless switching between screen sharing and camera capture to ensure user experience.
//Screen sharing await QNRTC.deviceManager.getLocalStream({ screen: { enabled: true }, audio: { enabled: true }, });
Live broadcast to push
For an online meeting, only about ten people may participate in the meeting discussion, but most people need to watch the meeting in real time (but not participate in the real-time discussion). This is the intersection of real-time audio and video and live broadcast. A small number of users with real-time interaction needs are allocated to the real-time audio and video cloud with ultra-low latency (200ms), and most users with only real-time viewing needs are allocated to the live broadcast cloud with low latency (2-3 s), which can minimize costs to meet the needs. At the same time, after the real-time video stream is pushed to the live cloud, you can use the API of Qiniu live cloud to store the stream data in various storage spaces for long-term storage. A complete business process has been completed from real-time interaction to live viewing to file storage (for on-demand, etc.).
First, associate the corresponding live video cloud space on the console page of the real-time audio and video cloud, and then turn on the switch of combining and pushing.
If you want to push to a customized RTMP address (instead of using Qiniu Live Cloud), you can also configure it through the backend API of the real-time audio and video cloud (see the document for details)
The next job is to use the SDK. It is also very simple to use the SDK to enable live broadcast forwarding. After adding a room, you can call a line of code.
MyRTC.setDefaultMergeStream (WIDTH, HEIGHT);//The width height here corresponds to the merging output size set above
Using this code, the SDK will, by default, average the layout of all streams in the room, and finally push them to the target RTMP address for merging and forwarding. If you want to customize the screen layout, you can use the following API:
MyRTC.setMergeStreamLayout ("target user ID", {w: 100, h: 100, x: 0, y: 0, muted: false, hidden: false });
In addition, we also provide a real-time whiteboard service for web pages. Like Zoom, users can share a whiteboard on the page with other users in the room for auxiliary presentation. At the same time, whiteboards also support PPT and PDF presentations. Demo of this section can visit our bullock class experience https://edu-demo.qnsdk.com 。
The above is just a list of common function scenarios of a conference software. By integrating these basic functions with the user's own scenarios, a simple conference software can be easily completed. If you are ready to try and experience it, it would be a good choice to visit our Demo (see above). If you plan to integrate your product into our real-time audio and video, here is a more detailed development practice of real-time audio and video applications https://developer.qiniu.com/rtn/sdk/5043/rtc-application-development-process , from From html/css to js, from each line of code to each function, there are detailed explanations and examples to help you access quickly.
