The road to the promised land.
For more than 6 years, we have been working on and looking forward to a simpler way to build RTC (Real Time Communications) applications on the web. In order for this technology to truly show its value, the major browser vendors needed to show up.
Mobile, mobile, mobile.
Now that Apple has joined the party in earnest, does the technology have the coverage required in order for developers to make good use of WebRTC on mobile devices? Let’s find out.
Until now, in order for WebRTC to work on iOS, we were relegated to wrapping WebRTC code in Objective-C and Swift, in our native iOS apps. Basically, we had to take the Chrome code and build an app that was sent to the app store for approval and wait in line, like all the other chumps (yours truly included). Conversely, on Android we could run much of that same code from our desktop Chrome apps, on the Android device as well, within reason of course.
Now that Safari and Chrome are shipping compatible WebRTC on mobile, we get to reuse the same code, right!? Well, mostly, they are different code bases, after all.
A word about hardware acceleration.
If ubiquitous mobile video is to take off, the battery life of the device has to last more than the length of the 10 minute video call (ok, I am exaggerating a bit, but I think you get the point) and the performance needs to be at least adequate enough to distinguish facial features. My bar is set a little higher, baby steps for now.
Without h/w acceleration the CPU is likely working too hard to encode the local video and decode the inbound video + service the other processes required at the same time. That really means there needs to be hardware onboard the device dedicated to video coding. That in turn means H.264, since there are very few vendors that offer VP8 or VP9 h/w acceleration.
Question: Does this mean that mobile apps written with VP8 will not be able to deliver decent mobile video conferencing?
Answer: No, not at all, but they will likely not be as performant as those taking advantage of hardware acceleration.
Suffice to say that SVC (Scalable Video Coding) usage would be another reason why we need h/w acceleration, but that’s for another day.
Who’s using what?
The majority of desktop and mobile WebRTC apps written today, are using VP8 for video.
Since Apple and Microsoft both use H.264 and Google uses VP8 and H.264 (recently shipped Open H.264 – on the desktop and mobile). Also, many of the Enterprise RTC developers are already on that H.264 bandwagon.
Question: If Apple and Microsoft devices ship with H.264, what is the case with Google Chrome on desktops and android, are they preferencing VP8?
Answer: Chrome for desktop and android now have H.264 native. Many of the Android devices that ship today all have H.264 hardware acceleration onboard. In order to understand which units have H.264 and hardware acceleration, you can run use the Android APIs to pull a list of available codecs, but in the case of WebRTC, you will only get H.264 in Android WebRTC if there is a h/w encoder on the device.
Is H.264 the answer for WebRTC video?
Here is a recent test:
Host 1 – (before joining):
macOS Sierra, Macbook, Safari (Technology Preview 32)
Host 2 (after joining):
Android 7, Samsung 7, Chrome 55
Host 1 (after joining):
According to the Chrome Status page, Chrome for Android should have H.264. So why is the session barfing when trying to set up video? The logs do not lie…
Safari – offer:
Chrome on android – answer:
Err, huh? No H.264 in reply?
So, I updated to latest Chrome on android (58) and tried again…
… et voilà!!
Next topic, paying the man!
Shipping your product with H.264 enabled, means you may potentially need to deal with the MPEG-LA royalty police for H.264 royalties, but there are some grey areas.
In the case of Apple and Microsoft, where H.264 royalties are already being paid for by the parent vendor, the WebRTC developer is riding on the coattails of papa bear, at least in theory.
Cisco’s generous OpenH.264 offer means that those using this binary module, can do so at potentially no cost:
We will not pass on our MPEG-LA licensing costs for this module, and based on the current licensing environment, this will effectively make H.264 free for use on supported platforms.
Q: If I use the source code in my product, and then distribute that product on my own, will Cisco cover the MPEG LA licensing fees which I’d otherwise have to pay?
A: No. Cisco is only covering the licensing fees for its own binary module, and products or projects that utilize it must download it at the time the product or project is installed on the user’s computer or device. Cisco will not be liable for any licensing fees incurred by other parties.
That seems to mean (I am no lawyer) every developer shipping WebRTC apps supporting Open H.264 binary module, get a free ride. Those using some other binary, or shipping the above source code for that module, could be on the hook for those royalties. That said, since there are royalties being paid by parent vendors where devices are shipping H.264 anyways, developers may not get hassled regardless.
So what did we learn here?
- Apple has joined the party, now we have a full complement of browser vendors!
- If you want to leverage WebRTC video to deliver a ubiquitous mobile and desktop experience for your users, you should likely consider including both H.264 and VP8.
- VP8 is (still) free and powers most of the WebRTC video out there today.
- You can make use of the Open H.264 project and get a free H.264 ride, albeit baseline AVC.
- WebRTC on Android does not support software encoding of H.264, so unless there is local hardware acceleration, H.264 will not be in the offer.
- H.264 is not fully enabled (or buggy) in Chrome 55 (I was using it on Samsung S7 Edge (Android 7), but it does work with Chrome 58.
- WebRTC is not DOA!
- SDP still sucks and ORTC can’t come soon enough!!
As a side note, it would be interesting to see something like this open sourced; VP8 / H.264 conversion without transcoding, if only to service the existing desktop apps currently running VP8 <-> mobile H.264. It would likely overwhelm the mobile device, but it would be cool if it worked!
Disclaimer: The views expressed by me are mine alone and do not necessarily represent the views or opinions of my employer.
The first ORTC API reference draft has been published as a report in the ORCA W3C Community Group today. This is the first step, one of many, in helping the WebRTC community understand the benefits of an object based API for WebRTC. It is still early but we do hope this web-centric approach will be taken seriously by all looking to the future of WebRTC. We welcome any and all feedback.
Robin Raymond, Chair of the ORCA Community Group – will be speaking on RTC Identity models at IIT RTC in Chicago next week. If you are attending please check the agenda for the correct timing of this panel. Robin is always up for some good conversation around ORTC, Federated Identities & Open Peer.
We wanted to get this initial informational document out right away so everyone that has any interest would be able to review the concepts and comment before we get too far along on the actual API.
“No SDP Offer/Answer” is a concept that has been bantered about for some time. It continues to rear its ugly head within the IETF and W3C Working Groups. There is evidence that some major browser vendors will not back the current SDP O/A methodology at all, but some of us continue to pin our hopes and dreams on SDP O/A nonetheless.
We are told by the developers we need something that will not only pass muster with them but will also be something that can be extended and innovated upon, uninhibited by large browser vendors or any vendor for that matter.
According to many, real adoption of WebRTC will not happen if we continue to force everyone to use this SDP Offer/Answer methodology. It is clearly blocking our way forward and the amount of specification documentation remaining needed for the browser vendors to produce a compatible SDP based WebRTC engine in a browser is much more daunting than most are willing to admit.
It’s time for a change.
What does that mean? It means we need to rewrite a portion of the current WebRTC specification and WebRTC API, which also means that many of the demo apps running out there will need some level of modification. Which is O.K. It’s better to break a few demo apps now than to proceed knowing there is a material flaw in the design, precluding the rest of the web from playing along and causing untold pain in future development.
Long live WebRTC.
I just received this email from Skype’s PR firm…
Here is Skype’s official comment regarding Skype for Asterisk. You can attribute this to Jennifer Caukin, spokeswoman for Skype.
“Skype made the decision to retire Skype for Asterisk several months ago, as we have prioritized our focus around implementing the IETF SIP standard in our Skype Connect solution. SIP enjoys the broadest support of any of the available signaling alternatives by business communications equipment vendors, including Digium. By supporting SIP in favor of alternatives, we maximize our resources and continue to reinforce our commitment to delivering Skype on key platforms where we can meet the broadest customer demand.”
Call me crazy but if I have to pay to integrate Skype into my phone system, where I already have a phone service that I am happy with, why would I do that? Maybe I just want to be able to make/receive Skype calls on my SIP-enabled desk phone? If it doesn’t hit the PSTN why do I have to pay? Seems like an odd approach for a company that has a long history of working around POTS, much to the delight of their users.
Integration with SIP is great, don’t get me wrong, but it would be nice if Skype talked SIP and was ‘still’ free. Seems like a massive oversight on behalf of Skype or am I missing something?
Imagine a new secure P2P (Skype like) offer that also supported SIP in the client. You could use the client software on it’s own (just like Skype) or attach it to just about any VoIP service or phone system for free.
Does it make sense for consumers?
Does it make sense for business users?
Is there room in the market?
Would you use it?
Martyn Davies chimes in…
I would use it, but as a telecom industry insider, I know that I’m not the average business user or consumer. As to whether there is room in the market, I think that depends a lot on what Microsoft do with Skype now that they own it. From a business point-of-view, their efforts are focused around OCS/Lync (and software licenses), so Skype there is not adding to their central proposition. Skype has a lot of users, but produces very little revenue, since the majority just use the free services. As a Skype competitor you would have the same problems getting to the cash.
Skype was really the first company to take VoIP and make it completely trivial to install and use. To do that, they had to take some liberties and deviate from standards (like SIP), so that they could add the magic that made it work from behind firewalls, add security and self-configuration, and integrate video so seamlessly. Like Facebook, once it is clearly the biggest of its kind of services, it becomes the community that everyone must join. I can’t see that another Skype-alike has a way in, unless Microsoft significantly change the rules now.
The RTCWEB Bof meeting starts in less than 5 minutes. To some, the most anticipated meeting of IETF 80.
Proposed charter (truncated)…
The WG will perform the following work:
1. Define the communication model in detail, including how session management is to occur within the model.
2. Define a security model that describes the security goals and how the communication model can achieve these goals.
3. Define how NAT and Firewall traversal is to occur.
4. Define which RTP functions and extensions that shall be supported in the client and their usage for real-time media, including media adaptation to ensure congestion safe usage.
5. Define what functionalities in the solution, such as media codecs, security algorithms, etc., that can be extended and how the extensibility mechanisms works.
6. Define a set of media formats that must or should be supported by a client to improve interoperability.
7. Define how non RTP datagram and byte stream data communication between the clients can be done securely and in a congestion safe way.
8. Provide W3C input for the APIs that comes from the communication model and the selected components and protocols that are part of the solution.
Aug 2011 Architecture and Security and Threat Model sent to W3C
Aug 2011 Use cases and Scenarios document sent to W3C
Sept 2011 Architecture and Security and Threat Model to IESG as Informational
Sept 2011 Use cases and Scenarios for RTCWeb document sent to IESG as Informational
Dec 2011 RTCWeb and Media format specification(s) to IESG as PS
Dec 2011 Information elements and events APIs Input to W3C
Apr 2012 API to Protocol mapping document submitted to the IESG as Informational (if needed)
Well, I am a little sad that I have to turn ON international mobile roaming with Bell in order to get my mobile phone working here, which it is still not, but all is not lost. I have been using FaceTime over the WifI on my MacBook Air and iPhone 4 to call my business partner and my wife back home. Kinda fitting actually, FaceTime is standards based and is all about SIP and RTP etc. Now if we can just get them to open up that API…