Tag Archive | h.264

ORTC / WebRTC Pioneers

webrtc_pioneers_award

TMC / WebRTC World & PKE Consulting have published a WebRTC Pioneers press release following a WebRTC Pioneers dinner at WebRTC Expo in Atlanta last week, paying homage to some of the early work being done around WebRTC.

Congratulations to W3C ORTC Community Group founders & core contributors…

Robin Raymond – Hookflash

Bernard Aboba – Microsoft

Justin Uberti – Google

There are however, many names missing from this list who have had a significant impact on early work being done around WebRTC / ORTC. Peter Thatcher (Google), Emil Ivov (Jitsi) & Shijun Sun (Microsoft), Roman Shpount (TurboBridge) and Iñaki Baz Castillo immediately come to mind.

ORTC API Editor’s Draft – Last Call & WebRTC WG

ORTC - Object RTC

The W3C ORTC Community Group has published an Editor’s Draft update to the ORTC API and is also asking for last call comments on the draft…

We are nearing completion of the ORTC specification for our initial ORTC community group draft. We are asking for last call feedback at this time. Some comments to explain some of the changes are still pending but all issues are tracked here and on github. If you would like to make comment, please have a read through and make comment either on this list or as part of our next CG meeting coming up.

There are a few outstanding areas which are “pending” synchronization with WebRTC, e.g. stats, data channel, and IdP. As these have dependencies on WebRTC 1.0, we will attempt to complete the our specification as best we can but those areas will be subject to synchronization should updates come out of the WebRTC community group.

Now is the time to have a read through for final review as the next stage will be to build implementations for implementation feedback.
So what does that mean?

Well, likely the most important takeaway is the fact that this spec now looks to be in a form where it could be implemented. The ORTC Libs (https://github.com/openpeer/ortc-lib) have been in stasis while the CG worked on getting the spec further along. Now that the CG is very close to a public draft on the API spec, that work will resume.

Something else that may be of interest is the direct reference to the W3C WebRTC Working Group in this draft and the issue tracker. There are a few components that require progress in the WebRTC WG before the ORTC CG can implement them. Now there seems to be a direct correlation between the work being done in the ORTC CG and the WebRTC WG, which should really not come as any great surprise to anyone.

At the last meeting, the CG discussed “What is the ORTC CG end game?” The resulting conversation equated to an answer that sounded something like, “Let’s create an implementable API, then if the WebRTC WG is interested in seeing the work being done in the ORTC CG merged into the WG then we should be happy to explore that.”  There was no definite objection apart from, “let’s make sure the timing is right and it makes sense to do that”.

So if you are fuzzy on where the ORTC CG is headed, you should really check out that video clip.

ORTC Sender / Receiver Capabilities Proposal is a big deal for SVC

ORTC Logo

There is a some interesting activity regarding Sender / Receivers on both the WebRTC WG and the ORTC CG.  Robin Raymond of Hookflash (ORTC CG Editor and Chair), submitted this beauty over an hour ago. To me, this is where we begin to see some real benefits offered by an Object model that is not encumbered by SDP O/A.

Hint: SVC (Scalable Video Coding or Scalable Video Codec) becomes the norm, not the exception.

From Robin’s proposal…

Introduction After attempting to write out some use cases using the existing RTCRtpSender and RTCRtpReceiver objects and parameters for ORTC, some issues were discovered. Specifically, the application developer would need to have a fair amount of knowledge on exactly how to tweak low level parameters for anything beyond very simple use cases. For example, setting up an SVC (Scalable Video Codec) would have required knowing about what codecs support SVC, how the layering is setup for particular codecs, and finally setting up specific geometric (or temporal) attributes and layering relationship details by an application developer.

Robin also includes some great SVC use cases..

Alice wants to use a SVC (Scalable Video Codec) to send to Bob

This is for illustration purposes only. Typical benefits of SVC are
greater in conference scenarios rather than traditional point to point
scenarios. However, this scenario can presume that an intermedia
conferencing bridge would be between Alice and Bob.

Current Parameter Based API

Step 1: (Alice)

var senderCaps = RTCRtpSender.getCapabilities();

mySignal(senderCaps);

Step 2: (Bob)

var senderCaps = mysignal();

var receiverCaps = RTPRtcReceiver.getCapabilities();

Read the rest here…

H.264 has won the WebRTC codec debate

H.264 AVC for WebRTC
A few things have happened that lead me to believe that H.264 has already won the WebRTC codec debate, regardless of what was not decided in the IETF RTCWEB or W3C WGs…

 

– Cisco delivers Open Source H.264 Codec via OpenH264.org

– Google and Cisco demonstrate H.264 interop in Chrome

– WebRTC Source Code for H.264 implementation surfaces in the WebRTC Developer mail list

So it looks like Chrome will end up supporting H.264 = they will support both VP8 & 264

Mozilla / Firefox already said they will support both.

Apple seems to be getting more involved, my guess is that they will announce some level of WebRTC support for Safari at WWDC. If they do, they will also favor H.264.

Since ORTC will be compatible with WebRTC to some degree,  IE modern APIs group has indicated that ORTC is “under consideration”. Microsoft also seems to favor H.264.

That covers all the major browsers.

It feels like the MTI Video codec issue is a non-issue.

MTI Video Codec Vote – Jabber will not be counted

Cullen Jennings - 264 Video Comparison

The IETF RTCWEB WG Chairs have not given up on the MTI video codec issue and they seem determined to see this through. Weather you feel that is a good use of our time or not there will be a vote, which is something generally not done in the IETF.

Jabber remote attendance will not be counted this time around, from WG Chair – Cullen Jennings on the subject…

“If there are any people in that category that wish to vote, could they email the chairs with full name and affiliation.”

The Chairs:

Magnus Westerlund <magnus.westerlund@ericsson.com>
Cullen Jennings <fluffy@iii.ca>
Ted Hardie <ted.ietf@gmail.com>

Here is the full note from the chairs today…

The WebRTC ecosystem needs to avoid interoperability failure to grow
optimally.  The RTCWEB working group took on the task of establish Audio
and Video MTI codecs as part of meeting that need. We have not succeeded
in finishing that task for video using normal IETF process, but it is
still important.

We (WG chairs) are proposing that the working group consent to a method
that will establish an MTI, even if the MTI chosen does not have rough
consensus.  We would far prefer the normal IETF process, but it is not
proving workable for this selection.

We initially proposed a method from RFC 3929 (external review team), but
now believe that the working group would not consent to that method.
Instead we are proposing a method that leaves the decision in the hands
of the WG.

The method we propose is based on Instant-runoff voting,
http://en.wikipedia.org/wiki/Instant-runoff_voting, with the
understanding that the choice will be the winner according to the
Instant-runoff voting process.

The steps in the proposed process are these (1-5):

1) Establish a final list of alternatives, based on the WG’s input to
Gonzalo’s email on the 13th of November that requires any additions to
provided by end of the 27th of November.

2) Establish those eligible to vote.  Any participant in the
working group process either electronically or in-person as of yesterday
(20th of November). Who has participated in the Working group process
will be anyone that can be identified from:
– The Blue Sheets for any RTCWEB WG session during an IETF meeting or
an interim meeting since the WG’s creation.
– posting of at least one email to the RTCWEB mailing list

The voter must at time of voting prove their eligibility, by pointing to
the mail archive or a particular blue sheet/meeting. Please verify your
own eligibility.

3) Start the the voting period. Those eligible and willing to vote send
their ballot to a vote collector (Matt Lepinski, former Nomcom chair)
within two weeks using email. The vote collector will check when
receiving a ballot the that the voter is eligible and send a
confirmation email on receiving the ballot. During the balloting period
the vote collector will keep all ballots secret.

Balloting:
– The voter MUST rank ALL alternatives in their ballot from the most
preferred, marked with rank 1, the second most with 2, all the way
to the least preferred marked with rank N.

4) When the voting period is over the ballot collector will publish the
results as well as all ballots, including the voters name to the RTCWEB
WG mailing list. This enables all voting individuals to verify that
their ballot is unmodified. And allows anyone to verify the result of
the vote.

5) The selection is recorded in the drafts.

— End of Process Proposal —

This message initiates the first step in the working group consensus
call process. Namely a one week comment and discussion period for the
above process.

After that week the WG chairs will update, if necessary, the proposal.
Then using the normal IETF process in which anyone is eligible to
participate, the chairs will ask for (rough) consensus to adopt this
extraordinary process to achieve the working group’s stated goals.  The
end date for this consensus call is 2-weeks after the announcement of
the consensus call.

If the working group does not consent to using this extraordinary
process, we will hold a consensus call if the WG can accept
“WebRTC entities MUST support at least one of H.264 or VP8.”.

If there is failure to establish consensus even for this statement, the
chairs conclude that the WG can’t establish what to say about a MTI
video codec.

The WG Chairs

Magnus Westerlund
Cullen Jennings
Ted Hardie

Here are the choices..

The proposed alternatives

The following alternatives has been proposed:

  1. All entities MUST support H.264
  2. All entities MUST support VP8
  3. All entities MUST support both H.264 and VP8
  4. Browsers MUST support both H.264 and VP8, other entities MUST support at least one of H.264 and VP8
  5. All entities MUST support at least one of H.264 and VP8
  6. All entities MUST support H.261
  7. There is no MTI video codec
  8. 5+6, i.e. All entities MUST support H.261 and all entities MUST support at least one of H.264 and VP8
  9. All entities MUST support Theora.
  1. All entities SHOULD support both H.264 and VP8. All entities MUST at least implement one of those.

Entities that do not support both H.264 and VP8 MUST implement H.261.

The deadline to propose additional alternatives are: 27th of November 2013

This may change so keep an eye on – http://trac.tools.ietf.org/wg/rtcweb/trac/wiki/WikiStart

Live Blogging – IETF 88 – MTI Video Codecs

Chair: Taking the discussion back to the list.

Harold: We are witnessing a massive abuse of IETF policies. We have spent months on this.

Long line at the mic now

IMG_3040

IMG_2053

IMG_3706

Consensus Call:

NEITHER CODEC WAS SELECTED

Consensus Call:

VP8 – about 30% of the room

Jabber: 30 for VP8 and 10 for 264

Consensus Call:

H.264 – about half the room

Chair / Ted Hardie:

Regarding the consensus call, raise your hand for either or both but not neither. This is not a call but simply calrification

IMG_2892

Chair / Cullen: Agreed, hugs coming your way.

Martin: Keep in mind there are some that can’t get up and talk about that.

Chair / Cullen: We want to ask if you can’t live with either one of these codecs please get to the mic.

Tim Terryberry: Re: JR we need a royalty bearing codec if webrtc is to take off. I don’t think I need to tell the people in this room how excited everyone is about video in a browser without a plugin, ll of those are VP8.

Justin ? : Risk and Trust. Free codecs sounds good but today that’s not a reality. One of the advantages of going through an SVDO process is that the members must declare IPR, free codecs do not go through that. VP8 is not a free and the wind codec, it’s owned by Google. Google controls all of what VP8 is, can they be trusted. Who has implemented VP8 from the spec, not source.

?? – Technology is for people. The decision is for what we have today, not yesterday. Google has done a great job.

Harold Alvestrand: If people think that this announcement from Cisco of “later to be provided code” then I have real issues with the IETF process. Sooner or later we should find a way forward to rely on un-encumbered patent- less codecs, and if not now, then when?

Eric Rescorla: Mozilla intends to support 264 and VP8. We intend to support 264 via Cisco’s code/plugin on desktop. We will try hard to use the hw codecs since that works better on mobile. The user experience, during the install the codec will be installed (part of the install process). VP8 will be shipped with both mobile and desktop.

Bernard Aboba: It’s ok JR we do love you, hugs later.

Blackberry ?: Like the idea we would have a track towards better quality. The license is a big issue. VP8 is not a specification in the MPEG LA, but since it’s not there we have no mitigation there so we can’t make VP8 the MTI.

Randell Jesup: Those who are not making a browser but instead a server like asterisk or the like, their users would also have to download this plugin codec as well. It’s unclear if this would fall into the personal non-commercial use category. IT is not going to like this either, dealing with plugin codecs will make their lives potentially rather difficult.

JR: Arguing that we must choose a MTI codec or they can’t ship

Robin Raymond: I think it’s great that this new 264 codec is out there but if we make it mandatory the little guys lose. Mobile can’t download a binary, it must be part of the app before deployed. If you need H.264 you can add it if you need it but don’t force that royalty bearing codecs on all of us.

JR: Users don’t need to get plugins, apps do that.

Dean Willis: I don’t build browsers. For the last 7 years I have made my career on IPR consulting. If we were to pick 264 as MTI and since Cisco is not part the MPEG LA there seems to be risk there. Seems to me no choice works as well.

Roman Mahy: The most compelling items were the embedded platforms. My experience it was at one point very hard to find hw accelerated encoding, in the past.

Peter Thatcher: We turned on transcoding for a well known chat service and no one noticed. We are better off allowing the market to decide as Martin indicated months ago.

Justin Uberti: Regarding Adam’s comments regarding someone recording calls does not the understand the current laws. If you want to ship h.264 you can, no one is stopping you but not as the MTI codec.

John Peterson: This Wg has made plenty of decisions re: interop and we have saddled this WG with SIP /OA, but I think we should go with H.264

Roberto ?: The bias seems to be present, that larger companies have an easier time sending people here. If you choose to make 264 the MTI you are cutting out the small guys. I think the best option is to choose neither.

Matt ?: There are plenty of lawsuits against 264 so to say there is no risk with 264 that is simply not true. External plugins are not cool.

Ted Hardie (independent) : I offered JR a hug as he seemed to feel the free 264 effort was not appreciated. One of things we were trying to avoid is a plugin scenario and this (other things were added) is the reason we can’t invoke 264 as the MTI video codec. Royalties will push away numerous developers, big and small. Since 264 is now free (not free from royalty), we can add that codec as a non-mti codec.

JR: Transcoding degrades quality.

Martin Thompson: Let’s stop messing about with semantics, 264 is clearly the correct choice here so let’s get on with it.

Chair / Cullen Response: That was put to the list months ago and no one came forth with a proposal so that is not on the table today.

Jeremy ?: We could also consider both codecs.

Jean ?: Calling Nokia to the mic to clarify why there is a patent claim on VP8 and not on 264.

Peter Thatcher: Seems there are 2 broad categories, H.264 is better and the patent argument. The first is simply perspective and the second the patent being mentioned with Nokia is not resolved and if we are making a decision to implement 264 maybe those making that decision should wait until that is resolved.

Justin Uberti: Interop is misunderstood. Transcoding is cheap so that does not hold in the commercial world. VP8 is shipping in plenty of endpoints. Hardware Codecs are quite a ways off, there is LOTS of works that needs to be done here.

Monty Montgomery: If we go down this 264 path we are staking our future on a binary law environment.

Harold: To Cisco, please publish your findings

Cisco: Certainly

—-

Open Mic

JR / Cisco Conclusion

IMG_1202

JR Arguing Risk Analysis

IMG_5547

JR Arguing Risk Assessment

IMG_3772

JR Arguing Distribution

IMG_2881

JR arguing the results in Germany (re: Nokia VP8 patent being put down in Germany) may or may not be a consideration

JR arguing that VP8 is a target for patent trolls

JR arguing patent risk

IMG_0272

Distribution

IMG_6600

JR arguing both codecs perform nearly the same so let’s get past that

IMG_0263

Justin asking, which of these represent (slide below) actual Real-time encoode/decode

Cisco response: much smaller percentage

IMG_3488

H.264 vs VP8 Deployment & Standard

IMG_9582

H.264 Shipping Date

IMG_4600

Jonathan Rosenberg on H.264 for Cisco is at the mic

IMG_2914

—- that was fast, thanks Harold! —-

Harold is at the mic..

IMG_2913

NOTE: This post may not be entirely accurate. I am not the fastest typist in the world and I ad-lib when I can to get the general gist in the post.

IETF 87 Berlin – RTCWEB Agenda

As we can see, MTI Video codec is not present in the agenda. It seems it was pushed back until the IPR issues surrounding VP8 have come to some sort of conclusion. Maybe we should include both VP8 and H.264 for MTI, that or the industry will do it for us if we wait long enough.

Draft Agenda
RTCWEB IETF 87
Chairs: Magnus Westerlund, Cullen Jennings, Ted Hardie

August 1, 2013
9:00 to 11:30

Should SDES be part of WebRTC security practice and, if so, how?
Presentations: 30 minutes
Discussion: 45 minutes

Post-Plan A/Plan B MMUSIC discussion of impact to RTCWEB documents
Presentation: 30 minutes
Discussion: 30 minutes

Security document updates
Presentation: 5 minutes
Discussion: 10 minutes

August 2, 2013
11:20 to 13:30

Chair review of dependencies in other groups: 10 minutes

Use Case Requirements updates:
https://datatracker.ietf.org/doc/draft-ietf-rtcweb-use-cases-and-requirements/
Presentations:: 15 minutes
Discussion: 15 minutes

Data channel:
https://datatracker.ietf.org/doc/draft-ietf-rtcweb-data-channel/
https://datatracker.ietf.org/doc/draft-ietf-rtcweb-data-protocol/
Issues list presentation: 45 minutes
Discussion: 45 minutes

WebRTC MTI Video Codec: VP8 Patent Holders & License Terms Defined

webm-logo

The plot thickens…

VP8 Patent Cross-license Agreement

Google is in the process of preparing an agreement that will assist companies and developers with the adoption of VP8 technology by making available a royalty-free license to certain patents that are necessary for the implementation of VP8 and which are owned by Google and a number of other major technology companies.

Google is considering licensing these patents pursuant to the terms of the attached draft patent license agreement. This draft is under review by Google and is therefore currently non-binding and does not constitute an offer to license any patents. A formal, binding agreement will be posted to this site in the coming weeks. Thus, the below draft is for informational purposes only.

 

The Primary Licensors are each of the following entities and, subject to any noted exclusions, all of their respective Affiliates (as defined in the VP8 Patent Cross-License Agreement).

CIF Licensing LLC

France Telecom

Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.

Fujitsu Limited

Koninklijke Philips Electronics N.V.

LG Electronics Inc.

Mitsubishi Electric Corporation

MPEG LA, LLC

NTT DOCOMO, INC.

Panasonic Corporation

Samsung Electronics Co., Ltd.

Siemens Corporation

H.264 for WebRTC MTI Codec

H.264 AVC for WebRTC

I have been supporting VP8 as the MTI codec in the WebRTC / RTCWEB WG, for various reasons including…

  • Royalty Free (drives innovation)
  • Open Source (Source code is optimized for RTC and readily available)
  • Supported by a rather large company (Google)

It seems some of us may be changing our minds (myself included), for various reasons, eg…

  • H.264 is prolific in RTC today. If we are to have interoperability with other endpoints out there we need 264.
  • Open Source. H.264 optimized source for ARM and X86 its coming, Cullen said so 🙂
  • Supported by many rather large companies (MPEG-LA for starters)
  • VP8 IPR is coming under heavy fire. Nokia being one firm that has boldly stated that they hold patents on VP8 and will enforce them, and apparently there is at least one more VP8 patent holder out there that is keeping rather quiet, which is rather disconcerting.
  • Since H.264 utilizes hardware acceleration, battery consumption on mobile devices should be lower as well.
  • Quality is arguably better in some cases, this can be somewhat subjective.

So where does this leave the innovators that need free software to create free software? Great question, and here are some potential answers…

  • If there is Open Source H.264 software out there that has been optimized for Real-time Communication (encoding and decoding in real-time) there are no “software” fees (excluding MPEG-LA), that is one less, rather large, obstacle.
  • MPEG-LA (last I looked) has stated that there are no patent fees associated with deployments under 100k endpoints. This is great for smaller deployments but larger deployments could still have a problem here?

Some are saying that the bulk of all mobile devices that could support video have a t least one license for H.264 already, so why then would there be another royalty for H.264 on that very same device? This is an interesting argument and one that may in fact provide the innovative developers with a leg up when it comes time to debate the issue with the MPEG-LA. That being said, most innovative developers I know want nothing to do with legal debates.

There is also another factor to consider, many of the MPEG-LA patent holders are behind WebRTC, so why then would they shoot themselves in the foot and hamper adoption of WebRTC by litigating those who adopt it? There has also been talk of creating a new H.264 license in the MPEG-LA related directly to WebRTC / RTCWEB, which would be free of royalties. Talk is cheap.

Those vendors who only run one codec today will have likely already chosen H.264 for their existing deployments and will likely vote 264 as the MTI codec. At the end of the day, the prudent vendors looking for the most coverage in WebRTC interoperability will support both H.264 and VP8, regardless of which codec is selected as MTI.

After weighing all the options it seems (to me at least) that H.264 is the better choice today, now we just need an open source (with a BSD or MTI license or the like) H.264 implementation that has been optimized for WebRTC.

I am interested in hearing what the developers out there think about this. What do you think? Should H.264 be the MTI (Mandatory to Implement) video codec for WebRTC?

%d bloggers like this: