The Eternal WebRTC Video Codec Debate – Consensus Finally!

But despite another 12 months of progress, one of the key points of contention that remained stubbornly unresolved was the great video codec debate: Should VP8 or H.264 be the Mandatory-to-Implement Video Codec for WebRTC? It was a welcome and surprising move that led the IETF Working Group to finally arrive at the following consensus just last week:

• WebRTC Browsers must implement both VP8 and H.264.
• WebRTC Non-Browsers/Devices must implement both VP8 and H.264. If compelling evidence arises that one of the codecs is available for use on a royalty-free basis then only that codec can be picked.
• WebRTC-compatible endpoints are free to implement any video codec they see fit.

Apparently, all it took was an IETF WG trip to Hawaii, a few Mai Tais and a bit of synchronized humming.  In all seriousness, this is a decision which we are really pleased with.  It echoes the pragmatic position that TokBox decided upon earlier this year, and is a decision which could not have waited any longer.

We wrote about the MTI codec problems almost a year ago. Since February of this year, the TokBox position has been: 

1. For web browsers (wherever they are implemented – PCs, tablets, phones, TVs, what have you), both VP8 and H.264 should be MTI video codecs for WebRTC. Offer order preference is up to the browser, but the browser must support (and offer) both.

2. For any other WebRTC endpoint (ie. a non-browser endpoint), the MTI video codec for WebRTC is either VP8 or H.264. Choice of which video codec to implement is left to the endpoint, but it must implement H.264 and/or VP8.”

We are glad to see the IETF WG decision reflects a close compromise to the position we took a year ago, and still hold today. Regardless of individual issues with specific codecs, what we have always wanted first and foremost was to move the WebRTC standard forward, and along with it to see applications continuing to flourish.

Moving WebRTC forward requires not a perfect solution but a practical solution.  

While not everyone will be happy with this decision, and it certainly introduces a series of follow-up questions, the MTI consensus is one we believe is right for the progression of WebRTC. It strikes a balance between the ideal and the possible, but we are not out of the woods just yet.

What’s good:

• Refocused energy. It’s time we stopped quibbling and moved on. There are plenty of other issues outside of the codec debate that need to be resolved in order for WebRTC to move forward. Open issues include RTP Usage, the need for efficient topologies, simulcast and WebRTC 1.1 APIs.

• Practical compromise. The entire codec debate has raged on for 3 years (what now feels like eternity), and has frankly exasperated a bunch of smart and dedicated people. We all owe huge thanks to Adam Roach for stepping in and echoing the sentiments of the community in his last minute appeal that struck a practical compromise and moves the standard forward. For over two years we have seen first-hand what developers need to successfully use WebRTC in the real world: an ecosystem that enables them to innovate and push the boundaries of applications being built. At the end of the day, WebRTC is just a technology. What creates the magic are the applications.

• Mobile device support.  A majority of mobile devices being shipped today by OEMs have H.264 hardware chipsets. Mandating H.264 along with VP8 as an MTI codec enables application developers to leverage the best codec available to them to deliver an optimal experience, whether they are concerned about quality, battery life or performance. Finally, with iOS8 Apple has opened up APIs that enable developers to directly access the underlying hardware encoder/decoders – a huge step forward. On other devices, we are starting to see VP8 hardware acceleration available as well. Essentially, the choice to pick the most optimal codec rests in the hands of application developers – which is exactly where it should be.

• WebRTC compatible end-points. The MTI consensus introduces this notion of WebRTC compatible end-points that can implement their codec of choice. Whether it’s a watch, a wearable, or any other device, this opens up the world of browser interoperability where people building applications on these end-points now have access to the WebRTC ecosystem. This is particularly interesting and will help break down walled gardens which exist today between the WebRTC and non-WebRTC RTC world.

• Future-proofing. People groan and moan about multiple codecs but at the end of the day we believe it’s right for the industry. It encourages the standards bodies to think about interesting and important problems such as dynamic re-negotiation, discovery, etc. It gets vendors thinking about performance and optimization of codecs and it helps application developers deliver the best possible experience in their applications. It also paves the way to vendors being more open to new and novel codecs which might be developed. Regardless of the outcome of the H.265 vs VP9 discussion, we will at least have fall-back options.

• Moving towards a more open world. Frankly, everyone is getting tired of the industry dancing around the question of truly open technologies and standards. MPEG-LA has been dragging its feet, continuing to burden H.264 with royalties for real-time software use cases, while pending claims against VP8 have yet to be resolved. If compelling evidence arises that one of the codecs is available for use on a royalty-free basis, that one codec can be picked. This is good for innovation and the industry as a whole, pushing everyone to reach for a truly open codec, unencumbered by liabilities/royalties, and accelerating the momentum of both adoption and innovation.

• Hardware acceleration.  The MTI consensus leaves us in an ambiguous world for endpoints that don’t have access to H.264 hardware chipsets. Here the burden of licensing & liabilities falls to the application developer. Application providers have to decide whether to bear the burden of H.264 licensing/royalties, which we do not believe is practical for most applications. Financial considerations will likely lead to most applications using VP8, which gives them browser interoperability without introducing royalty issues. We wonder if this will lead to more prevalent access to H.264 hardware support or provide an opportunity for VP8 based chipsets to proliferate on mobile devices.

But as encouraging as the MTI consensus is, there are still important questions about how the various browser vendors will actually comply to the standard.

What’s still unclear:

• Effects on the WebRTC Ecosystem. The MTI consensus has an interesting bearing on the businesses that service the WebRTC ecosystem. It raises questions such as: How important is transcoding? Does a multi-codec world change the architectural assumptions of platform and service providers?  Platforms providers like TokBox will now need to support increasingly complex cloud capabilities across both codecs, and perhaps across a mix of codecs simultaneously. In a world of increasingly sophisticated media processing (eg. archiving multiple streams from a single video conversation), this raises the stakes for platform and service providers both in terms of investment for new feature development, and ongoing maintenance costs for existing end-points.

• Google. It remains to be seen about when Google with ship H.264 support in Chrome. They are actively working on VP9, posting this update just today. We anticipate that they will accelerate VP9 in light of their desire to move to more a more open and high-performance codec world.

• Apple. In true Apple form, they have been characteristically silent (for the most part)  about the whole issue of WebRTC. H.264’s inclusion as an MTI may help them edge closer to the standard, but the question remains – will Apple ship support for WebRTC in Safari in a standards compliant manner? Apple did raise an issue at the IETF WG about not being able to work with VP8 pending type 3 IPR claims, laying what appears to most onlookers as the groundwork for ongoing justification of delays.

• Microsoft. Microsoft’s announcement to support Object-RTC (ORTC) API and H.264 in IE (https://status.modern.ie/webrtcobjectrtcapi) is welcome news. VP8 support in IE in a standards-compliant fashion would be essential for the sake of clean interoperability, but just like Apple, we’re not holding our breath.

• Legacy devices. There is substantial capital locked up in legacy devices and network infrastructure, both of which are largely based on the H.264 codec. It remains to be seen if the Telco-Industrial complex comprised of hardware vendors, carriers, device manufacturers and associated ISVs truly embrace VP8 for non-browser end-points. Mere WebRTC “compatibility” might not be the right answer in some cases.

In recap:

• The consensus decision reached by the IETF WG has been a huge (and timely) leap forward in the right direction.
• But in order for this consensus to be considered a success, browser vendors need to take this decision seriously and not continue to perpetuate a fragmented, pot-holed world of WebRTC endpoints.
• So it’s your move, browser vendors! As discussed above, with Mozilla having moved first, the real question now falls to Google, Apple, and Microsoft to “do the right thing” for the developers, the industry, and the future of real-time communications.

We laud the IETF WG for coming together (and Cisco, Mozilla, and Google for supporting the proposal) and reaching a practical position. So while this development is promising for the most part, while leaving open some significant questions for service providers and application developers, we fundamentally believe it is good for the community as a whole. The future of WebRTC depends on browser/endpoint vendors to do the right thing; uphold the IETF decision by delivering standards-compliant support in a timely manner to drive adoption, accelerate innovation, and move the standard forward at a faster rate.