I want the web to work the way it was intended, using only 100% free software. Is that so much to ask? Apparently so – and almost exclusively due to Flash.

Flash. I have concluded long ago that it’s impossible to have a good web browsing experience with or without it, so you might as well protect your freedom and go without it. As a GNU/Linux user, it presents so many problems. Even if using the proprietary player was acceptable, it is plagued by bugs such as the famous “Linux Flash plugin is unresponsive to mouse clicks” issue that Adobe doesn’t even seem to acknowledge. There are various workarounds, but that’s not the point. Then there’s the issue of 64-bit GNU/Linux distributions typically bundling 64-bit versions of the Firefox web browser. Too bad Flash dropped support of the 64-bit Flash plugin while it was still in beta, leaving users with an unsupported version with known security vulnerabilities. [Update: Seems Adobe changed their minds again! Most people still hellbent on using Flash have already had to switch away from the unsupported beta by now, so what is the point?]

Want to know what really gets on my nerves? 99% of Flash content that I am actually interested in is just video. The stuff that Firefox and Chrome has included native support for since over a year ago, via Xiph.Org‘s Ogg Theora/Vorbis formats. Heck, even Opera finally included support for these free video formats early this year. Those three browsers alone account for over 40% of web browser usage worldwide. Of course, Microsoft doesn’t want anything to do with free software, and Apple generally tries to pretend it doesn’t exist wherever convenient to do so. Since the majority of web browser usage does not include video support natively but does include the Flash plugin, for a lot of websites Flash is the easy fix. This of course forced more people to use Flash, which caused more websites to use it, which caused more people to use Flash… you get the idea. Even though Flash has been responsible for a huge number of system compromises, people feel forced to use it anyway.

W3C recognized the need for an open standard so that anyone could play video on the web regardless of Adobe’s approval. When HTML 5 was being drafted, VP8 was proposed as the video codec. Why VP8, when three of the five major browsers had native Theora support already? The answer to that was of course; video quality. Everyone was whining that Theora wasn’t as high in picture quality for H.264 and everyone wanted video quality to be as nice as possible. Due to H.264 being unusable (being encumbered with with patents), Google generously purchased On2 Technologies who created the wonderful VP8 codec and released the codec as a truly free format. As it is the highest quality open-spec free-of-patents codec which anyone can use, this paved the way for W3C to give it serious consideration.

Unsurprisingly, Microsoft made it clear that they would not support a free format. Period. Microsoft doesn’t need to provide a reason – given their normal attitude towards open standards or anything that would benefit free software or competition of any kind, rejecting the proposal was a given. Historically Microsoft deliberately doesn’t follow standards (eg. IE6, MS Office… anything really), so having the suits at Microsoft disagreeing with it was completely expected. Still if everyone else supported the standard, and with IE’s popularity continuing to fall, this might be enough to either force Microsoft’s hand, or make IE basically irrelevant – eventually.

There’s one other (somewhat) common browser – Safari. Apple’s response to VP8? Utter BS – patent FUD, and bogus hardware claims. Apparently a lot of Apple hardware has dedicated H.264 decoder chips (presumably iPods, iPhones and such), which Apple seems to suggest can be used by H.264 exclusively. I don’t believe it. Considering how similar H.264 and VP8 actually are, you’d think a company like Apple would be able to make it work. Anyway, Apple comes out with new iWhatevers every year, and Apple provides basically no support for older devices. Last I checked (a few months back – don’t have a link), there were various vulnerabilities in the 1st generation iPod Touch models which Apple has no intention of fixing. It was only superseded by the 2nd generation just on 2 years ago. That’s right – if you brought your iPod Touch around 2 years ago, Apple wouldn’t care about you today. Due to this forced rapid upgrade cycle, it should be no problem at all to have Apple get hardware support into all its devices relatively quickly – after all, we’re talking about the company that got Intel to redesign them a smaller CPU to fit their MacBook Air. If Apple can boss Intel around to get chips working the way they want, they likely can with any hardware company.

As for the patent FUD, Apple claims that H.264 hasn’t been proven not to infringe on patents in court. Steve Jobs famously claims that all video codecs are covered by patents. If this actually were true – that it was impossible to create a video codec without stepping on a patent, the patents in question would surely have to be invalidated by being obvious or demonstrating prior art. Either way, Apple’s talking trash. The real reason for rejecting VP8 is surely for the same reason as Microsoft – so they can keep themselves from being on a level playing field with their most direct browser competitors. Mozilla, Google and Opera won’t pay for MPEG-LA patent licenses on a per-user basis since the browsers can be copied and distributed to anyone without charge – and there would be no way to track the licenses anyway. Even if (for example) the Mozilla Foundation did find a way to overcome these obstacles, what of projects that fork Mozilla? Mozilla is free software. If all derivatives weren’t covered, Firefox wouldn’t be free anymore. If they were covered, any project would never have to pay the MPEG-LA again since they could just opt to borrow the Mozilla code – it would be a licensing deal that the MPEG-LA would never agree to. Clearly, the future of video on the web cannot possibly depend on paying patent licenses.

So where does this leave us? I predict that if HTML5 does not specify a format to use for the video tag, we’ll continue to see Flash dominate as the preferred video decoding option by website owners for many years into the future. Couldn’t we just dump Flash already and have the Microsoft fanboys install the Xiph.Org Directshow Filters package (which apparently comes with support for IE’s <video> tag)? That could work in a lot of cases, however if it really took off you could be sure that Microsoft would find a way to “accidentally” break the plugin somehow. It wouldn’t be the first time. I recall Microsoft IE 5.5 beta (if I’m remembering my version numbers correctly) would prevent executable files named hotdog.exe from working. This kind of file name was commonly used for Sausage Software’s HotDog HTML editor installation program – direct competition to Microsoft FrontPage back in the day. Rename the file to setup.exe and you were in business – not easy to do when the file came on a CD. Microsoft could potentially just argue that the incompatibility was only in its beta software, but web developers would likely have installed it.

Getting back on track… <cough>.. if the future of web video is in Flash, what can we do about it? How can we play our video using 100% software? We’re not out of options yet. Adobe has announced that upcoming versions of Flash will support VP8! How does that help us? If webmasters want to reach as close to 100% of their audience as possible right now, H.264 is the best option. As much as I hate it, H.264 can be played back via Flash on 90+% of desktops. Encoding in a second format to reach users that don’t have Flash installed might not be cost effective when time and storage costs are considered. However when Flash supports VP8, everyone can adopt that format and not need to worry about encoding in H.264 as well. People without Flash but using Firefox, Chrome or Opera can gracefully fall back to watching video natively. That way, the website video will work on all free-software-only desktops. Video numbers can be still further improved by updating the free software Java applet player video player Cortado to add WebM support. This would be a combination that would likely get us as close to 100% compatibility as reasonably possible using only a single codec.

There are some reasons why this could fail. Perhaps a percentage of IE users that don’t have Flash, Java or have the Directshow Filters plugins installed (but can play video natively due to having IE9 or later) will be larger than the number of GNU/Linux desktop users. I expect this to be very unlikely. However if H.264 remains the only option for iPhone-style devices, that might help tilt the scales in H.264’s favor. Another problem is that a lot of video recording devices such as webcams and some digital camcorders record to H.264 natively. It might be more efficient for the website maintainer to keep video in that format (even if heavy edits are required). Fortunately most web videos are so short that transcoding times probably won’t matter… but it’s a minor concern.

But what about playback today using entirely free software? Flash sucks on GNU/Linux! Enter Gnash and FFmpeg. The latest version (0.8.8 at the time of writing) works with YouTube 99% as well as Flash on Windows. Other video sites… not so much. In particular, I still have problems with Gnash when I try to play AVGN and Zero Punctuation – but I have a solution for these as well – the gecko-mediaplayer plugin with Greasemonkey. Once those are installed, grab the Greasemonkey scripts Download Escapist Videos and Gametrailers No Flash. You also will want to install rtmpdump. With those all installed, when you want to check out Zero Punctuation simply click the Download link that now appears under the video. Gecko MediaPlayer will kick in and give you video that takes up the entire browser window. As for AVGN, I discovered that GameTrailers hosts all the ScrewAttack content which includes many of the AVGN videos. Simply head on over to the ScrewAttack section – the latest video should be towards the top. Note that you have to sign in for the script to work, but basically it just takes the Download link and streams it to Gecko MediaPlayer, which gets embedded in the area that Flash normally resides. It works perfectly.

So there’s a lot of hacks involved. Gnash is buggy, and FFmpeg might have patent issues depending on the codec and your country. The AVGN solution involved finding an alternative (but still non-free possibly patent-infringing) format. Lastly, the Zero Punction hack basically involved a stream ripper, Gecko MediaPlayer and (probably) FFmpeg too. This is ugly as hell, but it works. When it does the first time, it’s a wonderful feeling. Unfortunately if you want native WebM in Firefox you need to upgrade to Firefox 4 beta, and today’s Greasemonkey checkout still has serious compatibility issues (although it’s being actively worked on). When Greasemonkey works natively in Firefox 4 and both projects release a stable build (expected this year), things will be looking very nice… and I imagine Gnash will get better in the meantime. YouTube is also testing encoding videos to WebM format, so hopefully they keep that up and encourage other video sites to follow suite. All systems are go!