Flash Co-Creator Jonathan Gay Responds to Steve Jobs
by Aaron Simpson and Sean McKenzie
Whether or not Apple allows Flash on the iPhone doesn’t really effect us animators. Our Flash animations are enjoyed by the viewer in the same way whether they’re delivered in an FLV, an HTML5 video player or an MPEG-4. But it sure has complicated things, especially for artist using the software for more interactive projects. Animators like Allan Dye (Pocket God) working are on iPhone games have had to go through complicated production methods to translate their Flash animated elements (and the accompanying XML) into the iPhone environment. Sites we love like Newgrounds, which feature primarily SWF-based animations and games, simply aren’t accessible on the growing suite of Apple’s mobile devices.
To shed some light on how this all started, and where it’s left us, we’re going back to the source. We welcome back Jonathan Gay, the iPhone-carrying co-creator of Flash (then called “FutureSplash”), who we interviewed in 2008. Gay left Adobe in 2005, and has since founded and sold an energy management software start-up along with fellow Flash pioneers Robert Tatsumi and Gary Grossman. Below, he starts by addressing some of the criticism Apple Chairman and CEO Steve Jobs has flung at Flash.
AARON SIMPSON: Do you think Flash should be more open, or as Jobs put it – less of a “closed system?”
JONATHAN GAY: It’s a messy question. With any technology, like Flash, the web or the Internet, where there are millions of people who have invested money and energy into making it part of their lives and their businesses, it’s important for there to be a good steward of that technology. The open source and standards body approach is one way for that stewardship to happen but having a good corporate steward of the technology is also a successful model. I believe that Macromedia, followed by Adobe, have done a good job of being stewards of Flash.
It’s disappointing to me that the media is letting Steve get away with dinging Flash on it’s openness while Apple advocates a much more closed model of application development.
The fact that Steve wrote a letter explaining their position suggests how powerful the demand for Flash is from their partners and customers. While I respect a desire to provide a quality experience to customers, it looks to me that keeping Flash out of the iPhone is a simple competitive choice on Apple’s part. Apple wants to displace Flash’s role in video delivery on the Web with the H.264 standard and Apple wants developers to build custom applications for the iPhone and not cross platform applications. Both of these goals support Apple’s business goals driving their closed iPhone application platform but are destructive to openness on the web. In particular, the stranglehold on video technology that the group of large consumer companies that control the MPEG video patent pool have been pushing toward for decades is very destructive to openness.
When we wanted to add video support to Flash, there were two key challenges we had to confront. First, we were afraid that by adding video, we would cause larger video companies like Microsoft, Real Networks and Apple to attack Flash competitively. We took the risk, added video to Flash and made delivering short form video integrated in a web page a good experience. As a result, Flash is a key video technology on the Web. In some ways, I think Steve’s campaign against Flash is finally the competitive response to the success of Flash in the video market by a powerful company that we were afraid of many years ago.
The second challenge was selecting a video codec. We wanted to use the cool new H.264 open standard but Macromedia did not feel they could afford the H.264 license fee. I believe that the capped $5M per year H.264 license fee was similar in scale to the annual Flash engineering budget at the time. The H.264 license fee model is very anticompetitive. H.264 licensing is free for very small users, expensive for medium size companies and inexpensive for very large companies. This model puts the midsize companies who could challenge the dominant companies at a significant competitive disadvantage and is the reason that we implemented the proprietary but affordable On2 codec in Flash instead of the open and expensive H.264 codec. The capped license fee also discourages large companies from building a competitor to H.264 because they can simply pay the capped license fee and know they are managing their patent risk and suppressing their smaller competitors. For example, it would have cost Macromedia $5M per year to add H.264 to Flash but it probably cost Adobe much less to add H.264 because they were probably already paying a substantial fee for their video editing products. You can probably thank the success of Flash video for the fact that streaming H.264 video over the Internet is free for another 5 years. Solving this patent license problem is probably why Google purchased On2. However, if they open source the latest On2 codec as people suspect they want to, it’s much easier to launch a patent lawsuit against them because anyone can inspect the source code. Given the large number of patents in the video space, it may not actually be possible to build an open source codec that does not inadvertently infringe on someone’s patent. The MPEG Licensing Authority solves the problem in a Borg-like way by adding any new patent challengers to their patent pool.
AARON: From your perspective, how much would the Flash player need to evolve before it would meet Steve Job’s strict efficiency standards?
JON: It’s worth noting that Flash was developed on a 66 Mhz 486 which is probably one tenth the speed of an iPhone. So I don’t think there is a fundamental architectural issue. I think the Flash architecture with the binary file format is inherently higher performance than HTML for multimedia.
I think Adobe is doing the right things by evolving the player to take more advantage of hardware acceleration when it is available. In Flash’s early days, we avoided becoming dependent on hardware acceleration because the computers that had hardware acceleration also tended to have fast processors and it was not really our mission to encourage people to create content that would only work well on a fast processor with hardware acceleration. With mobile devices, moving graphics and video processing to customer hardware saves battery power and Adobe is working on this. Generally, I think the iPhone should have plenty of processing power to run existing Flash content with reasonable performance. It’s always important when building media content to keep in mind the constraints of the delivery platform and I think that even without much performance work, Flash Player could deliver a good experience on the iPhone.
I think Steve Jobs is willfully missing a key point with his arguments against Flash. The important reason to put Flash on the iPhone is that millions of developers have invested millions of hours building Flash content in Flash. The Flash content out there in the world is an asset of our society and the people who created it. People built it in Flash because there was no other decent technology from companies like Apple, Microsoft or Real Networks that enabled this kind of content to be created and delivered. To say that all this content should be discarded because Steve Jobs is afraid that people will build Flash content that runs on mobile devices running any operating system instead of building content that will only work on Apple mobile devices is doing a disservice to the efforts of all those individuals. Personally, I think that Flash content will probably outlive iPhone and iPad apps because Flash is designed to deliver media content while the iPhone/iPad development tools are designed to build applications for a specific hardware platform that will be obsolete in 5 or 10 years. Many years ago, we talked about the idea of “Forever Flash.” The idea was that it should be possible to create interactive multimedia content with a lifetime like a famous book, painting, or movie. The content should be able to be part of human history and be able to be preserved for hundreds of years. I’m not sure if that will happened or not, but it’s easier to imagine that cross platform content like Flash can achieve that than mobile applications dependent on a particular operating system.
AARON: As an iPhone user, do you feel like the device would benefit from the addition of Flash?
JON: I do use an iPhone and think it would be great if it could use Flash, although I think that a lot of Flash content on the Web would be awkward to use on the small screen. I suspect that when it’s time to replace my iPhone, my next phone will be an Android phone.
AARON: Well, you can have that bigger screen on an iPad. Have you picked one up yet?
JON: I have not tried an iPad yet. The concept of the device is certainly appealing, but I don’t like the limitations that Apple places on application creation and distribution. The openness of the personal computer and the Web have led to so many great and unexpected things. The iPhone/iPad model certainly has some of the appealing traits of the personal computer market but it also borrows heavily from the business models used by the cell phone, and cable industries. With my computer, I feel like I own it and can do what I want with it. With my iPhone and with an iPad, it’s more like you are leasing a device for a few years until the battery wears out and it’s time to buy the new one. It’s a good business model for Apple, but as a consumer, I like to do whatever small things I can to encourage a world with more openness and hardware designed for a longer lifetime. It really seems to me that the easily scratched displays, easily dented cases and difficult to replace batteries in Apple mobile devices will cause people to replace their devices sooner than they might with a device that is designed for a longer lifetime.
AARON: Do you believe the assertion that Flash causes the majority of Mac crashes?
JON: I don’t know. Flash performance and stability on the Mac has been a problem. Lately, I’ve been experiencing as much slowness and hangs on DHTML pages in Safari as I have with Flash content. Certainly Adobe could have done a better job here, but Apple has also made it more difficult to build a reliable web browser plug-in than Microsoft. There aren’t very many browser plug-ins beyond Flash that people use to help ensure that the plug-in interfaces are reliable and up to date. My main thought here is that any problems that are present can easily be solved by Apple and Adobe working together to make web multimedia reliable for their customers.
AARON: Jobs also jabbed Flash for having a bad security record. Do you think that’s warranted?
JON: I don’t think so. I think Adobe has done a good job on security with Flash. Any Internet software written in C++ is going to have security challenges. It’s inherent in the complexity of the technology.
AARON: Let’s go back a bit. Do you recall your first interaction with an Apple product?
JON: I grew up on Apple products. I got an Apple II with 16K of memory and no disk drive when I was in junior high school. After my success with science fair projects on the Apple II, my parents bought me a Macintosh when it first came out. One of the key innovations with the Macintosh was the strong developer program that Apple created to encourage people to build software for it.
AARON: Your pre-Flash efforts with FutureSplash were aimed at taking advantage of pressure-sensitive screens and pen-based computing. Did you think it would take this long for multi-touch devices like the iPad to hit the mainstream market?
JON: The drawing technology used in the Flash authoring tool was first created for a pen based computer, the EO Personal Communicator. The device had a cell phone option so you could take notes, draw pictures and send faxes from anywhere. It was clunky and expensive but I think if the web had been strong then, the EO might have been successful and we would all have been using pen based tablets for many years now. A web now full of content and the advent of wireless networking are the key technologies that enable this kind of product now.
AARON: And the web is indeed filled with content, much of it programmed in Flash. But with such ubiquity, is Flash a victim of its own success?
JON: Certainly when a technology is successful, it leads to competition and Apple’s attacks on Flash and the work to add media support to HTML5 are a result of that. HTML5 may put some competitive pressure on Flash for video delivery and for interactive applications but for real interactive multimedia, even when HTML5 browsers are widely distributed in a few years, it will still be less capable in many ways than the older versions of Flash that users already have. I don’t see that Flash is a victim in any way; it’s just part of competing in the marketplace.
AARON: Can you elaborate on why you think HTML5 will be “less capable?”
JON: I am not an expert on the HTML5 standard but my understanding is that its goal is to reduce the need for rich media plugins not to eliminate the need. This suggests that it will be good for simpler content but that the more sophisticated media content will still be Flash.
A few issues that may slow the adoption of HTML5:
- Adoption: It will take time for the various browsers to implement the standard and it will take time for users to upgrade their browsers. This process takes years for Internet Explorer.
- Tools and Frameworks: It will take time to develop good tools for rich media in HTML. A tool is really important when building rich media content.
- Developer Skills: With any new platform, it takes time for developers to learn the platform and invent the techniques to make it shine. This process typically takes a few years with a new media technology.
- Browser Differences: HTML is a complex standard. Will HTML5 work exactly the same in all browsers? Will they support the same codecs? Will they render pixels in the same positions? Will some browsers support a subset of the spec? Will they have different bugs? All of these factors can be handled in simple content with some extra work, but can quickly get out of control for complex content.
- Performance: Flash is optimized for rendering and animating very complex vector graphics and lots of bitmaps on the screen. Will all the browsers perform as well as Flash at this? Odds are that they will all be slower because they have to support the HTML document model as well as the new media model. And odds are that performance differences between the browsers will make content harder to build.
- Synchronization: Flash has a simple and powerful model for synchronizing animation and sound and making sure all the media assets are available when they are needed. HTML5 has a more complex and I believe less capable model here. Because media assets will be spread across multiple files instead of compiled into a single file, it is complex and difficult to ensure that all assets are loaded when needed unless you want to start any animation until everything is loaded.
- Codecs: As I understand it, there are still questions about what video codecs browsers will support. Mozilla does not want to have to pay patent license fees for H.264 and Microsoft and Apple don’t want to risk patent liability by supporting an open source codec. Both are reasonable positions but they present a difficult conflict to resolve.
At the end of the day, it will take time to work through these issues and, in many cases, it will be significantly more expensive to develop HTML5 content than Flash content because developers will have to deal with browser differences, do a lot of learning at first and re-implement any content they already have in Flash. This is a expensive multi-year process and its not really clear what benefit site owners would gain from this transition. It will cost them more money than Flash content. There will be uneven customer experiences until developers get it all figured out and the browser implementations mature. It helps you understand why Apple feels that they need to avoid Flash on the iPhone to try and help catalyze this transition.
My rule of thumb is that if you want to displace an incumbent technology, you need to offer a very large benefit. HTML5 will offer better integration of media if you just want to add media to a web application you are already building in HTML, but if your goal is to build media content for the Internet, I think it will be more expensive to build and a worse experience for your customers. How many sites want that?
AARON: Do you think Apple’s exclusion of Flash could initiate a rapid decline of the software’s penetration and popularity?
JON: No. Flash is well established and has survived lots of competitors over the years. Years ago, we were afraid that DHTML, VRML, Java and other technologies would displace Flash but it’s actually hard to build a good multimedia platform and it takes a long time for developers to build skills in a new platform. In many ways, all the talents, skills and techniques in the millions of Flash developers are a more important asset than the technology in the player and tools. Like all products, Flash will decline some day but I don’t see that iApps or HTML5 have the capabilities to displace the core value of Flash anytime soon.
I don’t think there is any technical basis for Steve’s assertion that Flash is stuck in the PC world. Note that the Unix technology in the iPhone OS is a mini computer technology but it works well in the iPhone. Since Flash was developed, the two fundamental innovations in multimedia technology have been the development of sophisticated scripting engines and the availability of powerful graphics hardware acceleration. Flash has a state of the art scripting engine and Adobe is working on supporting graphics hardware. I think multimedia support in HTML5 will be less capable and lower performance than what Flash offers today. When Steve says Flash is stuck in the PC era he must mean that the Flash business model of free players, open content and affordable technology has been eclipsed by the closed, highly-profitable mobile platform of censored applications that Apple is building with the iPhone.
AARON: Many Flash developers were hoping the App Store would become yet another way to earn money through their games. Do you have a sense of how Flash’s exclusion from the App Store might affect the Flash game developer community?
JON: On one hand, I think it’s fantastic how the App Store has created opportunity for lots of developers but ultimately, I don’t think a closed system like Apple is building can own the mobile applications market. There is simply too much diversity in the marketplace. People buy lots of non iPhone/iPad mobile devices and Apple’s resistance to Flash will probably actually accelerate opportunities for Flash developers on these other devices. Ultimately, I think Apple will be forced by competitive pressure to open up the App Store but it might take a couple of years.
AARON: If Flash is indeed a resource hog, as Apple suggests, do both Flash developers, who could write more efficient code, and Adobe share the responsibility?
JON: As I stated before, the first version of Flash was built on a 66 Mhz 486 PC that was probably one tenth the speed of the iPhone, so clearly it’s possible for Flash to run well on a low power machine. If resources are really the issue, there are some simple things that could be done. Obviously, the first thing to do is to optimize the player and to encourage developers to create content that runs well on all devices. It would also not be hard to have the phone try to run a piece of content and if its starts to take too many resources, degrade its performance. A simple solution would be if a particular Flash movie is taking too much memory or CPU suspend the movie and overlay a resume button over it so the user could choose to spend their precious battery power on accessing a web site they really want to see instead of asking the world to redesign all the web sites that contain Flash.
In general, I think getting bogged down in discussing the details of Apple’s criticisms of Flash misses the point. The technical and performance problems are solvable and there would be real value to Apple’s customers to be able to view web sites that use Flash content. Apple does not want Flash on the iPhone and iPad because it conflicts with their business agenda of drawing a hard line between HTML web applications which are “open” and the closed censored world of iPhone/iPad applications which Apple controls and monetizes. Having Flash, which can build cross device applications that are richer than HTML but not in the Apple sandbox, disrupts this plan. Apple also wants to push video on the web away from Adobe’s control and to H.264 where they and other large companies control the patents. As an outside observer, it looks like there is some “emotional” baggage between Apple and Adobe. I’m not privy to any of that but it would be a shame if Flash developers and Apple customers are losing the opportunity to view content and build their businesses because some executive feels slighted by another.
It’s up to consumers to embrace Apple’s model or to push back against it and decide that they want a more open model like Android provides. Ultimately, I suspect the cell phone carriers will decide that it’s OK for Apple to have a high-end niche, but that they will not give control over their customers to Apple for a really large numbers of devices. For Flash developers, this whole conflict will probably lead to a better Flash implementation on the non-Apple smart phones and more opportunities for delivering Flash content to mobile devices. Apple has momentum in the mobile application market right now, but I think they may lose their mindshare leadership position as consumers and carriers see the benefits of more open models.
At the end of the day, the world benefits if there is a way to create multimedia content that is viewable on a wide variety of devices. Flash does that today in a reasonable and affordable way for hundreds of millions of people. HTML5 may do some of that in the near future, but in the meantime Steve Jobs is telling the world that cross platform multimedia should be non-interactive streaming video which is a loss to all of us.
AARON: With your Adobe and Greenbox experiences behind you, have you set your sites on the next career challenges?
JON: Mostly, I’m taking a break to explore some interests outside the computer world. I enjoy studying and designing systems. Computer software is great because complex systems can be built by an individual. But right now, my interest is in taking some of the systems experience I have from the software world and applying it to more physical systems. I’m exploring a bit of sustainable agriculture. We have a very small business selling grass-fed beef that we raise direct to consumers. I’m also learning a bit about habitat restoration and the messiness and complexity but magical renewal of biological and ecological systems.
AARON: Thanks for your time, Jon.