My friend and fellow Chicagoan Evan Miller wrote an excellent blog post over the weekend: Why I Develop For The Mac. It's full of great reasons why his software (which is also excellent, by the way) was written for the desktop, despite the fact that he's a web developer, even the creator of an Erlang web framework.
But I'm compelled to respond to it, specifically his statements about <canvas>:
large <canvas> areas seem laggy on most browsers
So I'm left with <canvas>, and <canvas> is slow.
I have become intimately familiar with <canvas> while developing Soundslice. I'd even venture to say Soundslice is one of the most advanced uses of <canvas> on the web that's not a tech demo — i.e., it's an application that normal people use. The site uses not one, but nine <canvas> elements stacked on top of each other to make a very rich UI, sort of like Photoshop for guitar tabs. (For a flashy demo of how those canvases interact, watch the tech talk I gave at 37signals, specifically the bit starting at 10:20.)
Here's what I've learned: <canvas> is not slow. In fact, I've been continually surprised by how fast it is — as long as you take care to do things right. Evan's article mentions the "magical" sensation of instantaneous feedback; I invite you to play with the zoom slider on any Soundslice page (example) to experience this same magic, all drawn dynamically with <canvas>.
Of course, <canvas> is certainly not as fast as the lower-level drawing routines that you can use if you develop a desktop app. No question. But it's fast enough that, unless you're doing something relatively insane, you'll be totally fine.
On Soundslice, we're drawing guitar-chord charts completely on the fly (again, see an example), which is a relatively involved drawing routine — and it's still near-instant performance. That's across all modern browsers (Chrome, Safari, Firefox and IE 10).
Here are some specific tips I've picked up to make <canvas> performance really shine.
Use requestAnimationFrame
Above all else, do this.
It's a JavaScript API designed to fix a very specific problem: your computer screen can only be redrawn a certain number of times per second (the "refresh rate"), so any calculations that redraw more often than your refresh rate are wasteful.
For example, say you have an event such as mousemove that results in a <canvas> redraw. A mousemove might happen hundreds of times per second, but your screen might only refresh, say, 75 times per second (75 Hz). That means, if your code is naively written, it will try to redraw several times within each actual opportunity to redraw (hundreds of times per second vs. 75 actual redraw opportunities per second).
The requestAnimationFrame API solves this by letting you say, "Execute this code the next time a redraw happens." Which saves your browser from having to do unnecessary work.
When I added this to Soundslice, the site became dramatically faster and more responsive. Here's more info about how to use the API.
Stack canvases
Above, I linked to a video of a tech talk I gave about Soundslice. In that talk, I demonstrated how Soundslice uses several different <canvas> elements, stacked on top of each other as layers, for maximum performance — and for nice, clean code. Definitely watch the demo at around 10:20 in the video to get a sense of it.
I'm planning to write a separate blog post about this, but the Cliff's Notes version is that you can stack transparent <canvas> elements on top of each other so that you only have to redraw the ones that need to change.
For example, on Soundslice, there's a separate <canvas> for the playhead — the vertical orange line that tracks the currently played moment of the video. That's a separate <canvas> with a z-index above the other ones, so that redrawing it doesn't require redrawing any of the other stuff. The less you have to redraw, the better.
Bunch calls to fillStyle
When you draw on <canvas>, you first have to tell it which color you're using. You can do that by setting the "fillStyle." It turns out that, each time you change the fillStyle, there's a slight performance penalty. Therefore, you can squeeze out some extra performance by bunching your calls to fillStyle — that is, rather than drawing a gray thing, then an orange thing, then a gray thing again, you should draw all the gray things, then draw all the orange things.
For example, Soundslice, which is all about annotating YouTube videos, needs to draw dozens, sometimes hundreds, of annotations on the screen at a time. Each annotation might use several different colors — the text color, the border color, the line color, etc.
My original implementation looped over each annotation and drew each one independently, which resulted in two to five fillStyle calls for each annotation. I changed this to bunch the fillStyles across all annotations — so that all of the light grays were drawn at the same time, then all the dark grays, etc. — and the drawing got a few dozen milliseconds faster.
For more background, see the "Avoid unnecessary canvas state changes" section in this great HTML5 Rocks article.
Cache text rendering
In profiling, I've found that rendering text on <canvas> is my next big rendering-related bottleneck on Soundslice. I haven't done this yet, but I'm planning to come up with a way of caching the results of fillText, possibly using this technique.
Final thoughts
A decent argument in Evan's favor is: "Well, if <canvas> is only fast if you use these various hacks, it's not really fast, then, is it?"
Two thoughts on that.
First, well, sure! I'd love it if <canvas> was super fast right out of the box, without needing to use these techniques. No doubt about it. But the reality is, it is fast enough, if you put in the work.
Second, there's the bigger question — a defining question for the current generation of web developers — which is: web or native app? I am squarely in the web camp, both for philosophical reasons (such as openness) and practical reasons (such as the fact that Soundslice has only one developer and one designer, and we can't justify building separate apps for separate platforms).
What I love about <canvas> is that it lets us make desktop-quality apps right in the browser, so we can get the benefits of being "of the web" along with the benefits of amazing, fast graphics. Fear not, my friends: <canvas> is great.
UPDATE, May 7, 2013: Evan has posted a thoughtful follow up, reacting to this.
Comments
Posted by Karel Crombecq on May 6, 2013, at 4:39 p.m.:
I'm sorry, but I have to disagree with your sentence "No question. But it's fast enough that, unless you're doing something relatively insane, you'll be totally fine."
Canvas is WAY too slow for serious game development on desktop, nevermind on mobile. I use all your tricks and much, much more, but in the end, there's one conclusion: once you have relatively large parts of the screen that need to be redrawn each frame, canvas becomes very slow. The fill rate is the major bottleneck.
And you can only avoid redrawing the canvas to a certain extent. I'm making a game and I simply have to redraw large portions of the screen each frame, because of animations. Once a considerably part of a full-screen canvas is filled with animations, it just slows down to a crawl.
I'm not saying canvas is useless - I use it for my own project, but I'm far from happy with its performance, and I'm severely limited in what I can do by it. Even with all the hacks/tricks, it's still very slow, and I wouldn't call 2D game development "doing something insane". It's one of the reasons canvas was made in the first place.
Posted by bjudson on May 6, 2013, at 5:29 p.m.:
Bombermine.com runs a pretty impressive 2D multiplayer game with canvas. Performance isn't perfect, but it's usually very playable.
Posted by DLev on May 6, 2013, at 5:40 p.m.:
Have you discovered any tricks to improve text quality? I've found canvas to be fast enough, but I'm often disappointed with the quality of text in firefox and chrome. Text in IE actually looks pretty good.
Posted by disease on May 6, 2013, at 5:51 p.m.:
bombermine? Try Score Rush HD if you really want to see an html5 game fly. It's basically an xbox360 level game.
Posted by Scott McMillin on May 6, 2013, at 6:06 p.m.:
@Karel: In the realm of application development, I'd place games in the "insane" category with regards to needing a certain level of performance, especially games that require pushing around a lot of pixels very quickly. I'm of the opinion that canvas will find its niche in creating rich UIs for more traditional app development, while WebGL will be the primary tech for browser-based game development.
Posted by Will on May 6, 2013, at 6:32 p.m.:
Mouseland.
Posted by Dong Fuoo on May 6, 2013, at 6:40 p.m.:
Will.
"Will"
"WILL"
"IT WILL!"
Has it occurred to anyone that experienced devs have learned the lessons of this broken platform more than once already.
I recommend these APIs be marketed to children learning development. Maybe by the time they grow up the sh*t will work.
Posted by Travis on May 6, 2013, at 6:50 p.m.:
When you say you're in the web camp for openness, what do you mean by that? I tend to think of services as more closed-off from end users in terms of data visibility and code openness.
Travis
Posted by Adrian Holovaty on May 6, 2013, at 7:55 p.m.:
Karel: This is in no way meant as an insult (in fact, it's a compliment!), but...I would indeed classify games as "relatively insane." :-) I don't know a lot about game development, but even with my limited knowledge, I'd say canvas is likely more trouble than it's worth for nontrivial games.
DLev: Tricks for text quality... I remember at one point discovering that built-in fonts (Arial, Helvetica, Times New Roman, etc.) perform much better than custom fonts, but I forget whether that was from the old Flash version of Soundslice or the canvas-based one. Aside from that, I can't think of any tips.
Will: Mouseland!
Travis: By "openness," I mean web sites are open with respect to the site creator -- we're not tied to a specific proprietary platform such as Mac OS, iOS or Android. And the HTML standard is open. Interesting point about services being more closed than desktop apps; that's sort of a separate axis, and I don't disagree.
Posted by gre on May 6, 2013, at 11:19 p.m.:
I've played a lot with canvas too, and quite agree with the post. Today's canvas performance are definitively better than 3 years ago.
The Cache technique is not only valuable for texts but for anything which is not a primitive draw.
We used a lot of caches in Illuminated.js (see http://bit.ly/M23BZ1 ) which uses a lot of gradients to make its lighting system (and basically we create a canvas cache for each gradient and only use drawImage for drawing it), and performance are quite ok, of course it would be even better with some WebGL shaders but that's another technology which has other problems (like quite hardware dependent).
Posted by Tim on May 7, 2013, at 12:17 a.m.:
I've found canvas to be more than powerful enough for my 2d game tower storm even with hundreds of animated minions and thousand of bullets moving across the screen. I'm using the impactjs engine which has made things easier as it has a few of these tweaks built in.
Using multiple canvas layers is a novel idea though, I'll be trying it out this week and I'm looking forward to your post on how you do them.
Posted by Freddy Wang on May 7, 2013, at 12:18 a.m.:
Canvas might be fast on desktop, but it's slow on some mobile devices especially older Android devices running older Android OS.
I tried soundslice on iPad 1, it wasn't very smooth dragging as compared to the desktop version.
Posted by Tomek Kopczuk on May 7, 2013, at 12:58 a.m.:
I just happen to be finishing a canvas-powered project (port of a rather complex, also ours, iOS app involving deep zoom and transforms).
canvas is VERY slow – for the modern standards. Even iPad 2 has much much better performance than canvas on a modern desktop. I've tried all of the things you mentioned in the last few weeks, and it helps a lot – but it's far, far from enough. Canvas is slowly getting to the 2001 desktop performance.
There's also a different thing – writing for the Mac or iOS is pure FUN. You can do everything you imagine – especially in terms of graphics and performance. Frameworks work for you, not against you. Smoothness is a standard, not an optional extra. It's expected.
It might require dividing your images in half to parallelize the rendering – but the means always make sense.
Work at it hard enough, push it, you'll get to the edge of technology – the effect is amazing, and it's been pure fun.
With canvas – weeks spent on working around multiple bugs in the rendering engines (of which Firefox is the shameful leader). You're not pushing the technology limits, just fighting with the software. That's the opposite of fun. And the effect is shameful.
Let's not be content with "acceptable".
If anyone happens to disagree about the performance achievable within the web apps, go run ages old Doom 3, watch it, then run the best ever done canvas tech demo. There's nothing to be proud of with technology that, when pushed to the limits, draws 1000x1000 2D not-so-complex graphics with a few layers "almost smoothly".
Don't get me wrong, it will be a great piece of technology and, in fact, a standard. It isn't now, and it's FAR from being even acceptable.
You've done marvelous things with the Soundslice, though! It is the limit of canvas, and you've done a truly great job. Which, to be frank, proves my point – it's not smooth on my desktop. And it is a work of art, I have to say.
Posted by David Stern on May 7, 2013, at 1:14 a.m.:
In response to some things Tomek said above:
"It might require dividing your images in half to parallelize the rendering – but the means always make sense." How does this make any more sense than not drawing more times per second than the screen refreshes?
"go run ages old Doom 3" Please keep in mind that Doom 3 couldn't run at full settings on ANY desktops that were available at the time. Many high-end machines even had trouble running it smoothly at medium settings. Also, an apt comparison would be between Doom 3 and canvas+WebGL, not 2D canvas drawing routines. See recent work with porting Unreal Engine 3 to the browser using asm.js and WebGL for more accurate estimations of the top-end of canvas 3D performance.
Posted by Louis Acresti on May 7, 2013, at 3:04 a.m.:
@David: Exactly my thoughts. Comparing a hardware accelerated 3D game engine to canvas is like comparing apples to rhubarb pie a la mode.
Posted by LeBron Collins on May 7, 2013, at 3:33 a.m.:
soundslice.com tries to load then crashes Chrome.
Posted by Hamlet on May 7, 2013, at 5:07 a.m.:
js1k.com，a lot of canvas demo
Posted by Tomek Kopczuk on May 7, 2013, at 6:47 a.m.:
"How does this make any more sense than not drawing more times per second than the screen refreshes?"
Because it increases the performance two-fold minus the context-switching overhead vs. making everything seem smooth only if you're already at 60 fps.
"Also, an apt comparison would be between Doom 3 and canvas+WebGL, not 2D canvas drawing routines. See recent work with porting Unreal Engine 3 to the browser using asm.js and WebGL for more accurate estimations of the top-end of canvas 3D performance."
"@David: Exactly my thoughts. Comparing a hardware accelerated 3D game engine to canvas is like comparing apples to rhubarb pie a la mode."
As I said, don't be content with mediocre.
Not to mention that 2D canvas is actually hardware accelerated, so are CSS 3D transforms.
Posted by Oliver Beattie on May 7, 2013, at 1:59 p.m.:
I too have developed a pretty "serious" Canvas application that people actually use on a large scale, http://www.luckyvoice.com/sing
While I agree with your suggestions (caching is absolutely a must, as is batched rendering etc), but I think the real issue is the lack of good-quality canvas *libraries* that have the kind of breadth of functionality that their native counterparts have have. All of these "performance hacks" are certainly not unique to canvas, but what is is unique is the developer (user of these APIs) having to care about them. They should be abstracted away.
Libraries like Paper.js (which I've used and extended a lot) and EaselJS are bridging the gap, but they aren't there yet — either because they aren't feature-complete in the way that they need to be, they're buggy, and/or they don't seem to be getting enough community development (or in some cases, want it).
Posted by Freddy Wang on May 7, 2013, at 10:34 p.m.:
Speaking of developing Mac-only software, evan was right. One of the most underlooked problems with developing web application is fending of security attack. It will more likely you will lose a lot of night sleeps due to hacking attempt on your web apps than your Mac apps.
Posted by rep_movsd on May 8, 2013, at 3:59 p.m.:
I did a canvas + JS perspective texture mapping thing last year
It does pixel by pixel rendering of a 800x600 image with perspective projection in about 8 ms per frame using 8x8 interpolation like Quake, this on a fairly old 2 Ghz core 2 duo notebook.
Canvas is definitely fast - All you need to do is to use byte arrays, and use it as a frame buffer.
Posted by Andrew on May 8, 2013, at 5:31 p.m.:
I believe the problem of the <canvas> lies in the spec itself.
The canvas spec (as it defined in HTML5) mandates that it has to use offscreen pixmap buffer.
The only option to fill that pixmap buffer is to use software based rendering primitives. Bit blitting of that buffer to the screen is probably the only place where GPU acceleration can be used in such rendering schema. That's why all canvas based solution at the moment are very sensitive to the dimensions of the canvas element (number of pixels to fill by CPU).
That's why in my Sciter2 [1] engine I've decided to use different approach:
In Sciter script can draw on any element by providing element.paintBackground(graphics), element.paintContent() and element.paintForeground() methods that are called at the rendering time (when DOM/CSS gets rendered). Graphics object in my case is a thin wrapper that translates graphics.***() calls into native Direct2D calls on Windows. So all this is GPU accelerated at very large extent.
[1] http://www.terrainformatica.com/sciter2/main.whtm
Posted by Irae Carvalho on May 14, 2013, at 12:22 p.m.:
I love the point you make about `canvas` being fast, and Soundslice is an awesome use of it.
On the other hand, Ejecta [ http://impactjs.com/ejecta ] is another expressive example of how much faster native drawing is. It basically mimics all the `canvas` API but draws using native APIs on iOS. The result is that games that drop a lot of frames on Mobile Safari runs at an awesome FPS with the native app wrapper.
Comments have been turned off for this page.