For a few of the projects I have done recently, there has been a need for animated masking of images (i.e. clipping an image into an irregular shape, and then animating that clipping area). Let’s look at a simple example I mean by this by looking at a demo using photos from one of my favorite television shows from the 1960’s, The Mod Squad

Mod Squad

Mod Squad

TUESDAYS 8PM / 7 Central

Play Demo Again

Start Demo

Note the masking of the images going on in this example. If the shape of the masks were rectangles, we could use the CSS2 clip property to animate the mask, but there is no way to clip a image into an arbitrary polygon fusing just CSS, so I refactored my polyClip.js library (which was able to do this on page load) to dynamically change the clip path in JavaScript in order to animate the clip path. While I was at it, I thought I would try my hand at using this requestAnimationFrame that all the cool kids are talking about.

Although a simple idea, it took longer than I thought it would because of all the things I didn’t know about how requestAnimationFrame works, how different browsers behave with it, and how well browsers are at animating both Canvas and SVG. In the end, it was well worth the time spent — what I learned will help when I do any type of future animation work in JavaScript. This article will be a brain dump of all I have learned so far (a future blog post will discuss the newly refactored polyClip.js)

Things You Should Know About First.

Animations made with requestAnimationFrame appear less jumpy and stuttery than those using setTimeout or jQuery.animate by ensuring that animation frames are generated at the most optimal times according to the CPU load, what the browser is doing, what the graphics card is doing, etc. If you aren’t up-to-speed on how and why it works, I suggest these two articles:

• requestAnimationFrame: The secret to silky smooth JavaScript animation! from creativeJS.com.
• Paul Irish’s requestAnimationFrame For Smart Animating which includes the IE7-8 polyfill that is used in the examples in this article

Note that vendor-specific variations of requestAnimationFrame (e.g. mozRequestAnimationFrame) are currently supported by Firefox 12+, Chrome 20+, IE 10+ and Safari 6+. For all the other browsers, I have used this polyfill (which also normalizes the vendor specific variations to the generic requestAnimationFrame. The polyfill doesn’t do any of the fancier features that a native requestAnimationFrame has (i.e. generate less frames when the tab is hidden, or wait until the optimal time when the GPU is ready to render another frame), it just “does it’s best” to execute at the same speed and smoothness as a native implementation.

Is GPU Acceleration Enabled On Your Browser/OS Combination?

GPU acceleration (also known as hardware acceleration) pushes all the processor-intensive graphical work from the CPU to the graphics card, which makes animations smoother and more battery efficient. It has been implemented in Chrome 18+, Firefox 4+, Internet Explorer 9+, Safari 5.1+, and Opera 12+. However, GPU acceleration support is not as simple as “version X and higher of browser Y” — the browser must be able to use the driver of the graphics card installed on the host OS. Furthermore, just because the GPU acceleration is available on one browser on a given machine doesn’t mean that it will be true with all browsers. For example, I have a Lenovo G560 with an built-in Intel HD Graphics Card. Google Chrome on this system uses hardware acceleration, but Firefox and Internet Explorer 9 don’t (funny enough, there is an updated driver for this card, but it won’t install because it is not supported by Lenovo — so frustrating!).. In addition, Opera has hardware acceleration turned off by default. More on this later.

If there is one take-away from this article that I would stress the most, it’s that you should never assume that a desktop browser, even on a new machine, will have hardware acceleration on by default (even on browsers on new mobile devices, as the Chrome for Android stats show), and that you should test your animations with or without hardware acceleration. So, the question is, how can you figure out if the browser you are using can support hardware acceleration with your machine’s graphics card? Here is a table that can help you out.

Sadly, this information is not exposed in a JavaScript object (wouldn’t that be nice, though! A lot of the pages that generate these screens get their information from JavaScript, after all …).

Note that Internet Explorer 9 periodically grabs this XML file from their website to determine if it should use GPU acceleration or not — take a look at gpu tags at the bottom and you will see the graphics card drivers that IE9 will not allow GPU acceleration with. (As an interesting aside, it also has a list of domains that have been deemed not compatible with IE9 … quite a long list, frighteningly enough).

What does GPU acceleration have to do with requestAnimationFrame? Most of the time, if GPU acceleration is active in a browser, it means that requestAnimationFrame callbacks happen a lot more frequently, resulting in a smoother animation. The exception to this rule is Opera 12, which has a software rendering engine that is as good and sometimes better than its GPU rendering engine.

Drawing Polygons In SVG Is (In General) Much Slower Than Canvas

When I first made polyClip.js, I used HTML5 Canvas to create the clipped shapes because (a) I knew how to do it already, and (b) because I could use the excanvas polyfill to make it work in IE8 and below. When I started work on having it support animating the clip path, I assumed that I would need to refactor the library to use SVG instead of the <canvas> API to ensure animations were fast. My reasoning was that since it is programmatically dead simple to modify the clipping path of an image test in SVG, it must be faster for the browser to render the new shape. For example, let’s say you had a 726×939 image, test,jpg, that you wanted to clip with a clipping path of "0,0 ,0,939 ,726,939 ,726,0". The corresponding SVG would be:

<svg xmlns="http://www.w3.org/2000/svg" version="1.1" 
     xmlns:xlink="http://www.w3.org/1999/xlink" width="726" height="939" >

     <defs>
          <pattern id="polyClip-img-for-full-cast" patternUnits="userSpaceOnUse" 
                      width="726" height="939">
             <image xlink:href="test.jpg" 
             x="0" y="0" width="726" height="939"/>

          </pattern>
     </defs>
     <polygon id="myPoly" points="0,0 ,0,939 ,726,939 ,726,0"
         style="fill:url('#polyClip-img-for-full-cast');" />
</svg>

The defs tag contain a pattern that is used by the polygon tag to fill itself with. In order to change this clipping path, all that is needed is to use jQuery to change the points attribute of the polygon tag, like this one-liner:

document.getElementById('myPoly').setAttribute('points', "0,0 ,0,939 ,726,939 ,726,0");

Doing the same using Canvas requires a bit more work:

/*
 * Assume canvas is a canvas DOM node.and that points contains an array 
 * of numbers representing all the x,y coordinates of the clippath.
 */

// get canvas context
ctx = canvas.getContext("2d");

// freeze the canvas while we are drawing so the animation doesn't flicker
// (kind of like double buffering).
ctx.save();

// clear all the pixels in the canvas
ctx.clearRect (0, 0 , canvas.width, canvas.height);

// begin to draw the clipping path.
ctx.beginPath();

// for each point in the clipping path, draw a line.
for (var i=0; i<points.length; i+=2) {
	
	//var point = points[i].split(',');
	var x = parseInt(jQuery.trim(points[i]));
	var y = parseInt(jQuery.trim(points[i+1]));

	if (i == 0) {
		ctx.moveTo(x,y);
	} else {
		ctx.lineTo(x,y);
	}
	
}

// finish the clipping path.
ctx.closePath()

/*  
 * excanvas doesn't implement fill with images, so we must hack the 
 * resultant VML.
 */
if (window.G_vmlCanvasManager) {
	ctx.fillStyle = '';
	ctx.fill(); 
	var fill = $('fill', canvas).get(0);
	
	fill.color = '';
	fill.src = src;
	fill.type = 'tile';
	fill.alignShape = false;

/*
 * For true canvas complient browsers, fill the image using createPattern
 * and fillStyle..
 */	
} else {
	
	// fillImage contains a DOM image object.  You should ensure the image
	// is already in the image cache.  
	// See http://html5canvastutorials.com/tutorials/html5-canvas-patterns-tutorial/
	// for more info,
	var imageObj = fillImage;
   	
    var pattern = ctx.createPattern(imageObj, "repeat");
        
    ctx.fillStyle = pattern;
    ctx.fill();
}
ctx.restore();

About as verbose as talking your parents through a tech support problem over the phone, isn’t it? In general, however, Canvas code runs much faster, and when repeating this code over and over again with requestAnimationFrame to produce an animation effect, the canvas code is able to produce more frames per second (or fps) than the code using SVG, even when using requestAnimationFrame. I found this out by making a larger version of the animation above (726×939) that could do the animation with either Canvas or SVG (according to the MSDN article How To Choose Between SVG and Canvas, Canvas performance degrades the larger the canvas is). Let’s first take a look at a table that shows the number of frames per second generated for this larger version:

As you may expect, the higher the frame rate, the smoother the animation. How many fps is acceptable? A few fun-facts:

• The average film is shot at 24 fps.
• The average camcorder can record at 60 fps
• The new Hobbit film series is apparently being filmed at 48 fps.

Browser manufacturers are aiming to give you tech that can produce 60 fps, since your average computer monitor has a screen refresh rate of 60 cycles per second, or Hertz. However, most animations are still acceptable at around 24 fps (they just won’t be as awesomely smooth). As you can see from the stats, Canvas is much better when performing this task (and Opera 12’s non-accelerated graphics engine is about as fast or faster than the GPU accelerated one). Surprisingly, Chrome for Android is incredibly slow compared to Android’s stock web browser (creatively named “Browser”), even though Chrome for Android’s literature says it has “GPU acceleration for the canvas element“. Finally, Internet Explorer 9 has the worst framerate at 22 frames per second for a GPU accelerated animation using Canvas (25 using SVG) — IE9’s framerate is worse than using the excanvas polyfill in IE7 and 8, which report above 40 frames per second (!!!!!). I double-checked just to make sure .. I wonder if I can speed it IE9 a bit ….

Just to take the test even further, I made the animation scale to the size of the browser’s window height when the page loads and when the window resizes (not only did I think this was a good way to stress the graphics card, I had a project where this was a requirement). This is done with CSS3 Transforms in modern browsers and using the Matrix Filter for IE8.

Note that, in general, the animations are much smoother without using CSS Transforms/IE Visual Filters to resize the animation, which is to be expected, since Transforms require a lot more floating-point arithmetic (in IE7-8, it makes the animation slower than IE9). However most GPU accelerated browsers stayed as smooth as your Dad when he was first introduced to your Mom (almost 40 fps or higher) since GPUs can do these scaling calculations a lot faster than the CPU can.

What’s requestAnimationFrame Doing When A Tab’s Not Visible? Depends On The Browser!

As Paul Irish says, “if you’re running the animation loop in a tab that’s not visible, the browser won’t keep it running”. However what this means differs slightly from browser to browser. Firefox will run way less requestAnimationFrame callbacks, but will keep track of where the animation would have been if the tab was visible so that the user won’t notice any time delay. Chrome, however, will do the same until the end of a requestAnimationFrame loops being run at the time the tab loses visibility, and will not run any new loops until the tab becomes visible again. What this means will still run the animation (albeit with fewer frames), but once that animation ends, it will not start another requestAnimtionFrame animation until the browser tab is visible again.

Clean Up After Yourself When The Animation Is Done!

Let’s take a look at the following sample animation code:

var startTime = null,
    endTime = null;

requestAnimationFrame(step);

function step(time) {

	/*
	 * if this is the first time running this function,
	 * set startTime and endTime. 
	 */
	if (startTime === null) {
		startTime = time;
		endTime = startTime + 3000; //(animation will end in 3 seconds)
	}

	var relTime = time - startTime;
		
	// animation logic goes here ..
	.
	.
	.
		
	/*
	 * If the animation isn't scheduled to end, do another 
	 * requestAnimationFrame call. This will generate a
	 * new frame when the GPU is ready.
	 */
	if (time < endTime) {
		requestAnimationFrame(step);
	} else {
		// logic that happends when animation finishes. 
	}
}

This code assumes that the requestAnimationFrame callback will be run in between startTime and endTime. When the browser tab isn't visible for a long time, however, the final execution of the callback can happen way later than expected (i.e. when the tab gains focus again), so the callback should see if the time passed to it is greater than the endTime. If it is, it should generate the last frame of the animation. This can be done by setting its parameter (in this case, time) to the time index of the end of the animation (in this case, endTime):

var startTime = null,
    endTime = null;

requestAnimationFrame(step);

function step(time) {

	/*
	 * if this is the first time running this function,
	 * set startTime and endTime. 
	 */
	if (startTime === null) {
		startTime = time;
		endTime = startTime + 3000; //(animation will end in 3 seconds)
	}

	var relTime = time - startTime;

	/*
	 * If the step function is being called after the animation
	 * was scheduled to end, set time to the endTime in order
	 * to generate the last frame of the animation.
	 */
	if (time > endTime) {
		time = endTime
	}
	
		
	// animation logic goes here ..
	.
	.
	.	
	
	/*
	 * If the animtion isn't scheduled to end, do another 
	 * requestAnimationFrame call. This will generate a
	 * new frame when the GPU is ready.
	 */	
	if (time < endTime) {
		requestAnimationFrame(step);	
	} else {
		// logic that happends when animation finishes. 
	}

}

jQuery.animate() Does Not Use requestAnimationFrame (Yet).

I've known this for since the last jQuery conference I attended last year, but it's worth stating here: jQuery.animate(), which I have used in a number of projects, does not use requestAnimationFrame at the moment (version 1.8.). This is due to a number of issues that came up when trying to bake it into jQuery in the past, but the jQuery team has promised that it will be baked into a future release. In the meantime, I have used a temporary subsitute, Silk.js, to do the animations in a similar way to jQuery using requestAnimationFrame. Here is an example of how I used it to replace the jQuery.animate() code above:

/*
 * Let's animate $title to -320px in 3 secs.  We will also mask the image 
 * represented by $jNode using the Silk's step property.
 */
var animation = new Silk($title, {
	top: -320
}, {
	stepStart: 1439,
	stepEnd: -70,
	duration: 3000,
	complete: function () {
		stats.innerHTML += 'animation frames: ' + (animation.framesRendered / 3) + ' per second\n';
		complete(); 
	},
	easing: 'easeInOutCubic',
	step: function (now, fx) {
		$jNode = polyClip.clipImage($jNode.get(0), '0, ' + now + ", 726, " + (now - 413) + "526, 726, 0, 0, 0");
	}
})

I look forward to using jQuery.animate() for these types of animations instead of Silk, since it has been tested a lot more than my code. :-)

Firefox and Opera Eat A Lot of Memory While Animating Using the GPU and Canvas Compared to Chrome

While running the animation example linked above, Firefox eats up way more memory compared to Chrome. To prove this, take a look at this video of the clean-room version of the demo above running under both browsers on a MacBook Pro running Mac OS X 10.6.8 with 8 GB of RAM Pay special attention to the green part of the pie chart on the right which shows how much free RAM is available.

Note how fast the memory increases when using Firefox for Mac. This is not as pronounced in Google Chrome for Mac, and the memory that is used up by that browser is eventually freed up.

I have also noticed that Opera eats a lot more memory than Firefox when hardware acceleration is turned on, but since it is not on by default, this may not be an important consideration for developers. In all browsers, when a machine starts running out of memory, animations become more “stuttery” since requestAnimationFrame is not able to generate as many frames. Hopefully Opera and Firefox will fix these issues soon, but the take-away here is to test memory usage, especially for complex animations.

Filled SVG Polygons Are Buggy in Safari (Mobile and Desktop)

Okay, this last bit doesn’t really have anything to do with GPU acceleration or requestAnimationFrame, but I think this is interesting to those interested in browser graphics and animation. In all current iOS devices I tested with (iPhone and iPad), filling the SVG polygon with an image resulted in the image being shown upside down:

Screenshot of SVG image fill in iPad2 (same happens in iPad3). Not everything is upside down and not filled in haphazardly. Click to see full sized image.

On the desktop version of Safari, when the user scrolls the SVG image on and off the screen, it seeems there are some refresh errors that occur:

Graphic errors happen when the SVG image is scrolled on and off screen. Click to see full image.

Furthermore, according to caniuse.com, SVG is not supported in Android’s stock browser prior to version 3. Canvas, however, is supported on all modern Android machines. This is yet another reason why Canvas is a much better choice for the use case I am working with (for the time being).

In Conclusion?

In this particular use case, Canvas wins over SVG, but this is probably not true always. Since I have been doing a lot of animation work lately, I will be doing a lot more research on the subject. If anyone has anything else to add to my findings above, please comment below: I’d love to get your input from your experiences.

Footnotes

^* Uses requestAnimationFrame polyfill
^‡ Uses excanvas polyfill
^† GPU acceleration enhanced by using the -webkit-transform: translateZ(0) trick.
¹ Although Safari for Windows reported more frames per second for SVG and Canvas without using transform: translateZ(0), it definitely looks smoother with the transform set.