Bring data to life: How to use Canvas with D3
Creating our D3 data visualization for 5,000+ courses
There’s a good chance you stumbled here not even really knowing what Canvas is or D3 stands for, so before we dive in, let’s take a minute to do a quick rundown. D3 (or D3.js) is short for three important D-words: Data-Driven Documents. It’s a common – and excellent – choice for building interactive visualizations for the web. D3 excels when data must be bound to interactive elements, and when you want hassle-free transitions.
And Canvas is way of embedding pixel-based graphics into your HTML webpages, which enables performance and effects that wouldn’t be possible with regular HTML (or HTML’s vector art cousin, SVG).
While often used separately D3 and Canvas can be combined to produce engaging and high performance interactive visualisations. And that’s why I chose to use D3.js and Canvas together—to build Pluralsight’s interactive landing page to showcase 5,000 courses and counting. Let’s take a deeper dive into developing with D3 and look at how.
D3 tutorial: D3 and Canvas in action
To celebrate passing the 5,000 course mark (wow, that's a lot of courses!), Pluralsight wanted a fun visualization that would help capture the milestone. We wanted to produce something a little like Peter Cook's Freelancer Rates Survey visualization with boxes representing each of the 5,000+ courses, interactively animating into different groups.
Pete's visualization used the standard approach to D3.js, creating seperate SVG RECT elements for each box. The standard approach with D3 is to create DOM elements, either HTML or SVG. This is usually a good way of working, but has limitations in the number of elements it can handle at once, particularly how many elements can be animated at the same time. So this worked great for 566 boxes but, as you can imagine, wouldn't work so well for 5,000. This is where Canvas came into play. To work around this issue, you can render to Canvas instead of the DOM. (This also opens possibilities for graphics and effects that would be difficult with DOM elements.)
D3 tutorial: So, what is Canvas?
Canvas is an HTML element that you can embed in your HTML document, and draw shapes and text using JavaScript. On a conceptual level, regular HTML is a tree of elements from which the browser works out how to draw the page, whereas canvas is a grid of pixels over which you have direct control.
And if having direct control isn’t enough to persuade you, consider Canvas’ simplicity. To draw onto the Canvas from JavaScript, all you need to do is obtain a 2D "context," which is an object you can use to draw onto the canvas using simple method calls to make lines, circles, text and anything else you might need. You can obtain a drawing context by calling the getContext method on the canvas element passing in the string "2d".
var myCanvas = document.querySelector('canvas');
var context = myCanvas.getContext('2d');D3 tutorial: Using Canvas
Now you have a context that you can use call methods to draw rectangles, circles, lines, text and paths (which consist of lines and curves). (See the documentation on MDN for a complete list.) A drawing context has a current fill colors, a current stroke color (the line color of the outline) and a current alpha value (how transparent it is) which anything you draw will use.
The important thing to remember is that you're drawing pixels directly onto the canvas' buffer. So, unlike SVG, once you've drawn something, the things that were underneath are gone for good (unless you're drawing semi-transparently using the alpha value). You’ll want to keep this in mind because the only way to erase something is to draw over it again.
To draw a box representing a course, first set the fill color. Once that’s done, draw a filled rectangle at a given point. The coordinate system is in pixels and is relative to the top lefthand corner of the canvas element.
context.fillColor = "red";
context.fillRect(left, top, width, height);There are a few approaches you could take to updating graphics drawn on a canvas. The first is to draw over the existing content, changing only those areas that need it. This is efficient if you only need to update small parts, but it can get complicated figuring out exactly what has changed each time. A simpler approach is to just erase the contents of the canvas and redraw everything each time you change anything. This erase-and-redraw approach sounds slow, but depending on the complexity of your drawing, it can actually be blazingly fast.
We're now ready to write a draw function that can be called every frame when updating the view. The draw function for this visualization just needs to draw a box for each course, we'll handle the text headings and other bits later. The function will be given position and color information for each box and a single size for all of them. It will also be given a flag for each box indicating the alpha (its level of transparency). We'll pass in the values as parallel arrays instead of a single array of objects because this makes things easier later on.
function draw(context, size, left, right, color, alpha) {
// First clear the canvas of any existing drawings
context.clearRect(0, 0, context.canvas.width, context.canvas.height);
for(var i = 0; i < left.length; i++) {
// Optimisation for completely transparent (eg hidden) blocks
if (alpha[i] == 0) continue;
context.fillStyle = color[i];
context.globalAlpha = alpha[i];
context.fillRect(left[i], right[i], size, size);
}
}D3 tutorial: Position, position, position
You can see that the draw function doesn't actually decide anything about where the boxes are placed. Because of this, it's good practice to completely separate the layout from the the actual drawing code. We can now write a function which calculates the location of every box completely independent of the drawing code.
In order to work out the position for each box in this particular visualization, we need to first group them by the current grouping (as set by the user) and sort them. We'll want to also store the position where each group starts because we'll need that information to position the labels later on.
The position calculation code calculates where each course box should be moved, but we don't just want to move them directly there. Instead we'd like to smoothly animate from their previous position to the new position. When working with DOM elements in D3.js this is as simple as adding a called .transition() in before we set the attributes, but it's slightly more complex when we're drawing to canvas. Luckily, we can still use D3's utilities for interpolation and easing.
The d3.interpolate function can take two values – a previous value and a new one – and return a function that "interpolates" between the two. This returned function accepts a value between 0 and 1; at 0 it returns the previous value, and at 1 the new value. For values between 0 and 1, it will return a value somewhere between the previous and the new.
D3 is pretty smart at interpolating. It can interpolate numbers, colors, dates – and is even good enough to interpolate multiple numbers in a string (for example "300 12px sans-serif" to "500 36px Comic-Sans"). If given arrays, d3.interpolate returns an interpolator that interpolates each value in the first array to the corresponding value in the second. It even works if the second array is longer than the first (it just returns the new values for those items and doesn't bother to interpolate).
So we can get the coordinates, color and alpha (aka transparency) into separate arrays and then use the d3.interpolate function to create a interpolators for them. We only need to do this once for each time the grouping changes.
let x = [], y = [], color = [], alpha = [];
for(let i = 0; i < courses.length; i++) {
x.push(courses[i].x);
y.push(courses[i].y);
color.push(getColor(courses[i]));
alpha.push(isVisible(courses[i]) ? 1 : 0);
}
let ix = d3.interpolate(lastX, x);
let iy = d3.interpolate(lastY, y);
let icolor = d3.interpolate(lastColor, color);
let idraw = d3.interpolate(lastAlpha, alpha);You can see that the alpha is set to 1 (fully opaque) for currently visible courses, and to 0 (fully transparent) for invisible ones. The interpolation means we'll get a nice fade in and out when courses become invisible or reappear.
With the interpolators in place we now need to run an animation loop that redraws to the canvas every frame. D3.js provides a useful function named d3.timer that will call a callback function once for each frame passing in the number of milliseconds since the timer started.
The callback function needs to first convert the time elapsed since the transition started into number between 0 and 1 indicating progress. It then calls each of the interpolators to get updated values for positions and colors, and passes the resulting values to the draw function defined above. When the transition is complete we return the value true from the callback, which signals that the timer should stop.
d3.timer(function (timeSinceStart) {
let t = Math.min(timeSinceStart/duration, 1);
draw(context, size, idraw(t), ix(t), iy(t), icolor(t));
if (t === 1) {
return true;
}
});D3 tutorial: Getting interactive
So far we've drawn a bunch of elements to the page, but what if we also want the user to interact with them? For this visualization, mouseovers needed to be detected on the boxes representing the courses as well as clicks. In a regular HTML and SVG page you could just set event handlers on the elements representing each box, but since we're using Canvas that’s not possible.
Instead, we need to set event handlers on the canvas element itself. We'll still get the mouse events but they won't automatically be tied to a particular item we've drawn. We can, however, get the coordinates of the event relative to our canvas element using d3.mouse.
canvas.on('click', () => {
// Get coordinates of click relative to the canvas element
var coordinates = d3.mouse(canvas.node());
// Coordinates is a 2 element array: [x,y];
var course = getCourseAtCoordinates(coordinates[0], coordinates[1]);
if (course) {
window.open(course.url, '_new');
}
});As you can see, it's up to us to write code to take the coordinates of the mouse event and figure out its corresponding item. Remember that we already have the x and y coordinates stored in a data structure for each of our boxes, so the simplest method is to loop over each of them and check each one in turn until we find a match. But as you might have already guessed, this is inefficient – and for such a regular layout there's a much quicker way.
For this specific visualization I decided to find which group/heading the mouse was under first (using the groupTop array calculated earlier). I then figured out the index of the course within that with some simple math.
function getCourseAtCoordinates(x, y) {
for(let i = groupKeys.length - 1; i >= 0; i--) {
let g = groupKeys[i];
if (groupTop[g] < y) {
// We know we're in this group, we know the size and spacing of the blocks
// so figuring out which the row and column we're pointing at is easy.
var row = Math.floor((y - groupTop[g] - groupSpacing) / (blockSize + spacing));
var col = Math.floor(x / (blockSize + spacing));
// Now get the index of the course
var index = row * cols + col;
// And finally the course itself
var course = groups[g][index];
return course || null;
}
}
return null;
}This works well for our specific visualization mainly because the hit areas are simple (they're just boxes) and regularly laid out in a grid. But for different situations there are more complex techniques. For taking a large amount of objects in 2d space, then locating the closest one to a given point, there's D3's d3.quadtree.
Though if you have irregular shapes, you should consider drawing everything twice; once to your normal canvas element and once to a separate off-screen collision detection canvas. When drawing to the collision detection canvas, you’ll pick a different color for each element (you have 16 million to choose from). Then, when you want to detect an object you can sample the color at the current mouse coordinates.
D3 tutorial: Mix and match
The canvas works great for certain types of items but you'll most likely also have other parts of your visualization that would be easier to build using regular HTML or SVG DOM elements. Of course, there's nothing wrong with using a bit of both, and you can even position DOM elements over the top of your canvas to combine them. The visualization uses this technique to create the group headings and position them in the gaps left by the canvas drawing function. Because we calculated positions for groups separately from the drawing code, it's easy to reuse those coordinates to position the header elements.
D3 tutorial: Takeaway
You’ve now seen how I used D3 to bring data to life and create Pluralsight’s 5,000 courses data visualization.
Now, it’s time for you to put it into action. If you’re ready to learn D3.js and get started with interactive D3 visualizations, check out my introduction to D3.js and Pluralsight’s D3 courses here.
