Category: Dataviz

As a POC (proof-of-concept) for a presentation, I wanted to generate a heat map, similar to the kind you see for Toronto Raptors players and for other examples, such as for airline flights or overlays on physical world maps.

Because I was presenting this in front of a fairly mixed audience, I did not want to make assumptions about how well they could make the mental ‘leap’ while I explain the narrative I had built for them. In other words, for each part of the story I was telling them, I wanted them to see the transition from one idea to another. Therefore, I decided to find a way to animate the heatmap to make the transition as smooth as possible.

A great lightweight heatmap is simpleheat.js (demo) that I integrated into a wireframe in the form of a web page. I stripped my presentation down and posted it to GitHub here.

Because this is just a concept, the data is not actually generated by the client; it was data that I created for this purpose (see my blog post on creating placeholder data). For the demo, there are actually several data sets.

However, the beginning heatmap result and the final result at the end can be really different than one another. Therefore, if you just hit ‘load’ on the next set of data to re-populate the heat map, it will ‘jump’ and not be a smooth transition.

To get a smooth transition, I implemented an animation concept called inbetweening which in this case loads all the states in between the beginning and the end of the two heatmaps. Because this was just a fast proof-of-concept, there is lots of room for code optimization to make an even smoother transition and error checking.

The order of operations is the following:

1. Bring in the first json dataset and populate heatmap.
2. Load in the second json but don’t populate the heatmap, yet. Make sure they have the same xy coordinates and that those coordinates in the second json are in the same order as the first json. (This is assumed to be generated at source. I didn’t add a try-catch to the code.)
3. Find the two coordinates that have the greatest difference in z-values. This means running through the list, loading the values into the array and performing Math.max on the array. (Note this is a different ‘maximum’ than ‘themax’ variable in the code.)
4. The max value from the step above is the number of incremental changes.
5. The function looptheloop() is called as many times as needed, as determined by the steps 3 and 4 above. This function looks at the difference of the z-values for each coordinate.
6. If there is no difference in the z-value for a given set of coordinates, do nothing and ride out the loops until the end (see ‘Same value’ section below).
7. If there is a difference in z-value for a given pair of coordinates, it updates the starting json by moving it gradually closer and closer to the target json (see ‘Different z-value’ section, below).
8. In the code, you will see there is an incremental update to a value called ‘themax’, but this might not be needed in your case. I used it to avoid the predominance of too much red in one of the results I was seeing.
9. Therefore, the starting json is updated and reloaded, step-by-step. Once the starting and target json are the same, the incremental refreshing of the heatmap stops.

Same z-value for source and target?

Different z-value? Change and re-load by increments.

In developing this code, I had a look at some dataframe capabilities to avoid so many loops. However, because I wanted to get this out quickly, I did not finish exploring javascript dataframes. If the number of loops causes your animation to grind, then a dataframe would be highly suggested.

Want a smoother animation? Simply make the increments to be fractions (right now the fractions are 1 at a time, but could be 0.5 or 0.25 as simpleheat can handle it). Then call looptheloop function more often.

I recently needed to create an interactive wireframe concept using a heatmap. This was for a presentation and I needed to walk the client through a particular story. As a result I needed just enough data to populate the heatmap but had to somehow create the data myself.

Getting the right data wouldn’t be straightforward: The heatmap, using json, requires both an x-and-y coordinates, and then a third value to denote intensity of colour on it (I’m calling it z or z-value). Since this was a POC (proof-of-concept) I needed just enough for it to work, but didn’t want to over-engineer it.

Even though the heatmap is about 750 by 500 pixels, you don’t need 500 rows and 750 columns in Excel to get it to work. In my example I had 30 rows and 47 columns, and then I just multiplied them to get them to reach across to the full range of pixels in the heat map. This is because the points are fuzzy and spread out and cover any gaps. You’ll see this in the Excel example for download, below.

Since I was kind of hacking this out, I simply created a grid in Excel which had x-coordinates along the top, y-coordinates on the side, and then in the middle, the z-values. In order for me to see roughly what it would look like, I used conditional formatting in Excel to show colour values based on a scale.

Below the Excel ‘heatmap’, I concatenated values in the format of the json file, multiplying the coordinates so it would fill in the whole canvas in html. Then I cut the ‘json’ file into somewhere the javascript would find it. Running the simpleheat.js code on the placeholder file was the final step.

In case the explanation isn’t enough, I’ve made the Excel file available for download here.