Friday, February 15, 2013

Visualizing The Census With #D3js

imageNot long ago D3js came on my radar as a library I needed to look into.  At first I dismissed it as ‘just another jQuery wannabe’.  There is some overlap, but having played with it a little, I’m coming to think of D3 more as jQuery’s nerdy cousin.

I still have some learning to do, but my first foray into it is a quick census data visualization.  Truth be told, I janked one of the examples and tweaked it to allow choosing any census measure, adjust the colors, automatically determine the scale, and fade between color changes.

var width = 960,
    height = 500,
    centered,
    quantize;
 
var path = d3.geo.path();
 
var svg = d3.select("body").append("svg")
    .attr("width", width)
    .attr("height", height);
 
var color = d3.scale.linear()
 .domain([0, 20])
 .range(["#75B7CF", "#00114F"])
 .interpolate(d3.interpolateLab);
 
queue()
    .defer(d3.json, "/content/us.js")
    .defer(d3.csv, "/content/censusquickfacts.csv")
    .await(ready);
 
var us, census;
function showData() {
    var rateById = {};
    var maxVal = 0; var minVal = 0;
    var vals = [];
    census.forEach(function (d) {
        var fips = parseInt(d.fips);
        if (fips != 0 && fips.toString().substr(3) != '000') {
            var key = document.getElementById('measure').value;
            var val = parseFloat(d[key]);
            if (val > maxVal) maxVal = val;
            if (val < minVal) minVal = val;
            rateById[fips] = +val;
            vals.push(val);
        }
    });
 
    quantize = d3.scale.quantile()
     .domain(vals)
    .range(d3.range(20).map(function (i) { return color(i); }));
 
    svg.select('g').selectAll('path')
         .transition()
        .duration(750)
         .style("fill", function (d) { return quantize(rateById[d.id]); });
 
}
 
function prepareMap() {
 
    svg.append("g")
          .attr("class", "counties")
        .selectAll("path")
          .data(topojson.object(us, us.objects.counties).geometries)
        .enter().append("path")
          .style("fill", function (d) { return '#ccc' })
          .attr("d", path);
 
 
    svg.append("path")
         .datum(topojson.mesh(us, us.objects.states, function (a, b) { return a.id !== b.id; }))
         .attr("class", "states")
         .attr("d", path);
}
 
function ready(error, usData, censusData) {
    census = censusData;
    us = usData;
    prepareMap();
    showData();
}

The result is pretty astounding.  With just a few lines of code, we grab some CSV census data and county map vectors, do the math to map the census value to a color, and draw it on the page. 

The ajax and selection are nice, but technically are things jQuery already does pretty well. Similarly, D3’s databinding model is powerful, but not enough to convince me to pick it over Knockout.  What D3 brings that is fairly unique are a slew of useful math, array, and scaling functions.  For example, this would be much more complex without D3:

//Create a function of 20 colors fading from 75B7CF to 00114F
var color = d3.scale.linear()
        .domain([0, 20])
        .range(["#75B7CF", "#00114F"])
        .interpolate(d3.interpolateLab);

As would this:

//Take the values from the CSV and map them to one of the 20 colors.
quantize = d3.scale.quantize()
        .domain(vals)
        .range(d3.range(20).map(color));

Perusing the examples, there’s still a fair amount to learn about this library, but I think it’s safe to say I’ll be putting it in the toolbox alongside jQuery and Knockout.

10 comments:

Unknown said...

Wow, pretty awesome example. I'm trying to learn by reading the code of others, so please bear with me and my newbie questions...
I'm wondering how you could specify a point on your map (ie lat,lng) but you seem to not be using one of the standard d3 projections for the geography? Are you just drawing the SVG with the GeoJSON path data which would make it impossible to locate a lat,lng point on the svg?
I hope I asked a question that is understandable...as I said, I'm just learning!

Daniel Root said...

Hey, thanks! I'm relatively new to d3 as well, but may be able to shed some light. This example does just draw the GeoJSON http://censusthing.azurewebsites.net/content/us.js. As the counties are drawn, they are filled based on their FIPS code, so there is no need for any lat/longs etc. to generate this map. That said, my understanding is that it would be possible to, say, drop a dot on the map given lat/long. This example does that given a csv of airports: http://mbostock.github.com/d3/talk/20111116/airports.html

Unknown said...

Daniel,
I picked up the airports.tsv example and successfully plotted the points via their lat/lon. Hooray! The map I'm working on doesn't go to the county level, but is only at the state level. I've mocked up a new Census file to support State FIPS. Dummy data right now, but I know where to go to get the actual data. I've removed the County geography rendering from the map and am just rending the states. I'm also trying to replicate the "Zoom to Bounding Box" behavior that Mike Bostock demonstrates on http://bl.ocks.org/mbostock/4699541, but no joy so far. I'll continue to work/learn...it is pretty fun and I'm shocked that something like this can be done with so little code. Thanks again for your example...yours is the only one I've found that dynamically refills the chloropeth with new values without redrawing/reloading the entire map. Very nice. I do have one question...I don't really understand how the legend works..quantile/quantize...Can you help me out there?

Daniel Root said...

Cool - I'm glad this example was somewhat helpful at least. The legend I added after I posted this, so I probably should do a follow-on post just around that. I suspect it could be done more efficiently, though, which is why I haven't done a new post. If you view source on the site, it's lines 135-168 that do most of the legend. The 'quantize' variable on 136 is a misnomer - the 'quantile' scale is actually being used. On that line, it maps all of the values (sorted) to one of 20 colors. At this point quantize is a _function_ that can be fed a value, and output a color. line 141 binds that color to the counties. The rest of the script block data-binds those to the legend. Conceptually similar to data-binding the colors to county geometries, but just using 20 circles instead. line 165 gets the legend text by getting the value associated with a point in the scale. Again, there are probably better ways to do this, but it gets the job done...

Unknown said...

Seriously, thanks for the help. Hmm..the keySVG has 11 circles (not 20). What does line 148 do? It looks like that is where you append circles (11 of them) to the keySvg. The first circle is always the minimum value and the last circle is always the maximum value, so you have 9 circles that represent the quantiles (of which there are 20)?

Daniel Root said...

Ah, you're right - 11 circles.
Line 148, d3.range(0, 21, 2) says, "give me an array 0 to 21, skipping every other number". I did this because 11 circles fit better, but I still need to tie back to the 20 colors used in the map. Technically, the circles don't get added until 'append' is called on line 151. You are right about what it does following that - it adds the min circle first, and the max circle last, and uses quantile for the middle 9. This I did because the first and last quantile values do not always equal the min and max, and I wanted those in the legend. Again, it's a wonky bit of the code, and could probably be done better.

Unknown said...

Daniel,
Thanks for all your help and I've gotten a start (a good one, I think) on d3 from this example. I am wondering when I see a click on the state and the function uses "d.id" to identify the polygon that was clicked....when I look in the DOM at the svg, I don't see the id attribute, so I was wondering how that worked? I'd like to be able to write a function that zoomed in on a state when a button was pressed (not a click on the state). I know the id is the fips code,so I thought I could use d3.select to pick out the svg polygon I wanted (by id), but I'm stuck. Am I looking at this completely wrong? :) Thanks!

wergeld said...

How would we go about doing a mouse over on this map to show the area and value assigned?

wergeld said...

How would we go about doing a mouse over on this map to show the area and value assigned?

Thomas Ansart said...

Hi, Your visualization link is no longer available. Is it possible to see the complete code of this ghost-viz ? I'm working on a project with functionality (drop down list) very close to yours. Thanks