Archive of articles classified as' "Portfolio"

Back home

AskCHIS NE map: All D3 Everything

November 19, 2014

Last week saw the launch of AskCHIS Neighborhood Edition, a product of The California Health Interview Survey and the UCLA Center for Health Policy Research, with whom we worked to develop map and chart components for this new interactive tool. The short story of AskCHIS NE is that it is a tool for searching, displaying, and comparing various California health estimates at local levels such as zip codes and legislative districts. Take a look at it if you feel like setting up an account, or you can watch a demo at its launch event (demo begins at 14:00).

AskCHIS Neighborhood Edition

The long story is interesting too, as this is fairly sophisticated for a public-facing tool, so we’d like to share a few details of how the map and charts were made.

We built that big component with the map, bar chart, and histogram. It lives in an iframe, and as a user makes selections in the table above, various URL hash parameters are sent to the iframe telling it what data to load. The map and bar chart then make requests to a data API and draw themselves upon receiving data. Almost everything here uses D3, either for its common data-driven graphic purposes or simply for DOM manipulation. As usual, this much D3 work made it a fun learning experience. And it once again expanded my awe for Mike Bostock and Jason Davies and their giant brains: more than once during this project, I asked about apparent TopoJSON bugs far beyond my comprehension, and each time they fixed the bug almost immediately.

Davies/Bostock mega-brain

Tiled maps in D3

The map shows one of six vector layers on top of a tiled basemap that is a variation on Stamen’s Toner map, derived from Aaron Lidman’s Toner for TileMill project. Normally we use Leaflet for such tiled maps, but our needs for the vector overlays were complex enough that it made more sense to adapt the basemap to the vector layers’ framework (D3) rather than the other way around. Tiled maps in D3 are pretty painless if you follow the examples of Mike Bostock and Tom MacWright. The six vector layers are loaded from TopoJSON files and drawn in the usual fashion in the same Mercator projection as the basemap.

Dynamic simplifcation

The most interesting technical detail of the map (to me, anyway) is that it uses dynamic scale-dependent simplification. This is a nice design touch, but more importantly it ensures that the map performs well. Zip codes need to have reasonable detail at city scale, but keeping that detail at state level would slow panning, etc. tremendously. We took (once again) one of Mike Bostock’s examples and, after some trial and error, got it to work with the tiled map. Here’s a stripped-down example. Simplification is based on the standard discrete web map zoom levels. As the user zooms, the vector layers redraw with appropriate simplification when those thresholds are crossed, with basic SVG transformations for scaling in between those levels. Keep your eye on the black-outlined county in the gif below, and you should be able to see detail increasing with zoom.

Dynamic simplification

Pooled entities

One of the more sophisticated capabilities of this tool is combining geographies for pooled estimates. For example, maybe you want to look at Riverside and San Bernardino Counties together as a single unit:

Pooled entities

The API does the heavy lifting and delivers pooled data; our challenge was to display pooled geographies as a single entity on the map. Although TopoJSON seems to support combining entities (after all, it does know topology), I had no success, apparently because of a problem with the order of points. Instead we use some trickery of masking, essentially duplicating the features and using the duplicates to block out interior borders, as outlined below. If you wonder why we couldn’t just skip step 2, it’s because our polygons are somewhat transparent, so the interior strokes would still be visible if we simply overlaid solid non-stroked polygons.

Combining SVG shapes while preserving transparency

Image export

The biggest source of headaches and discovery was the image export function. Users can export their current map or bar chart view to a PNG image, for example the map below. (Click it.)

Map export

I learned that converting an HTML canvas element to PNG data is built-in functionality, but getting this to work in a user-friendly way was not so simple. For one thing, our map is rendered as SVG, not canvas. Luckily there is the excellent canvg library to draw SVG to canvas, so we build the entire layout above in SVG, then convert the whole thing to canvas (except the basemap tiles, which are drawn directly to the canvas). Pretty cool. Thanks, canvg!

The nightmares begin when trying to trigger a download, not just display an image on screen. Firefox and Chrome support a magic “download” attribute on <a> elements, which does exactly what we want, downloading the linked content (image data, in this case) instead of opening it in the browser. If only everyone used those browsers! Cutting to the chase, we couldn’t find any way to ensure the correct download behavior across browsers without sending the image data to a server-side script that returns it with headers telling the browser to download. The final task was showing a notification while the image is being processed both client-side and server-side, which can take long enough to confuse the user if there is no indication that something is happening. Failing to detect the download start through things like D3’s XHR methods, we ended up using a trick involving browser cookies.

COOKIE?!?!?!?!?!?!?!

Phew! Turns out making a static graphic from an interactive is complicated! But it was a valuable experience—subsequently I’ve even used JavaScript and D3 deliberately to produce a static map. It works, so why not? (Most of this dot map was exported from the browser after drawing it using this method.)

ALL the D3

For all of the above, it’s hard to post code snippets that would make sense out of context, but we hope that the links and explanations are helpful, and are happy to talk in more detail if you are working on similar tasks. There’s a lot more going on than what I’ve described here, too. Suffice it to say, D3 is a pretty amazing library for web mapping!

No Comments

Geography of Jobs: animated mapping with D3

October 29, 2014

One of our recently completed projects is a new Geography of Jobs map for TIP Strategies. Have a look at it, and read what they have to say about it.

Geography of Jobs

It’s a month-by-month map of job gains or losses over the prior twelve months for most (or all?) of the metropolitan areas of the United States, from 1999 to present. Proportional circles are colored to indicate gain or loss, and the map can either be animated or controlled by moving a slider.

This is a very straightforward project, and we like it as an example of animated proportional symbol mapping using the D3 library. Data are loaded from a CSV file and plotted on a map drawn with D3’s convenient Albers USA projection, which automatically handles Alaska and Hawaii insets. Then we use basic transitions and timers to create animation. The timeline slider uses Bjørn Sandvik’s d3.slider plugin.

Smooth transitions between frames are achieved by tweens (created with D3’s transition method) that are the same length as the frames. Although this may obfuscate specific point values, tweening improves change blindness problems, and is just plain nice to watch. Compare the smooth transitions:

Tween

With an abrupt version:

No tween

Perhaps the latter would be easier to interpret numerically, but in general we expect people to retrieve exact values by poking around the map, rather than watching the animation.

In any case, view source on the map if you’re looking for code examples of animated D3 maps!

Geography of Jobs

No Comments

The ‘Why Not The Best’ map: thematic mapping with Leaflet

July 31, 2012

We’re pleased to mention our most recently completed project: an interactive quality-of-health-care map for Why Not The Best, done in conjunction with IPRO for the Commonwealth Fund. This is actually the second incarnation of this map, which is now built with more modern technology and is better integrated with the Why Not The Best site.

Why Not The Best map

We first built the Why Not The Best map in Flash about two years ago, using the Flex-based indiemapper framework we had developed for ourselves. Flash was a good solution the time, as this was a relatively simple vector thematic map. However, the map’s functionality and features later grew to the point where it incorporated multiple shapefiles, several layout modes, and embedded Google Maps—resulting in a very complex project and a large SWF for users to load. At the same time, non-Flash web mapping technology was improving rapidly, opening new doors for building this kind of map. As such we again worked with IPRO and completely rebuilt the map in HTML 5 and JavaScript.

We’ve learned a few things from this and similar projects and thought we’d share some experiences from this map, in case other folks just getting into this kind of work can benefit from the same tricks.

Leaflet

The web mapping library I’ve settled on as my choice lately is Leaflet, which is developed by CloudMade. Leaflet offers “slippy map” and overlay basics but also a lot of opporunity for easy custom extensions. While there is little built-in support for thematic cartography like choropleth maps (although the very recent release makes more strides in this direction), it’s not difficult make it happen in completely custom ways. Below is an overview of what we did in this case.

Choropleth

Our choropleth map, in brief, is drawn from tiled GeoJSON served by TileStache and colored by data requested from IPRO’s database. Although I suppose it will be part of the Leaflet release some day, there’s currently no built-in support for tiled GeoJSON layers; however, as you’d expect with an open-source project, third-party developers have come up with solutions. We used this TileCanvas layer with some additional modifications.

GeoJSON is commonly drawn to SVG, but just as we learned long ago in Flash, a ton of vectors rendered on screen at one time can really bog things down. We wanted to try the canvas element instead, rendering the vector data as static rasters and faking the mouse interactivity that SVG would have allowed. Sure enough this did seem to perform better, especially when we needed to draw our county layer in high detail. Thus the GeoJSON TileCanvas layer, which builds upon Leaflet’s canvas layer, was handy.

To make this all work interactively and allow on-the-fly coloring based on data, we had to perform a couple of behind-the-scenes tricks. The main technique was introduced to us by our friend Jeremy White, a Graphics Editor at the New York Times whose fantastic interactive work you have undoubtedly seen and who has explained this in a couple of talks. Basically, it’s this:

Canvas choropleth: behind the scenes

Beautiful, right? This green thing is a map layer that ordinarily is invisible underneath the actual choropleth layer. It draws each enumeration unit (counties here) with a unique color based on a unique ID number for that unit. That number could be its position in an array, or in our case an arbitrary number included in the original shapefile (again, though, it’s just the row number of each record). On a canvas element we can’t do much more with the mouse than detect the coordinates of the mouse position and the pixel color under the mouse. By assigning colors this way, we can read that pixel color and turn it back into the ID number, thereby learning which county is under the mouse.

Unique colors for canvas choropleth

Our method for assigning colors, also more or less following Jeremy’s tips, was to use the red and green channels, each of which has 256 possible values. It works like this for my county of residence, which had an ID number of 1222:

  1. 1222/256 = 4.77. We only want the integer for this part, so Red = 04.
  2. 1222 mod 256 = 198, or in hexadecimal, C6. Green = C6.
  3. Blue = 0.

In other words, we just run down the sequential list and increment the green value by 1 until reaching maximum green (255), at which point red is incremented by 1 and green rolls over to 0. In our code, a feature’s color looks like this: "rgba('+Math.floor(feature.properties.ID/256)+','+feature.properties.ID % 256+',0,1)"

Having detected a color on the map (#04C600 here), we can reconstruct that ID number.

  1. Red = 04. 4 * 256 = 1024.
  2. Green = C6 (198). 1024 + 198 = 1222.
  3. Looking at our data, we find that 1222 is Middlesex County, Massachusetts.

The blue channel is not used at all for the counties and is instead reserved for drawing borders between them. Some confusion of pixel color can happen where two or more counties come together, sometimes appearing as a totally different color that belongs to a unit elsewhere. We draw blue over the borders as a kind of buffer. If any blue is detected while moving the mouse around, we know that it’s very close to a boundary and we don’t try to discern a county. Using only the red and green channels in this manner, 65,536 unique colors—and thus unique map units—are possible. If the blue channel were also used, it would be more than 16 million.

Our map doesn’t actually store a list of the all the counties and ID numbers. Instead we store the JSON attribute information along with the geometry on the custom TileCanvas layer. On a county mouseover, we loop through the features on the tile to find which one matches the ID number of the detected color. That way we know both the attribute information and the geometry, allowing us to display the name, etc. and also redraw the choropleth tile with a yellow highlight added to the correct county.

Proportional symbols

Marker symbols in general are easy to implement in Leaflet, but using them thematically is not straightforward. For one thing, it’s difficult to maintain an association between the symbol and the data that it represents. Again, though, it’s not hard to extend Leaflet’s capabilities, so we just added our own modified type of marker layer that allowed us to pass in data to control the symbol size and label. We shied away from SVG here too and used div elements for the symbols, something which was also just introduced in this week’s new Leaflet release. The CSS border-radius property can be used to make circles.

Proportional symbol divs in Leaflet

Multi-scale mapping

Even though raster basemaps tend to be designed for multiple scales (often 18ish unique scales), it’s easy to overlook the importance of doing this for vector data in web maps, too. Our map needed to work from continent level all the way down to street level, at 15 or so scales, half of which include the vector choropleth layer. We ended up using three levels of generalization, thanks of course to MapShaper.

Generalization at different scales

Beyond good practices, generalization was important for performance. Note the file sizes on the image above. At the smallest scale, all of the country is visible, meaning we need to download and render all of the vector data. If we didn’t generalize the geometry, that would mean the whole 13MB!

Each of the three generalization levels is a different layer in TileStache. Rather than implementing three separate layers in Leaflet, we added a couple of methods to the TileCanvas layer to allow for several different source URLs, automatically switching between them as the zoom level changes. We could give it an object like this and not worry about it thereafter:

var countiesURL = [
{url:tilestacheBase+'counties-smallscale/{z}/{x}/{y}.geojson',min:0,max:4},
{url:tilestacheBase+'counties-medscale/{z}/{x}/{y}.geojson',min:5,max:7},
{url:tilestacheBase+'counties-largescale/{z}/{x}/{y}.geojson',min:8,max:20}
];

Thematic web maps: getting better!

There’s much more to the new Why Not The Best map than has been described above, so go check it out. To say nothing of the many design challenges, these were just a few of the most interesting technical challenges, which, thankfully, Leaflet and TileStache made it pretty easy to solve. (Now if only it were as easy to avoid the Mercator projection in choropleth web maps!) It’s been a good experience in learning how non-Flash web mapping libraries have improved for thematic cartography, even if they don’t support it natively. If you’re new to this, check out the Leaflet documentation and tutorials, as well as TileStache and our blog post tutorial on how to set up a tile server.

1 Comment

London Low Life

May 5, 2011

This week, we’re happy to begin a 30-day preview of one of our most distinctive interactive mapping projects: The London Low Life Map. This map was produced for Adam Matthew Digital, a digital publishing company based in the UK. Adam Matthew produces digitized archives of historic primary source documents, collected around a central theme, for higher education institutions. This map was built as part of their London Low Life collection that explores the seedy underbelly of Victorian London. It examines the documents of sex, drinking, gambling, and the institutions that sprang up to combat those very vices. As the map is integrated into Adam Matthew’s collection, we will only be able to grant access to our readers for 30-days. I encourage you to explore the map and enjoy Adam Matthew’s fantastic collection of historic maps and images. Since we’ve pulled the map out of the collection, I wanted to give you some context on what is included.

View the map >>

London Low Life-2-2

Historic Basemaps

The full London Lowlife project gives users access to a mountain of primary source documents from Victorian London, so it only makes sense that we start with maps made of greater London during this era. We’ve included some metadata with each of these maps (author, date, publisher) to give historical context and placed them on top of a custom Cloudmade basemap (with adjustable opacity) to give some modern context as well.

London Low Life-3-2

Taking these historical maps from raw image to geographically accurate overlay was an incredibly intricate procedure. While some maps existed as single, contiguous images, others were scanned in their current forms as pages, separated by fraying canvas. Before we could rubbersheet the maps to OpenStreetMap data (with the help of the UW Cart Lab), we had to manually remove the seams and align the resulting image fragments to one another. Furthermore, because these maps were at different sizes and scales, we had to build a system that would identify maps with limited resolutions and restrict the zoom levels available on the fly.

standford1884.tif-1-2

It’s fascinating to view the changes in the street maps over time, especially which streets were given primary status then and now.

Tallis Streetviews

In the mid-nineteenth century, John Tallis drew detailed views of the fronts and façades of buildings along Central London’s streets. Originally designed as a “visual yellow-pages” (businesses would pay to have their shops labeled), his 88 plates now survive as a first-person perspective into the streets of Victorian London.

London Low Life-4-2

We wanted to make these plates as immersive as possible. With the help of AMD’s editorial staff, we took Tallis’ original image and added some color to sharpen the images, shadows to depth, and a blue-sky background to increase the realism of the images. Finally, the images were run through Google Sketchup to create the perspective views that place you in Victorian London and allow you to look down either side of the street.

Be sure to click the “view original” button to see the original plates to view the intricate detail surrounding the street images.

Thematic Data

While the historic basemaps and Tallis Streetviews of the London Lowlife map attempt to provide insight into the qualitative aspects of Victorian London geography, we also wanted to explore the changing character of the population through quantitative thematic mapping. Simple demographic measures like population and density can provide a major insight into the nature of a city. However, we believe the most telling indicator of life in Victorian London, and certainly true to the name “low life” ascribed to this project, is the explosion in social services that sprung up throughout the city to deal with the new urban population.

London Low Life-5-2

By dragging the timeline to view an animation of the entire century, you can watch population increase in various sections of the city and then the services emerge to try and meet the need.

Victorian London

The final section of the map gives a taste of the larger London Lowlife project by placing a selection of the primary source documents on the map. It’s an engaging way to explore city by viewing some fantastic records of the landscape and people of Victorian London.

London Low Life-6-2

Mouse over a category point on the legend list to quickly highlight all corresponding points.

View the map >>

1 Comment

Video Demonstration: Illinois Public Health Map

March 9, 2011

We’re very happy to be able to show-off our collaboration with the Illinois Department of Public Health and IPRO. This map makes information about the quality of health in communities available to the public, highlighting socioeconomic disparities that may exist. When combined with our indiemapper platform as well as linked graphs and charts, the clinical data in the map can be used to examine the health needs of a community, county or region for better policy and planning.

Above is a quick demonstration video showing the basic functionality. After watching the video, check-out the map itself.

No Comments

Death’s Door Spirits: Mapping Wisconsin’s Finest Craft Distillery

November 22, 2010

I only have a few rules in life. One of them is when the makers of this gin offer you a straight-up trade–bottles for maps–you take it… and you don’t cut your partners in on the deal. That was the situation when a friend-of-a-friend approached me to build this simple locator Google maps mashup which I took on as a side-project away from my normal Axis Maps work.

fuck_me

Death’s Door Spirits is a craft distillery based out of Washington Island, WI and distilled in Madison, WI. They use locally-sourced ingredients for their gin, vodka, and whisky which they distill in small batches. While their small-scale makes their products excellent, it also makes them tricky to locate. The purpose of the map is to simply show where you can buy bottles or cocktails containing their spirits.

Since the map was so straightforward, it gave me the opportunity to experiment with a few new technologies I’ve been meaning to check out. Here’s what I thought…

Death_s Door Spirits Locator

Styled Google Maps: Death’s Door has a fantastic graphic identity thanks to some great work by Grip Design (who also contributed some designs to this map). I wanted the map to fit in with their existing style while stripping out a lot of the extra Google Maps data that would clutter up a map like this. Google Maps Styled Map API allowed me to do both. The styled map wizard was helpful for sketching out styles but mostly I was making whole groups of features invisible and going with stock colors for the roads and water. Not being able to input RGB values for these styles made the process harder than it should have been.

Amazon SimpleDB / CloudFusion: I’ve been a big fan of Amazon’s SimpleDB technology since we used at as the foundation for indiemapper user management. I’ve found it to be more reliable than MySQL and faster as well (but that could just be our budget hosting plan). It was a little unapproachable to me at first but the CloudFusion PHP library made it as easy–if not easier–to query the database and parse the results for the map. The biggest missing feature here is a nice front-end browser like PHPMyAdmin which would have allowed the client to edit their data directly. Instead, I had to build some rudimentary tools so they could manage their data themselves.

Google Font Directory: It was nice to have access to a wider variety of fonts than just the standard web-safe font-family sets. The Google Font Directory gave me access to a limited but FREE selection of a nice variety of fonts distributed across Google’s massive network. Most useful was Yanone Kaffeesatz which is a dead ringer for the Death’s Door title font. The fonts can be a lot to load at runtime so it was important to pare down only to the fonts I was using in the map.

Death_s Door Spirits Locator-1

jQuery Ajax / jQueryUI: Made it incredibly easy for someone with my level of programming skill (let’s call it “recreational” to be polite) to efficiently get the data into the map and build some nice UI effects like animation and the accordion component. I can’t recommend it highly enough so there’s going to be no italics section with the caveats in it.

You can view the full map here …and if you’ve got any good cocktail recipes, that’s what the comments are for.

Death’s Door Spirits Locator

4 Comments