Tuesday, 1 April 2014

Using GeoEnrichment on ArcGIS Online

Background

Many Thematic Maps benefit from having demographic data, which can be hard to come by. Moreover, what data from government statistics agencies (like Stats Canada) have recent profile data, it’s often only freely available at very course geographies (province / state). Things are a bit better in the US, but while US data is available, it’s often not in an easy to use format. Also, many municipalities now have Open Data portals, but again, the data is in “human readable” format, not necessarily a GIS ready format.

This document outlines a set of steps you can use to enrich your existing GIS data with demographic data from Esri’s ArcGIS Online. ArcGIS Online is Esri’s hosted GIS solution.  While it’s not ‘free’ – ArcGIS Online is based on a subscription model – Esri does provide free, limited use developer accounts. Also, if you already have your own copy of ArcGIS Desktop (Basic or above) you have an ArcGIS Online account.

Before you begin

You’ll need an ArcGIS Online account. If you already have one – great. If you have a desktop ArcGIS seat, work through your organization to activate your account. If you still need an account, sign up for a free developer account here.


You’ll also need your target geography or areas for your map.  To proceed, you’ll need a shapefile of your subject geographies (ward boundaries, provinces, states, counties, Census divisions etc.).  You can also use other geographies (drainage basins, ecozones etc.).  It depends on your map. ArcGIS Online will aggregate data to your polygons.

Your shapefile needs to be in a ZIP file ready to go.  Make sure all the shapefile components are in the ZIP (.SHP, .SHX, .DBF etc.).

Estimate your credits.  ArcGIS Online is not ‘free’.  Your free account has 50 credits per month.  You’ll use anywhere from 5 to all 50 of your credits – depending on the number of geographies (polygons) you’re enriching, and the number of variables you choose. Only choose the variables you think you need for your map. If you’re frugal, these steps will probably only use 5 to 10 credits (and your 50 renews each month).

Remember to source your data.  If you use ArcGIS Online data for your map, you must list “Demographic data from ArcGIS Online” (or something like this) as a source.

Let’s Go

In the example outlined below, we’ll be using a shapefile of City of Toronto Neighbourhood Planning Areas (obviously, you’ll use your own data for your map).

These can be downloaded at the City of Toronto Open Data Catolog here


Initially, there’s not much to this data – really just neighbourhood names…

The data from the website is already in a ZIP file.  If yours is not, add all the shapefile components to a ZIP file before you proceed.

Step 1 Start ArcGIS Online

You need to know the URL for your ArcGIS Online account, and your credentials.  If you have created the developer account, you should have these handy.  For this example, we’ll use the 43 North GIS developer account.

Go to your ArcGIS Online portal. It should be something like:

 https://<myorganization>.maps.arcgis.com/

where <myorganization> is your ArcGIS Online organization name…
Click on “Sign In” and Log in with your account.

Once you’re signed in, you’ll be at your organization page…
Click on “Map” to launch the ArcGIS Online map viewer (your initial extent might be different)…

Step 2 – Upload Geography

Next, we need to add the geography we want to enrich. From the tool bar on the map, choose “Add”, then “Add Layer from File”.

Click on “Choose File”, and navigate on your local file system to where the ZIPped Shapefile is located.
Important! Choose the option to “Keep original features”.

Click on “Import Layer”

After a few seconds, your map will refresh, and you should see your geographies…

Step 3 – Choose Variables and Enrich

Your geography is now added as a layer in your map Table of Contents. Click on the small down arrow beside your layer name. From the layer context menu that appears, choose “Perform Analysis”.
In the Analysis options that appear, expand “Data Enrichment” and click on “Enrich Layer”…
From the drop-down list, choose your country (obviously, we’ll be using Canada here). The list of available data variables will appear in the tree list…
There are a large number of variables.  Remember to choose only those that are relevant to your map. If you’re unsure, a good set of summary data is available in two groups
  • Key Global Facts (Basic population)
  • Key Canada Facts (Population projections out a few years, Income data, Employment data

For this exercise, we’ll select just the Key Global Facts group, but explore the data for relevant variables before you perform the enrichment. You can select only individual variables from the group. Remember to make a note of what variables you chose, so you can work with them later.
Click on “Show credits” to see how many credits your request will use (and to make sure you have enough). When you’re happy, give the result layer a name, and click “Run Analysis".

After a short delay, your new layer will be added to the map. In this case, it’s called “Enriched neighbourhoods”.
You can explore the data while still in the map.  From the layer context menu, you can “Show Table” to see the variables.  You can also make a thematic map here by exploring the “Change Symbols”.  This is optional.

Step 4 - Export the Data

Last step is to extract the data back out so you can use it in ArcMap.  There are lots of ways to do this. The simplest is to export the layer you have just created as a Shapefile, and bring it back into ArcMap. 

From the main menu, go to “My Content”
In your Main folder, you should see your layer (There is a lot in this account, so find the layer you just created – in this case “Enriched neighbourhoods”…
Click on the layer name (“Enriched neighbourhoods”) to bring up the details page for the layer. 

Scroll down until you see the “Layers” section. In the dropdown arrow beside the layer name, choose “Export to Shapefile”…
You’ll get the Export dialog.  Give the export a name, choose at least one tag (can be anything) and give it a description…
Click “Export”. After a short delay, you’ll be at the new details page for the Shapefile.  From the “Open” menu, choose “Download” and save your shapefile…
That’s it! You can work with new shapefile on your PC in ArcMap with the additional variables.

If you unzip the shapefile you just downloaded, and put it in the same folder, you can see it has new attributes…
Map Away!

Sunday, 2 March 2014

Tanaka Contours in ArcMap 10.2

Create Tanaka Contours in ArcMap 


Background


As part of some development work I've been doing for an advanced Geovisualization course I'm teaching, I had to come up with a simple way of creating Illuminated contours (a.k.a Tanaka Contours) in ArcMap. There are a few methods out there that use more sophisticated math for computing the shade values on the contours.

There is an ArcNews Article by Patrick J. Kennelly, and A. Jon Kimerling, that outlines several advanced cartographic methods (but does not provide detailed steps). Also, a blog post by Josh Ryan shows detailed steps for computing Tanaka Contours, but involves some robust raster algebra which probably produces 'truer' results.  I wanted as simple a method as possible, so have forgone the advanced math on hillshades in favour of simply using aspect.

Before diving in, it's important to note results will be dependent on data. Contour interval, DEM resolution and final display scale all factor in to your results. Experiment on a small data subset first. Interestingly, you'll get better results with a more generalized DEM and a higher contour interval. Smooth contours also work best.

This blog outlines some basic steps that can be done in ArcMap to generate Tanaka Contours. While I developed this in 10.2, I see no reason why it wouldn't work in earlier versions - nothing fancy here. These steps are only a guideline - users may want to refine the steps, and perform further data refinements, generalizations and also explore other symbolization options. You'll need either the Spatial Analyst or 3D Analyst extension for some of these steps.

Of course, if someone was keen, all this could be strung together in a Model Builder Model, but I wanted my students to go through the steps.

Steps


Start with a Raster DEM. Here's one you can use for Mt St. Helens. If desired contour lines already exist, skip to step #3

1. Generate contours in Spatial or 3D using "Contour" GP tool. Select desired interval…

2. (Optional but recommended) Smooth the generated contours using the “Smooth Lines” GP tool. You may want to test the smoothing tolerance..

3. Generate aspect surface using the "Aspect" GP tool. Results will be added as classified raster.


4. Export aspect surface - right click on layer, and choose Data > Export Data. Be sure to check "Use Renderer" (Note: you could also use the Spatial Analyst "Reclassify" GP tool, but since the data is already reclassified in the TOC, I just use Data > Export). 


5. Convert new raster to polygons using "Raster to Polygon" GP tool. Caution, this can take a long time for large areas and/or fine resolution DEMs.


6. Perform a spatial intersect between the contours and the aspect polygons using the “Intersect” GP tool. This will create new contours, with the classified aspect as an attribute of each new segment. Be sure to select “LINE” for the Output Type…



7. Symbolize the final line feature class. You will have an attribute called “gridcode” which you need to symbolize by.  It contains values from 1 to 9.  Use the table below to set the symbolized values for a grayscale map.  You can modify these to use other colours or even use line widths

“gridcode” value
Direction (reference)
Colour (gray)
Line width
(any colour)
1
N (0+)
10% Gray
0.50 pt
2
NE
30% Gray
1.00 pt
3
E
50% Gray
2.00 pt
4
SE
70% Gray
3.00 pt
5
S
50% Gray
2.00 pt
6
SW
30% Gray
1.00 pt
7
W
10% Gray
0.50 pt
8
NW
White
0.10 pt
9
N (360-)
10% Gray
0.50 pt

Your "Symbology" tab for the "Layer Properties" should look something like the examples below. These were done using a DEM for Mt St. Helens.

Using the gray shading method, you'll have symbology that looks like this...
And will give a map that looks something like this when placed on top of the symbolized original DEM (try setting the layer transparency to 40%)...

If you choose to vary the contours by line thickness, you'll have symbology that looks like this. I'm using a brown line colour here, and varying the thickness as per the table above.  The shade of brown here is Esri's "Leather Brown" (R:137 G:112 B:68)...
This gives you a map that looks something like this when placed on top of the symbolized original DEM (try setting the layer transparency to 40%)...

To reiterate, Contour interval, DEM resolution and final display scale all factor in to your results. Experiment!


Wednesday, 29 January 2014

Welcome to GIS Geek North

Well, here is the first post for GIS Geek North.  In this blog, I will rant, rave, comment and sometimes drone on about a random set of topics.  Well, not totally random - everything will have to do with GIS, mapping, spatial analysis, spatial data - just about anything to do with GIS and related topics.

Comments are always welcome and encouraged.  Chime in!