In this monthly series, I turn my attention to one of the most powerful software packages for raster processing available–the Geospatial Data Abstraction Library (GDAL)–and as an added bonus, it’s free! Each month, I focus on a different utility offered in GDAL, walking the reader through its applications and offering up a series of tips and illustrations so that you can emulate this process in your office and/or home. In this edition of WIG, I show our readers how to use GDAL to create a topographic position index (TPI) map from a raster digital elevation model (DEM).
About GDAL
GDAL is an open source software application that was launched in 1998 and has been updated multiple times since. While the current version of GDAL (i.e. version 1.9) can be downloaded from multiple locations, I find this site to be the most up-to-date and easy to navigate. I suggest downloading the stable version of the MSVC2010 build for either 32-bit or 64-bit PCs depending on which you have access to. You can also download GDAL as part of an easier to install program, FW Tools, but the version of GDAL included with this installer is usually not the most recent.
GDAL contains a wide array of utilities to help you process raster files. I find that GDAL is more stable and runs quicker than ArcGIS, ENVI and PCI for data production–although the toolkit is more limited than in these applications. GDAL does not rate high for user-friendly functionality as it is command line driven (i.e. words and text, not graphics and mouse clicks); and this is the inspiration for creating the WIG series.
What is the Topographic Position Index (TPI)?
The topographic position index is a relative measure of a location’s elevation with respect to its surroundings. It is calculated in GDAL by subtracting the mean elevation of the 8 cells surrounding a central location from the central location’s elevation. Given that TPI is calculated by comparing a central point to the mean, surrounding elevation, there are negative values unlike the terrain roughness index which I wrote about last month. Here is a quick summary of how you can interpret TPI values:
- Zero or near-zero– these are areas that are flat or have a near continuous slope; for example, the pixel is in the middle of a evenly sloping hill
- Large positive– the central pixel is, on average, much higher than the surrounding areas; for example, on the top of a ridge or hill
- Large negative–the central pixel is, on average, much lower than the surrounding areas; for example, at the bottom of a valley or is a gulley
It is also important to note that TPI is scale-dependent in that you can compare a central point to the mean elevation of just the 8 immediate surrounding pixels (as is done in GDAL); or you can compare this pixel, for example, to the 5 closest neighbors in all directions. By alternating the radius of the mean elevation calculation, you can identify either fine or large scale features. In GDAL, you will be locked into identifying fine scale features such as ditches and mountain peaks given that it will only calculated TPI with a 1-pixel radius (i.e. a central point’s 8 immediate neighbors).
Creating a Topographic Position Index Map
In order to create a TRI map from a DEM, I employed the GDALDEM functions–you can find out more about them here. Here are the steps I took to create this layer:
- I moved my DEM file, a digital terrain model (DTM) produced by DigitalGlobe with 2-meter resolution over Gadara, India, to a folder called, ‘Test,’ located the root of my C: Drive
- This is the same data used in last month’s edition of WIG
- I chose a simple folder name and file location (i.e. C:test) for the data to make things easier when working with the command line interface of GDAL
- I also changed the name of the DTM file to ‘DTM.tif’ to make things easier when working with GDAL
- After starting up GDAL, I used the command: cdtest
- This command points GDAL to the folder I created with the DTM
- If you would like to get a list of the files in the folder you are working in, use this command: dir
- To calculate TPI, I used the following command: gdaldem TPI DTM.tif TPI.tif
- In this command, the words ‘gdaldem TPI’ define the utility that GDAL uses to process the data
- ‘DTM.tif’ is the full name of the input (original) DEM file
- ‘TPI.tif’ is the name I chose for the output topographic position map as a TIFF file; you can change the file extension ending to any in this list if you would prefer an alternate raster format
- There is no extra functionality commands that can be added to TRI in GDAL
Tips for Interpreting and Working with TPI Maps in ArcGIS
As has been the trend with these WIG pieces, creating symbology in ArcGIS required to interpret an output from GDAL is a multi-step process. The first step in the process is determining the maximum and minimum range for TPI. I will follow the same general procedure set out in previous WIGs to determine this. As TPI has both negative and positive values, that will add an extra wrinkle to this process from what I presented last month. As such, I created a short video tutorial using ArcMAP 10.1 to show you the steps I use.
Now that you have seen how to determine your maximum and minimum values for TPI, it is time to move on to creating a color scheme that helps us locate fine geologic structures. To do this, you will want to right click on your TPI layer and go to Properties -> Symbology -> Classified. From here, click on the Set Exclusions… button and enter the correct expression to exclude the values above and below your true TPI range. What you are trying to do here is exclude the ‘fake’ TPI data that is created by GDAL at the borders of your raster. For my exclusions, I used this expression:
-16500–-0.697, 0.725–20600
To create a color scheme that helps find fine features, it makes sense to use a Classified technique whereby ranges of TPI values are represented by easy to spot colors. You can do this from the same menu as the exclusions were entered into, i.e. Properties -> Symbology -> Classified. To do this, I ran multiple classifications offered in the pull down menu with 5 classes; and I found that Natural Breaks (Jenks) made the most intuitive sense with my data.
From there, I hit the Classify… button and changed the Break Value closest to zero to exactly 0.0. What this did is create two categories of TPI with positive values (i.e. hills) and then three with negative values (i.e. gullies). I then selected a simple color ramp of red to green so that the colors red, orange and yellow were negative; and then dark and light green were positive.
As a final step, I added high resolution imagery available through the ArcMap 10.x basemap menu to help bring the TPI map to life so to speak. The animation below shows the result of this color scheme and base layer imagery. By using this combination, I could identify thin ridges and gullies as well as roads that might have been missed by inspecting the high resolution imagery and/or elevation data on its own.

The image above shows an animation of the TPI I created from a 2-meter DTM over laying high resolution imagery. Red represents the most negative TPI values, while green represents the most positive. This visualization scheme for the TPI helps to reveal small features you might miss in a DEM and/or imagery by themselves. (DTM Source: DigitalGlobe)
Brock Adam McCarty
Map Wizard
(720) 470-7988
For this month’s Geospatial Frequently Asked Question (G-FAQ), I turn my attention to the topic of projections in ArcGIS with a focus on geographic transformations. Geographic transformations are those options that appear at the bottom of the Project Toolbox interface for both features and rasters. While the transformation will auto-populate (in Arc 10.1 at least), there is reason to be mindful of the option that is chosen.
With this in mind, the November and December G-FAQ’s will address this core set of questions:
What exactly is a geographic transformation? How do the various Projections and Transformations toolbox functions differ? What advice can you offer when working with shapefiles, rasters and geographic transformations?
In order to address the first core question, we need to start with a lesson on the basics of projections. I will focus on datums and coordinate systems as these are the two items you need to understand to use ArcGIS’ Projections-Transformations toolbox functions. Before we delve into this G-FAQ, let me be up front and mention that this discussion is not meant to be a complete lesson on projections; rather, it is a primer to give you a working understanding of the topic so as to avoid common errors in ArcGIS.
The Basics of Horizontal Datums
The building block of a projection is the datum. In ArcGIS, we are typically working with surfaces that are defined in the XY directions only, hence many of us are only aware of horizontal datums; and this GFAQ will focus on horizontal datums only. As a side note, all geospatial datasets are also defined by a vertical datum which is similar in nature to a horizontal datum.
A horizontal datum is a representation of the shape of the planet upon which a XY coordinate system is based–such as decimal degree latitude and longitude values that many of us are familiar with (e.g. 85 degrees North, 40 degrees West). Horizontal datums measure positions on the surface of our planet (vis-á-vis vertical datums which measure land elevations). A datum is defined by a mathematical model, and over time multiple models have been created and then revised. Some datums are meant to be accurate for a specific region–such as the North American Datum 1983 (or NAD83); while others are created for global datasets–such as World Geodetic System 1984 (or WGS84).
In the world of physics, a datum is described as the theoretical shape of the planet whereby there is equal gravitational potential (or zero potential energy) on every point of the surface. The reason I say a theoretical shape is that a datum will not match the actual shape of our planet’s surface as we have features, such as mountain ranges and deep valleys, where potential energy is stored. When you calculate the gravitational center of our planet and create a datum from this, the shape is ellipsoidal with flatter poles and a bulge at the center. Depending on where you place this center, the shape of the ellipsoid will change, and hence the shape differences in NAD83 and WGS84 for instance. To further complicate things, as our calculation of the earth’s gravitational center improves, so too do the mathematical models that define WGS84 and NAD83. As such, we really should define a datum with its full name to include the revision number, something that is rarely done in our industry–for example, NAD83 (CORS93) and NAD83 (CORS96).
The Basics of Coordinate Systems
A coordinate system is the other building block of projections in ArcGIS. A coordinate system is a way to define a point in space by a regularly spaced grid that is based on an origin or starting location. There are two basic types of coordinate systems: geographic and projected. We may be more familiar with geographic coordinate systems, such as latitude and longitude, given how commonly they are used in school environments and online mapping. Geographic coordinates systems are based on angular positions from the center of the planet, for instance Boulder, Colorado’s position which is 40.015 degrees North of the Equator (or the defined north-south centerline of the planet) and 105.27 degrees West of the Prime Meridian (or the defined east-west centerline). Geographic coordinate systems typically work globally or at least on the continental scale.
A projected coordinate system is meant to move three dimensional geographic coordinates to a flat, two dimensional surface or a planar surface, such as a piece of paper. Therefore, all projected systems are simply mathematical transformations of three-dimensional geographic coordinates to planar coordinates (i.e. X and Y values). To create a projected coordinate system, you need to define an arbitrary origin where the plane you are projecting on to intersects a 3D globe. Your mathematical model then transforms all of the coordinates that land in this zone from angular geographic values to a distance from the arbitrary origin in the X and Y axes. As you move farther from this origin, distortion will increase. Hence, most projected coordinate systems have multiple zones that define smaller geographies to minimize this distortion. Unlike geographic coordinate systems, projected systems are typically used for more local, fine-scale geospatial analyses. One of the most common projected coordinate systems is Universal Transverse Mercator (UTM) which was created by the US Army and divides the globe into 60 zones approximately six degrees wide. In this system, coordinates are defined as False Eastings and Northings in meters from the origin of the zone.
What Then is a Geographic Transformation?
Now that we have an understanding of the components of a projection in ArcGIS–i.e. the datum and type of coordinate system–it is time to move on to an explanation of geographic transformations. Geographic transformations come into play when moving your geospatial data from one projection to another, which is also called re-projecting your data. The geographic transformation is the often ignored third step of a re-projection as it is typically filled in with a default value–which may or may not be the correct option to chose (we will address this topic more fully in Part II of this G-FAQ).
A geographic transformation is a mathematical formula that converts coordinates in the source geospatial dataset to the output coordinate system of choice. As was discussed above, all coordinate systems are based on a datum. So ultimately, a geographic transformation converts coordinates based on one datum, for instance WGS84, to coordinates based on a different datum, for instance NAD83. Geographic transformations are used for both projected and geographic coordinate systems.
Now, let’s layer a final consideration on this topic before moving on. Most (perhaps all) datums have undergone revisions over time as the mathematical models which describe them are improved along with our improved estimation of the location of the Earth’s gravitational center. Therefore, in some instances, a geographic transformation is not as simple as moving from NAD83 to WGS84, you might really be moving from NAD83 (CORS93) to WGS84 (EGM96) with may or may not require a selection other than the ArcGIS default. Stay tuned for more details on the implication of this…
How Do I Determine if my Geospatial Data is Projected?
In this final part of the November G-FAQ, here is a short ArcGIS lesson on determining if your geospatial data is projected or not. There are three methods you can use to determine this; and in the short video that follows, I show you each of them:
- When you load geospatial data–if your data is not projected, you will get a warning message.
- By looking at the Source tab in the Layer Properties menu–you can access this by right-clicking on your data layer, selecting Properties and then opening the Source tab. This tab shows you info on the layer’s datum, coordinate system and more.
- By looking at file structure–in order for a shapefile to be projected, it must have a projection file (.PRJ) associated with it. For raster files, however, there is not a reliable way to determine if the data is projected by looking at its file structure.
Until our next edition of G-FAQ, happy GIS-ing!
Do you have an idea for a future G-FAQ? If so, let me know by email at brock@apollomapping.com.
Find Out More About This Topic Here:
- Esri ArcGIS Resources–About Geographic Transformations
- Esri ArcGIS Resources–Map Projection Animations
- Esri Knowledge Database–How To Article 24159
- National Park Service–Datums & Coordinate Systems
- University of Washington Course Notes–ESRM 250
Brock Adam McCarty
Map Wizard
(720) 470-7988
The International Space Station (ISS) has seen a lot activity in the past few weeks; including the first commercial resupply cargo ship to visit the station. On October 10, 2012, the SpaceX Dragon capsule was captured by the International Space Station’s robotic arm as it sped towards the station. The cargo ship was maneuvered toward the Harmony module where it docked and the hatch opened. For the next 18 days, the cargo was unloaded and included hardware along with science and crew supplies that totaled 882 pounds. It was then reloaded with 1,673 pounds of cargo to be returned to Earth. The robotic arm then detached the SpaceX Dragon capsule from the Harmony module, where it reentered the Earth’s atmosphere and splashed into the Pacific Ocean.
Three members of the Expedition 33 crew also joined the rest of their teammates on the International Space Station on October 25, 2012, just in time to help with the reloading of the SpaceX Dragon capsule. Three of the crew members had already arrived on July 17, 2012 after launching into space on July 14th. The mission of the Expedition 33 crew is research on microgravity as the International Space Station’s unique microgravity environment allows for in-depth studies into it affects on crew members. The crew will conduct spinal ultrasounds to assess and study changes in the spinal cord before and after space flight. This information will be incorporated into a mission health risk assessment for microgravity-induced spinal changes as related to potential injury and back pain.
Alongside the spinal tests, Expedition 33 will monitor crew member immune function by collecting blood, urine and saliva samples to assess the effects of spaceflight. The crew will also experiment with the effectiveness of low-volume, high-intensity exercise in order to reduce the loss of bone and muscle function while in space for long periods of time.
The crew on the ISS will also do their own Earth observations–taking photographs of human and natural events occurring on our planet. These images, taken over time, record changes on the Earth’s surface along with short-term natural events like fires, floods, volcanic events and weather disasters. Finally, the crew will test in-orbit satellite phone networks to plan for future spacecraft telecommunication systems.
The International Space Station has seen a lot of activity in the past month from both NASA crew and commercial spacecraft. Three members of the Expedition 33 crew will be leaving the International Space Station and landing on Earth on November 12, 2012. Expedition 34 will then commence, and they will be joined by three more crew members in December, continuing the ongoing research on the orbiting space station.
In May’s edition of Out of This World, I highlighted NASA’s Nuclear Spectroscopic Telescope Array (NuSTAR), which images high-energy X-rays in the universe, before its launch on June 13, 2012. Since its successful launch and orbit, a number of images collected by the telescope have been made available. One of the first images was a black hole in the Cygnus Constellation which was chosen due to the brightness of the X-rays in the region. A number of other targets were chosen to calibrate the telescope’s detectors and to adjust where the X-rays hit the detectors.
More recently, NuSTAR has turned its attention to the black hole in the center of our Milky Way Galaxy, and it happened to capture the black hole in the middle of a rare flare up. Most black holes consume surrounding entities like stars, gas clouds and any other ‘fuels’ within their reach. When they do, black holes flare up with intense energy. The black hole in our galaxy appears to be more subdued than most of its counterparts, rarely sweeping up surrounding fuels. Researchers were excited to catch this event with the new telescope, giving them an opportunity to study our own black hole.
NuSTAR’s observation of the black hole in the center of our galaxy is the first focused view of it in high-energy X-ray. The bright, white section of the central area is the hottest part of the black hole, and the pink areas are surrounding hot gases. NuSTAR also happened to capture a flare in the black hole during observation, depicted in the panel on the right side. (Image credit: NASA/JPL-Caltech)
Researchers are hoping that NuSTAR, in combination with other telescopes, will help them better understand black holes, their eating habits and how they grow in size. In the next two years, the telescope will image black holes in our own galaxy and beyond, probing into these objects and the surrounding regions and measuring how fast these entities are spinning. The telescope will also look at dead supernovae stars, clustered galaxies, high-speed jets and our own sun.
Are you working on a monitoring project for agriculture, forestry or water quality? Or perhaps you are just interested in learning out more about the RapidEye imaging system? If so, there is a new white paper that summarizes the value of RapidEye’s red edge band. A bibliography accompanies the white paper for those seeking a more detailed handling of the topics covered in the document which can be downloaded here.
You can find out more about the RapidEye imaging system here or by contacting Apollo Mapping at (303) 993-3863 or at sales@apollomapping.com.
The RapidEye Constellation
RapidEye is a constellation of five 5-meter medium resolution satellites each offering five spectral bands of information. The RapidEye constellation offers a daily revisit time to every location on the planet with a huge footprint that is 77-km wide. The data is priced competitively with a starting cost of $1.28 per square kilometer for all five spectral bands–academics receive a discount on this price. RapidEye adds a fifth band, the red edge, to the ‘traditional’ multispectral set of blue, green, red and near-infrared (NIR). The additional spectral data available in the red edge band allows users to extract more useful land ‘information’ than can be extracted from traditional 4-band imagery sources.
Our second Free For All this month highlights the Global Multi-Resolution Topography (GMRT) dataset. This elevation dataset separates itself from others by incorporating sea floor bathymetry and is constantly updated.
Are you looking for an elevation dataset that includes sea floor bathymetry? If so, look no farther than the Global Multi-Resolution Topography (GMRT) dataset hosted by Columbia University. The GMRT is a constantly updated elevation dataset that includes both sea floor bathymetry and land topography. There are more than 3,500 bathymetric datasets that have been complied in the GMRT with resolutions down to 50-meters in some coastal areas. For terrestrial areas, there is a combination of 30-meter ASTER elevation data and 10-meter USGS NED.
There are multiple ways to access the GMRT dataset. The most simple is the online viewer which can be accessed by clicking on the map on the GMRT homepage. If you want to download a complied GRID version of the dataset, you can click on the Create Map/Grid link located on the left side of the homepage. While this is an easy to use interface, one disadvantage is that the resolution of the GRID file is dependent on the area you download–as the area grows, the resolution shrinks. There are also streaming web services you can access here.
If you are in need of the full resolution GRID files, you can access them by:
- Clicking on the Search for Data button.
- Select From List the Bathymetry option.
- Now click on Run Search.
- Finally you can narrow resolution with the Location filter.
Until my next edition of Free For All, happy hunting for free GIS data!
Brock Adam McCarty
Map Wizard
(720) 470-7988
Brock@apollomapping.com
Have you had a chance to test out our new online imagery search engine, Image Hunter?
If not, give it a test drive now as all orders placed with Image Hunter will receive a 5% discount!
Image Hunter can be accessed 24 hours a day, 7 days a week and is a one-stop shop for every major source of medium and high resolution satellite imagery on the market today, including:
- WorldView-2
- QuickBird
- WorldView-1
- GeoEye-1
- IKONOS
- Pléiades 1 (coming soon)
- The SPOT constellation (coming soon)
If any of our readers have struggled with other geospatial search engines that are clunky; only query one data source; and/or do not provide preview images, then Image Hunter is the solution to your woes, as it features:
- Near-instantaneous loading of low resolution preview imagery with associated collection date and key metadata
- Ability to define your area of interest by address and with coordinates, geometric shape or shapefile/KMZ
- Responsive base layer with street names and imagery to provide a spatial context when discovering data
- Online ordering of satellite imagery from 3 data providers and 7 constellations
- Global coverage with tools to measure distance and create annotated maps
During the month of October, we made several upgrades:
- You can search for archive coverage by season
- The default search parameters have been changed to included imagery options dating back to the start of each satellite’s archive with less than 20% cloud and that were collected at less than 30 degrees off-nadir.
Image Hunter can be accessed here. If you would like a Quick Start Guide to get up and running, it can be downloaded here. We are excited to find out what you think about Image Hunter and any ideas you might have to improve the user experience–we can be reached anytime at (303) 993-3863 or at sales@apollomapping.com with these suggestions.
Since 1 August, Astrium Services has provided regular status reports for the international community on the clean-up and repair work underway at the Fukushima nuclear power plant in Japan. Produced with GO Monitor, Astrium Services’ monitoring and analysis service, the reports are derived from high-resolution imagery acquired by the satellite, Pléiades 1A.
Information is regularly refreshed on-line at the GO Monitor website, where Astrium keeps track of:
- Movements of radioactive waste and materials
- Analysis and monitoring of processes employed to treat contaminated materials (cement encapsulation, vitrification, storage, etc.)
- Demolition and construction work
- Clearance or reinforcement of infrastructure damaged by the tsunami (buildings, reactors, cooling systems, etc.)
- Use of support tools such as cranes, storage and public works plants and machinery
- Analysis of the volume of activity to gauge involvement of all stakeholders working on the site
This analysis follows on from that based on FORMOSAT-2 satellite imagery in the month after the tsunami of March 11, 2011. This imagery, which can be viewed via the same interface on the GO Monitor website, tracks the trail of damage left by the tsunami on a daily basis.
Astrium Services’ full satellite constellation and domain experts dispatched vital data to emergency relief teams in the aftermath of the tsunami:
- SPOT 5 delivered images of the stricken coastline and nuclear power plants on March 12, 2011, the day after the tsunami.
- GO Monitor experts provided daily analysis of what soon became a “no-go” area from images acquired by the satellite, FORMOSAT-2.
- TerraSAR-X data enabled comparative analysis of the Sendai region before and after the earthquake, highlighting the tsunami’s destructive power and the debris left in its wake.
We registered on the GO Monitor website and it was an easy process–you should register as well to see the wealth of information and imagery available on the site. For questions about Astrium Services’ GO Monitor product, let the Apollo Mapping team know at sales@apollomapping.com.