ArcGIS Pro Essentials: How to Understand Spatial Errors in Map Datums and Projections
Welcome to ‘ArcGIS Pro Essentials: How to Understand Spatial Errors in Map Datums and Projections,’ a comprehensive tutorial designed to guide you through the complexities of spatial accuracy in geospatial science. In this tutorial, you will use ArcGIS Pro, the industry-leading software in geographic information systems, to explore the crucial concepts of map datums and map projections. These fundamental aspects significantly influence the spatial accuracy of your maps. As a geospatial professional or enthusiast, mastering these elements is essential to minimize spatial errors and enhance the reliability of your mapping projects.
Estimated time to complete this tutorial: 4 hours
Learning Outcomes
Readers should be able to accomplish the following outcomes by the end of this tutorial:
- Summarize the steps for creating and organizing a project workspace folder structure.
- Illustrate the ability to download data from a public source.
- Create a file geodatabase.
- Practice converting shapefiles to a geodatabase feature class.
- Add XY data in ArcGIS Pro.
- Exemplify the use of proximity operations such as buffers.
- Analyze distortion patterns using Tissot’s Indicatrices.
- Troubleshoot the causes of spatial errors.
- Solve data alignment problems in ArcGIS Pro.
- Insert essential map elements using ArcGIS Pro Export a high-resolution map.
Need a GIS skills refresh? Sharpen your skills with this comprehensive spatial analysis challenge in ArcGIS Pro.
Setting Up Your Workspace
By now, you should be familiar with file management protocols for GIS. In a typical workflow, you work on geospatial data using a local hard drive. When done, you compress your data and back up your work to your cloud storage so that you can retrieve the files from anywhere. For this tutorial, use the desktop as your local hard drive location.
On the computer desktop, create a project folder named “Exploring_Errors.” Be sure there are no spaces. You may use underscores instead of spaces. Create the following three subfolders inside this folder: original, working, and final.
Setting up a Project in ArcGIS Pro
When creating a new project, there are a couple of ways to define your workspace in ArcGIS Pro. You can either create your workspace at the same time as the project or define an existing workspace for the project to use. In this example, you will define the folder ‘Exploring_Errors” that you just created on the desktop as the project workspace for ArcGIS Pro.
Why is this important? The tools in ArcGIS Pro will default their outputs to locations within the workspace you designate. This behavior helps to keep track of GIS files, which are easily lost.
Open ArcGIS Pro. If it is your first time working in ArcGIS Pro, you may be prompted to sign in (Figure 1.2).
On the ArcGIS Pro home page, click “Map” under the new project options (Figure 1.3).
When the Create New Project window opens, name the project “Exploring Errors.” Spaces are OK in this instance. The next two steps are important, so don’t skip them. Click the file folder icon so that the location is set to your “Exploring_Errors” folder on the desktop. Then uncheck the box next to Create a new folder for this project (Figure 1.4). You don’t need to create a new folder since it would be redundant.
When you create a project in ArcGIS Pro, several elements are added automatically to your workspace files. In Windows File Explorer, open your workspace folder to see what is inside (Figure 1.5). You should notice that beyond the three subfolders you initially created (original, working, and final), there are a few other items added to your workspace.
The folder “Exploring Errors.gdb” is actually a file geodatabase. You should also see an ArcGIS project file with the same name. There are a few other items created. It is important that you do not delete or move any of these items, or you run the chance of corrupting your project.
Note: Though it may be redundant, some professors may ask you to create an additional subfolder to contain these default items.
In ArcGIS Pro, you can see these default files in the Catalog pane by twirling open the Folders (Figure 1.6).
Skill Drill: Downloading Data from Natural Earth
As you learned previously, Natural Earth is a website created to provide vector and raster data to meet cartographers’ needs using various software applications. The data on this website is free to use without restrictions. Open the Chrome browser and navigate to the Natural Earth website. Click the Downloads link. Under Small scale data, click the Cultural link (Figure 1.7).
If, for some reason, the website is down or the data is no longer available, this link takes you to a Google Drive page where you can download a backup copy of the data: https://bit.ly/uncertainty-prj-datum.
Under Admin 0-Countries, right-click Download countries (Figure 1.8). Select, Save Link As, then navigate to your original folder and save.
Scroll down until you see Populated Places (Figure 1.9). Right-click the button that says Download populated places, select Save Link as, then navigate to your original folder and save.
When you are ready, hit the back button and return to the main downloads page. Click Physical under Small scale data. On the Physical Vectors page, scroll down until you see Graticules (Figure 1.10). Right-click on the button that says Download 10. Select, Save Link As, then navigate to your original folder and save.
Previously, you learned how to decompress a file using 7zip. In Microsoft Windows, navigate to your original folder. You should see the three zip files, one for countries, populated places, and graticules. For each zip file, right-click, select 7zip, then Extract Here. Be sure to delete the zip files when you are done decompressing them. You won’t need them anymore. Eliminating them saves space and helps to avoid confusion later.
Working with a File Geodatabase
Whenever you create a new project in ArcGIS Pro, an accompanying geodatabase gets created (Figure 1.11). A geodatabase is like a container that can hold many types of geospatial datasets. It is a way to organize and manage related datasets. There are several different types of geodatabases. The one covered here is called a file geodatabase, which can store individual datasets up to one terabyte (TB) in size. When saving the output from various tools, ArcGIS Pro will often automatically save to the project geodatabase. This functionally makes it convenient for keeping the data within the project workspace.
Creating Feature Classes from Shapefiles
Until now, most readers should primarily have experience working with shapefiles, which is the most common file format for vector data. In ArcGIS Pro, a shapefile appears as a single file. However, when viewed in Microsoft File Explorer, you can see that a shapefile consists of many files linked together. Some of these files store spatial information, such as geographic location. Some files store attribute data as a database file. Others will save the geometry of a feature. Separating any one of these pieces makes the data unusable.
In this step, you use the shapefiles downloaded from Natural Earth to create feature classes within the geodatabase. In many ways, a feature class works similarly to a shapefile. Like a shapefile, it is a collection of geographic features stored in vector format that have the same geometry type, such as point, line, or polygon. For most readers, the most noticeable differences relate to the file structure and file size. When stored in a file geodatabase, a feature class can save a maximum of one terabyte (TB) of data. A stand-alone shapefile has a size limit of only two gigabytes (GB).
In the Catalog pane, right-click on the Exploring Errors geodatabase (.gdb). Select Import, then Feature Classes (Figure 1.12).
When the Geoprocessing pane opens, use the yellow file folder after Input Features and navigate to your original folder. Select the countries, graticules, and populated places shapefiles you downloaded from Natural Earth (Figure 1.13). Leave the rest of the default settings and click Run.
Close the Geoprocessing pane when finished and check your Exploring Errors geodatabase for the new feature classes. If you don’t see them, right-click on the geodatabase and select Refresh. Rename the feature classes in the geodatabase so that they have human-friendly names such as “countries,” “graticules,” and “cities” (Figure 1.14).
Add the countries, cities, and graticules to the map. Rearrange the layer order in the table of contents so that the graticules are under the countries (Figure 1.15).
Adding XY Data
Download the following CSV file and save it to your original folder. The link takes you to a Google Drive download page:
Tissot’s Indicatrices centroids
Take a moment to view the contents of the file using Microsoft Excel. As you can see, it consists of a table with minimal information (Figure 1.16). There are only two fields, longitude, and latitude. The values are in degrees represented by plain integers.
Besides the latitude and longitude coordinates, the CSV table contains no additional spatial information, such as the geodetic datum. To properly align the data, you must be sure to correctly define the datum from which these geographic coordinates were derived. The geographic coordinates come from WGS 1984, the datum most commonly used by GPS receivers and the internet.
In ArcGIS Pro, refresh the original folder to be sure that the CSV file appears in the Catalog pane. When you see the Tissot’s Indicatrices centroids CSV file, right-click on the file and select Export, then Table to Point Feature Class (Figure 1.17).
The XY Table to Point tool should appear in the Geoprocessing pane, pre-filled with the latitude and longitude in the X and Y fields (Figure 1.18). Check to make sure that the Coordinate System is set to “GCS_WGS_1984.” Leave all other settings as default and click Run.
The centroids should be added to the map. If not, you can add them manually.
Skill Drill: Creating Indicatrices Using the Buffer Tool
In 1859, the French mathematician Nicolas Auguste Tissot introduced a method to visualize the distortions of map projections using regularly spaced circles. On the globe, each circle was precisely the same size. When one transforms the globe into a flat plane using mathematical equations, the size and shape of the circles reflect the distortion patterns of the map projection. In this instance, you use the centroids feature class and the buffer tool to create a similar indicatrix.
Begin by changing the Map pane’s projection so that Earth looks closer to the globe. On the Map pane properties window, navigate to the Coordinate Systems. Under, XY Coordinate Systems Available, twirl open the Projected Coordinate System option. In the Projected Coordinate System folder, locate the World folder. Right-click on The World from Space and choose Copy and Modify (Figure 1.19).
In the Modify Projected Coordinate System window, change the longitude of center to 180 (Figure 1.20). When ready, click Save. Then click OK. The map should now appear like a globe centered on the Pacific Ocean.
Open the Pairwise Buffer tool in ArcGIS Pro by going to the Analysis ribbon and clicking the button in the default tools window (Figure 1.21).
Create a five-hundred-kilometer buffer around the layer containing the Tissots indicatrices centroids. Set the buffer distance to five hundred kilometers. Save the output to your geodatabase and call the feature class “indicatrix.” Under Method, choose Geodesoc (shape preserving) (Figure 1.22). When using the geodesic method, the ArcGIS Pro software measures distance based on the elliptical model of Earth’s shape. As a result, the buffer distances remain accurate in all map projections. Leave all other default settings and click Run.
Your buffers should appear as regularly spaced circles around the globe. Take a moment to remove the centroid layer from the Table of Contents while leaving the indicatrix layer. Using the skills learned previously, rearrange the map layers and change the colors of the basemap so that the indicatrix layer is easily visible (Figure 1.23).
Evaluate Distortion Patterns in Map Projections
As learned previously, a map projection is the geometric transformation of the round earth onto a flat plane using mathematical equations. One cannot perform this transformation without a high degree of distortion. However, some map projections can maintain a high degree of accuracy of particular geometric characteristics called preserved properties. Distortion occurs in one or more of the following properties:
- Area
- Shape
- Distance
- Direction
- Continuity
A map projection may be able to maintain more than one of these properties, but no map projection can preserve all of them at once. Take a moment to capture a screenshot of your ArcGIS Pro window. You can do this by pressing the Alt key and the Print Screen key on your keyboard while the ArcGIS Pro window is active. Open a blank Microsoft word document.
In Microsoft Word, press Ctrl V to paste the screenshot into the document. Right-click the image and choose Insert Caption (Figure 1.24). Type the name of the map projection as the figure caption. Then, write down the answer to the following questions:
- Which of the five properties appear to be preserved based on the size and shape of the indicatrices?
- Which of the five properties appear to be distorted based on the size and shape of the indicatrices?
- Where on the map does there appear to be minimal distortion?
- Where on the map does there appear to be the most distortion?
Save your Word document to your final folder. You answer these same questions for other map projections in a later step.
Evaluating Spatial Error
In ArcGIS Pro, go to the Map ribbon and click the Measure Distance tool. The icon looks like a ruler (Figure 1.25).
The first measurements need to establish an accurate baseline. To do this, change the Measurement Type to Geodesic (Figure 1.26). Recall that a geodesic measurement uses the spherical model of Earth when calculating distances. Because the circles on the indicatrix layer are precisely one thousand kilometers across, change the units in the Measure Distance tool to kilometers.
Practice using the Measure tool on one of the circles on the indicatrix layer. Zoom into the circle closest to Alaska. It has a shape that is nearly a perfect circle. Then, with the Measure tool active, move the mouse cursor over one side of the circle and click once. Then, move the cursor to the opposite side of the circle and double-click to complete the line segment. The Measure tool records the information on the dialog box (Figure 1.27). Don’t worry about getting it perfect. This step helps you practice using the Measure tool while also demonstrating the accuracy of a geodesic measurement.
Next, measure the distance between Tokyo to Vancouver. You may need to zoom out to see both cities clearly (Figure 1.28).
Open a blank Microsoft Excel workbook and record the geodesic length in kilometers between Tokyo and Vancouver. Also, record the scale factor by entering the following formula in the cell next to the distance in kilometers (Figure 1.29). Be sure to include the dollar signs in the second half of the equation.
=SUM(B2/$B$2)
On the Measure Distance tool, change the Measurement Type to Planar (Figure 1.30). Once again, measure the distance from Tokyo to Vancouver. You should notice a slight difference in the distance value.
Record the planar distance from Tokyo to Vancouver in your Excel table. Copy and paste the scale factor formula into the cell next to the distance in kilometers for the World from Space projection (Figure 1.31).
As you learned previously, the scale factor is the relationship between the principal scale and the actual scale (Figure 1.32). One uses the principal scale, based on the scale of the generating globe, to construct the map projection. Cartographers refer to a map scale measured locally as an actual scale.
In this instance, you are not using scale ratios for actual and principal scale. Instead, you are dividing the planar map projection measurement by the geodesic measurement (Figure 1.33). Like the principal scale, the geodesic measurement is based on the scale of the generating globe. The results are similar.
A scale factor of 1 means that the planar map projection distance and the spherical geodesic distance are the same. A scale factor of less than one indicates that the planar map projection distance is less than spherical geodesic distance. Therefore, the map projection is distorting distances by making them smaller. A scale factor of greater than one means that the planar map projection distance is greater than spherical geodesic distance. Thus, the map projection is distorting distances by making them larger. Knowing the range of the scale factor throughout the map is a good indicator of error and uncertainty related to size and distance.
On your Microsoft Word document, record the answer to the following question as it applies to the World from Space projection:
5. What does the scale factor indicate in terms of distortion for this map projection in the region between Tokyo and Vancouver?
Save the Excel workbook to your final folder. Later, you enter additional measurements and scale factors for multiple map projections. As you can see, the difference between the geodesic and planar measurements of the map projection is significant. Understanding how map projections influence accuracy is especially important when conducting spatial analysis.
Skill Drill: Evaluate Spatial Error
Repeat the steps for changing the projection of the Map pane and evaluating distortion patterns for the following map projections:
- • Cylindrical Equal Area (world)
- • Goode Homolosine (Ocean)
- • Mercator (world)
- • North America Lambert Conformal Conic
- • North Pole Azimuthal Equidistant
For consistency, change the central meridian or longitude of center to 180 for each map projection. You should measure the distance from Tokyo to Vancouver across the Pacific Ocean.
Be sure to take screenshots of each map projection. Zoom out when taking the screenshot so that the full extent of the map projection is visible. You may place them all into the same Microsoft Word document. Also, you should write down the answers to the same questions for each as well:
- Which of the five properties appear to be preserved based on the size and shape of the indicatrices?
- Which of the five properties appear to be distorted based on the size and shape of the indicatrices?
- Where on the map does there appear to be minimal distortion?
- Where on the map does there appear to be the most distortion?
- What does the scale factor indicate in terms of distortion for this map projection in the region between Tokyo and Vancouver?
Additionally, you should record each distance measurement from Tokyo to Vancouver on your Excel table (Figure 1.34).
Be sure to save both your Microsoft Word document and your Excel workbook when done. Be prepared to discuss your answers to the questions and your measurements later.
Troubleshooting Datum Shift
Now that you have explored how different projections distort the geometric characteristics of features on a map, you will now investigate how the ArcGIS Pro software handles multiple datasets with distinct spatial reference systems. ArcGIS Pro represents geographic data visually using the Map pane. As you learned previously, the Map pane uses a specific display projection defined in the Map pane Properties Coordinate System tab. Earlier, you changed the display projection to determine distortion patterns of the indicatrix layer visually. Each time you changed the Map pane spatial reference properties on the Coordinate System tab, the spatial reference of the feature classes in your geodatabase remained untouched. ArcGIS Pro makes this possible through a process called project-on-the-fly.
When ArcGIS Pro encounters two datasets with different spatial reference systems, it tries to dynamically line up the two layers by performing a datum transformation on the fly. A datum transformation is a mathematical process in which the geographic coordinates of one datum are converted into the geographic coordinates of another datum. Recall that geodetic datums are the reference ellipsoid and origin point that model Earth and form the basis for geographic coordinates such as latitude and longitude (Figure 1.35). You can’t have latitude and longitude coordinates without a datum. Additionally, each datum uses different latitude and longitude values due to their position relative to the geoid model of Earth.
Some datums are optimized to increase accuracy over a specific region, such as North America or Europe. Other earth-centered datums, such as WGS 1984, try to average out their fit uniformly across the globe (Figure 1.36).
Typically, the ArcGIS Pro software does a decent job when transforming the latitude and longitude coordinates from one datum to another. However, there are multiple ways to perform this transformation mathematically. In some cases, the method of transformation does not always provide the best results and can introduce spatial error into your analysis.
Creating a New Map in the Same Project
In ArcGIS Pro, save your project file. Then click the New Map button located near the upper left side of the ArcGIS Pro window (Figure 1.37).
A new blank Map pane should open with the same basemap previously used. This map will have no other layers in the Contents pane. The default name is “Map 1” or something similar. To avoid confusion, open the map properties and navigate to the General tab (Figure 1.38). Rename the map, “Datum Shift” and click OK.
Skill Drill: Download Data from the Humboldt County Website
Next, navigate to the Humboldt County GIS Data Download page. Under Frequently Requested Data Sets, download the Humboldt County GIS Roadway Centerline shapefile (Figure 1.39). Save the zip file to your original folder. When done, decompress the file and delete the zip.
Updating the Map Pane Spatial Reference System
In ArcGIS Pro, add the roads shapefile to the map. Open the Map pane properties and navigate to the coordinate system tab. Match the spatial reference of the Map pane to the new road layer. You can do so by opening the Layers folder, which displays all of the spatial reference systems currently loaded into the Table of Contents (Figure 1.40). Select NAD 1927 State Plane California I FIPS 0401 from the list. When ready, click OK. The Map pane now matches the spatial reference of the road layers.
Return to the Humboldt County GIS Data Download page and download the Fire Hydrants Shapefile under Fire Plan Data. Save the file to your original folder and decompress it. When ready, add the fire hydrant shapefile to the map. Turn off all of the layers except for the roads and fire hydrants. You can leave the basemap visible. Zoom in to Humboldt Bay and along the waterfront just north of Downtown Eureka (Figure 1.41). You should see the location of the fire hydrants relative to the streets. Most are located alongside street segments and near intersections.
Next, use the Project tool to transform the fire hydrants layer from NAD 1927 State Plane California I FIPS 0401 to GCS WGS 1984. Recall that the Project tool does not alter the original data. Instead, it creates a copy of the data during the transformation process. Save the output as a feature class in your project geodatabase (Figure 1.42).
When the geoprocessing completes, you should see your hydrants feature class in your geodatabase. ArcGIS Pro may automatically add it to the map. If not, you should do so now. Zoom close and compare the original fire hydrant layer with the new feature class. Change the colors of the point symbols for clarity if necessary (Figure 1.43). You may notice that the original layer and the new layer do not align.
Geospatial scientists call this difference in location a datum shift. This datum shift is caused by the differences in the geodetic datums between the two datasets as well as the mathematical equations used by the GIS software when trying to perform on-the-fly transformations. In other words, ArcGIS Pro knows that the two layers have different spatial reference system properties, and it works to line them up in the Map pane using project-on-the-fly.
However, the process is not perfect, and spatial errors are introduced. In ArcGIS Pro, activate the Measure tool and change the units to meters. Measure the extent of the datum shift from a point on the original fire hydrant layer to the new one. In your Microsoft Word document, answer the following question:
• How many meters of spatial error did the datum shift introduce?
• How might spatial error impact an application of geospatial analysis, such as for city planning?
Don’t see a datum shift? Don’t worry. The Project on the Fly feature is inconsistent. The lesson remains the same: Spatial errors are not always obvious. Be aware of them and take precautions by using a consistent spatial reference system across all of the layers in your project.
Take a moment to consider the implications. Imagine not having two copies of the same data set to compare. For example, suppose you downloaded the street layer from the Humboldt County website and obtained the fire WGS 84 hydrant layer from a different source, such as from a friend or coworker. How would you notice the spatial error introduced by the datum shift? Most readers would likely be unaware of the spatial error. A potential datum shift is why using a consistent spatial reference system for every dataset is essential for spatial analysis. The best way to avoid spatial errors when conducting an analysis is to ensure that every layer in the Contents pane has the same spatial reference properties as the Map pane.
Exploring the PRJ File
As you learned previously, GIS data is often composed of multiple files working together. For example, a shapefile might appear as a single file in ArcGIS Pro. When one views the same shapefile using Microsoft File Explorer, one can see that many files are present (Figure 1.44).
Once in a while, these component files get deleted or corrupted for various reasons, and you may encounter a shapefile that is missing the spatial reference information. To correct this issue, you must first understand how a shapefile stores spatial reference information. The file that ends in the .shp extension stores the feature geometry, such as nodes and arcs. As you learned previously, a node is a single XY coordinate pair, such as latitude and longitude, that represent a point feature. An arc is a linear feature made up of nodes, vertices, and line segments. In an arc, the nodes refer to the beginning and ending points of a line feature. Vertices refer to any intermediate points in between the start and endpoints (Figure 1.45).
Though the .shp file stores the feature geometry, it does not specify any other spatial reference system information such as which geodetic datum or map projection is used. As you learned earlier, each datum uses slightly different geographic coordinates. Even though ArcGIS Pro may have a shapefile’s latitude and longitude coordinates, without knowing the correct datum to use, ArcGIS Pro cannot place the features on the map. For shapefiles, the spatial reference information, including map projection and datum, is stored in the file that ends with .prj (Figure 1.46).
In Microsoft Windows File Explorer, open the original folder. Right-click on the populated places .prj file and select Open with (Figure 1.47).
When the dialog box opens, choose Notepad from the list of applications (Figure 1.48).
Take a few minutes to examine the contents of the .prj file. As you can see, the .prj file contains only text. In this instance, it starts with the geographic coordinate system, followed by the datum information. After that, the prime meridian gets defined. Lastly, the units used for the XY data are provided (Figure 1.49).
The ArcGIS Pro software reads this textual information to understand how to use the geographic coordinates stored in the .shp file. Without this information, ArcGIS Pro cannot correctly place the features on the map. In the next few steps, you will intentionally corrupt the shapefile by deleting the .prj file. Close the Notepad application and open the original folder in Microsoft Windows File Explorer. Right-click on the populated places .prj file and select Delete (Figure 1.50).
In ArcGIS Pro, expand the original folder in the Catalog pane. Drag and drop the populated places shapefile into the Datum Shift map pane. A warning message should appear regarding an unknown spatial reference (Figure 1.51). Take a moment to read through the error message. When done, click OK.
When zoomed out, the undefined layer appears as a single dot. However, all the point features are still present. In the Contents pane, right-click the populated places shapefile with the unknown spatial reference. Choose, Zoom to Layer. As you can see, all the point features are drawn correctly relative to each other. The size and location are incorrect due to the missing spatial reference information (Figure 1.52).
Choose the populated places shapefile with the unknown coordinate system as the input. For the Coordinate System, click the button on the right. In the Spatial References Properties window, open the Layers folder and select GCS WGS 1984. Run the tool (Figure 1.53).
When geoprocessing is complete, zoom back to the west coast of the United States. The two populated places layers should now align. In Microsoft Windows Explorer, you should see the .prj file replaced.
Avoid confusing the Project tool with the Define Projection tool
Though it may have a similar name, the Define Projection tool and the Project tool work very differently. The Project tool does not alter the original input data. Instead, it makes a copy of the data. Most of the geoprocessing affects the geometry stored in the new .shp file. The Define Projection tool does not create an output dataset. It also does not alter the geometry stored in the .shp file. The Define Projection tool modifies the original input data by overwriting the .prj file. If the .prj file is missing, it creates a new one. You should only use the Define Projection tool when you have a dataset that has an unknown spatial reference or an incorrect coordinate system defined.
Confusing these two tools is a common mistake that many readers often make. Mixing them up will corrupt your data. For example, suppose you wanted your populated places shapefile to use the same spatial reference as the streets and fire hydrant layers. Your intentions are good because you know that having layers with different spatial reference properties can introduce spatial errors, such as a datum shift. In this instance, you want all of your layers to be in the State Plane Coordinate (SPC) system used by the streets and fire hydrants. The correct procedure would be to create a new shapefile with the desired spatial reference properties using the Project tool. However, like many others before, you choose to use the Define Projection tool instead.
Save your ArcGIS Pro project, your Microsoft Word Document, and your Excel workbook. Be prepared to discuss your results at a later time. Back up your project folder to a safe location like Google Drive.