AscMapOct Guide

Overview

The AscMapOct.py utility script combines the values of two Grid ASCII (.asc) map files, a source map and a destination map, to produce a new map with the dimensions and values of the destination map, but with the values of the source map for any cell locations that overlap between the two maps.

AscMapOct.py was written to assist JPDEM flat-processing of large DEM maps. JDPEM may require very long running times to flat-process DEM maps with large dimensions. An experimental method for dealing with that issue is to create multiple, smaller sub-maps from the single, original large map. If the sub-maps are created with due regard for the overall topography, and all of them satisfy the AscMapOct.py limitations listed below, then they may be flat-processed individually. However, to use them to replicate the original, full map, they must be flat-processed in order, from lowest (by elevation) to highest sub-map, and the overlapping cell values for adjacent maps must be transferred from flat-processed to un-processed maps during the overall flat-processing sequence. AscMapOct.py provides a way to perform this transfer step.

Note: AscMapOct.py has many limitations, and is not the only way transfer overlapping cell values.
JPDEM users with sufficient GIS experience may wish to employ a GIS tool of their choice instead.

Requirements

A Windows Powershell or Command Prompt.
Python 3 installed and available.
A source ASCII Grid (.asc) map file.
A destination ASCII Grid (.asc) map file.

Limitations

The source and destination files must ...

overlap by a region of at least 3x3 cells.
be cell-aligned and from the same UTM zone.
have the same header cellsize value (and, implicity the same unit for that value).
must use xllcorner and yllcorner, not xllcenter and yllcenter.
Custom border overlays are not permitted.

Note: "cell-aligned" means that the file's grids are cell-boundary aligned: i.e. the values of:
(abs(source_xll - destination_xll) / cellsize)
and
(abs(source_yll - destination_yll) / cellsize)
are both integers.

Usage

Open a Windows Powershell or Command Prompt and type the following command:

py -3 AscMapOct.py --help

Depending upon where the AscMapOct.py script is located, you may need to include full or partial path info as part of its name.
The command above assumes it is being run in the directory where AscMapOct.py is located.
Running the command above echoes AscMapOct.py's help message to your console:

usage: AscMapOct.py [-h] [--lint] [--output_asc_file OUTPUT_ASC_FILE] source_asc_file destination_asc_file
Transfers overlapping cell values from source_asc_file to destination_asc_file.
positional arguments:
  source_asc_file       Grid .asc file to copy values from.
  destination_asc_file  Grid .asc file to merge source values with.
optional arguments:
  -h, --help            show this help message and exit
  --lint                Checks if source can transfer to destination; doesn't actually transfer values.
  --output_asc_file OUTPUT_ASC_FILE
                        Grid .asc file to write merged values to.

Per the help message above:

You must specify source ASCII Grid (.asc) file. The file is assumed to have already been flat-processed by JPDEM, and to have standard (outermost rows and columns) cell border, but AscMapOct.py does not verify either of these assumptions.
You must specify a destination ASCII Grid (.asc) file. The file is assumed to be "raw" data -- i.e. not already flat-processed by JPDEM, and to have no border, but again, AscMapOct.py does not verify either of these assumptions.

AscMapOct.py doesn't modify either the source or destination files in either way. Instead it outputs a combination of the two files to the console. The output will have the header values of the destination file, but overlapping cells will have the values from the source file.

Example Usage

Remember: you will typically need to include paths in the file names. Paths have been omitted in this example to keep the command line compact, but typically, the AscMapOct.py file will not be co-located with the data you are processing.

The following example transfers overlapping values from source file all_ones_border-zero_9x7.asc:

PS C:\Users\Me\Velma\My_Results> Get-Content .\all_ones_border-zero_9x7.asc
ncols 9
nrows 7
cellsize 3.0
xllcorner 0.0
yllcorner 0.0
nodata_value -9999.0
0 0 0 0 0 0 0 0 0
0 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 0
0 0 0 0 0 0 0 0 0

to the destination file all_twos_9x7.asc:

PS C:\Users\Me\Velma\My_Results> Get-Content .\all_twos_9x7.asc
ncols 9
nrows 7
cellsize 3.0
xllcorner 12.0
yllcorner 6.0
nodata_value -9999.0
2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2

Note: Get-Content is a Powershell command that echoes the contents of a file to the Powershell console.

The destination file's xllcorner and yllcorner values indicate that it is offset above and to the right of the source file. When we run AscMapOct.py on this pair of files, we should see section of "1.0" values in the lower-left corner of the output, and the output should retain the destination file's header values.

PS C:\Users\Me\Velma\My_Results> py -3 AscMapOct.py .\all_ones_border-zero_9x7.asc .\all_twos_9x7.asc
ncols 9
nrows 7
cellsize 3.0
xllcorner 12.0
yllcorner 6.0
nodata_value -9999.0
2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0
2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0
2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0
1.0 1.0 1.0 1.0 2.0 2.0 2.0 2.0 2.0
1.0 1.0 1.0 1.0 2.0 2.0 2.0 2.0 2.0
1.0 1.0 1.0 1.0 2.0 2.0 2.0 2.0 2.0
1.0 1.0 1.0 1.0 2.0 2.0 2.0 2.0 2.0

The above output is what we expected. Notice that the border cells (all set to zero) of the source map were not transferred. This is by design: the outermost rows and columns of the source map (whatever their values) are not transferred.

Command Line Options

--lint

When invoked, the first thing AscMapOct.py does is verify the specified .asc files.
The verification steps performed are:

source and destination cellsize values are the same.
source and destination maps are cell-aligned.
source and destination have overlapping cell area.
source map header dimensions match and data length.
destination map header dimensions match and data length.
when --output_asc_file is present, whether the output file it specifies already exists.

Including --lint on the command line tells AscMapOct.py to stop immediately after performing these verification steps, and report the first verification step that failed. If no problems are detected, a --lint-optioned command line returns without any output at all.

Some problems are not dectected during the verification steps, and are thus not part of what --lint can check:

Map files whose headers employ xllcenter and yllcenter instead of xllcorner and yllcorner cause a failure as their contents are loaded -- i.e. before AscMapOct.py runs its verification tests.
The overlap test is for any overlap, not the minimum adequate overlap area specified by the script's limitations.
If the bounding-box overlap area for the source map does not equal the bounding-box overlap area for the destination map, the script fails with an error message. The portion of the script that computes the overlap bounding boxes runs after the verification steps, and so is not included when --lint is run. (Note: maps with proper cell-alignment are unlikely to fail for this reason.)

--output_asc_file

Specifies the file that AscMapOct.py writes its results to.
If the --output_asc_file option is not specified, AscMapOct.py outputs results directly to your console. Writing to the console is useful for checking what the results look like before running the script "for real", but when running the script "for real", specifying the --output_asc_file option is the simplest way to save the results.

Note: AscMapOct.py does not permit you to overwrite an existing file.
If you specify a file that already exists as the argument for the --output_asc_file option, the script fails with an error message.
If you want to "reuse" that output file, you must delete it before running AscMapOct.py and specfiying it again.

Here is the Usage example, plus the --output_asc_file option:console itself:

PS C:\Users\Me\Velma\My_Results> py -3 AscMapOct.py .\all_ones_border-zero_9x7.asc .\all_twos_9x7.asc --output_asc_file overlap_map.asc

Note: You can "pair" the file to the option with either a whitespace (" ") or and equals ("=") character.
Either --output_asc_file overlap_map.asc or --output_asc_file=overlap_map.asc is acceptable for the above example.

Running the example command line above writes the script's results to the file "overlap_map.asc".

It is possible ignore the the --output_asc_file option and capture AscMapOct.py results into a file using only console commands, but it requires a longer command line and more knowledge of the console command language. In compensation, it also allows you specify the file's character encoding, and to easily overwrite a pre-existing output file (an unsafe "benefit").

As an example: for Windows PowerShell, we recommend capturing the output via a pipeline combined with the PowerShell Out-File cmdlet.
Here is the Usage example, plus the additional command line syntax to capture the output to a file named "overlap_map.asc" instead of to the console itself:

PS C:\Users\Me\Velma\My_Results> py -3 AscMapOct.py .\all_ones_border-zero_9x7.asc .\all_twos_9x7.asc | Out-File -Encoding ASCII -FilePath overlap_map.asc

The pipe ("|") symbol tells PowerShell to take the output of command on its left (the AscMapOct.py command) and pass it to the command its right (the Out-File cmdlet).

The Out-File cmdlet can be used in various ways and for various situations, but what we want it to do here is:

Force the output to be ASCII-encoded.
Save the output to the file "overlap_map.asc" specified by the -FilePath argument.

We force the output to be ASCII because PowerShell's default encoding is Unicode.
If you omit the -Encoding ASCII argument, the output is written to the file as Unicode characters.
If you find you don't need the -Encoding ASCII option, feel free to omit it.

Submit a Suggestion