Article Title: "Unknown data type" when importing DEM as ARC Binary
Article Name: GENERAL3
OS Platforms: ALL
Last Updated: 06/23/2004

TOPIC:

This Tech Tip explains how to resolve an error that can occur when attempting to import a DEM (such as a GTOPO30 or ETOPO5 tile) as Arc Binary Raster, in the case where it was first uncompressed/untarred with WinZip.

DISCUSSION:

GTOPO30 and other DEMs can be downloaded from the Internet as a set of Arc/Info files that have been bundled with the Unix TAR utility and then compressed with the GZIP utility. For example, if you go to the main GTOPO30 download page from our Data Links page, then you can download a file that has data for southern California and Mexico called w140n40.tar.gz. This file contains the following files: W140N40.DEM, W140N40.DMW, W140N40.GIF, W140N40.HDR, W140N40.PRJ, W140N40.SCH, W140N40.SRC, and W140N40.STX. The binary file with extension ".DEM" contains the grid of elevation values as 2-byte (16-bit) integers. The ASCII files with extensions ".HDR", ".DMW" and ".PRJ" are the header file, world file and projection file, respectively. These much smaller files contain georeferencing information for the DEM.

Once you have "untarred" and "ungzipped" the compressed file you downloaded, you can import the DEM into RiverTools by choosing Import DEM > Arc Binary Raster from the File menu. As long as the ".HDR", ".DMW" and ".PRJ" files are in the same directory as the ".DEM" file, RiverTools will be able to create its own ASCII georeferencing file in RTI (RiverTools Information) format. The ".DEM" file can be used directly by RiverTools because Arc Binary files are a special case of the RTG (RiverTools Grid) format. To avoid confusion, however, you may want to rename them to have the compound extension "_DEM.rtg".

If you used the WinZip program to untar and ungzip the downloaded file, then you may encounter the following error dialog while importing the GTOPO30 DEM as Arc Binary Raster:



This dialog tells you that something is wrong with the filesize of the ".DEM" file, such that the filesize is slightly larger than it should be. The filesize is 57,741,237 bytes instead of its expected size of 57,600,000 bytes.

The cause of this problem is that WinZip has an option called "TAR file smart CR/LF conversion" that should not be turned on when downloading binary files. This option is intended to correct for the different end-of-line character conventions that are used for text files (ASCII) on Windows, Unix and Mac platforms. The carriage return (CR) and linefeed (LF) characters are special, nonprinting ASCII characters with ASCII codes of 13 and 10, respectively. For some reason, ASCII files on computers running Windows indicate the end of a line of text with both the CR and LF characters, while ASCII files on Unix computers use just the LF character and Macintosh computers use just the CR character. The binary file called W140N40.DEM must have had 141,237 bytes that when viewed as representing ASCII characters got resolved as a CR or LF. When WinZip then tried to convert these the CR/LFs, it corruped the binary elevation data and caused the size of the file to grow by 141,237 bytes.

The solution to this problem is to open WinZip, choose Configuration from the Options menu, click on the Miscellaneous tab, and then uncheck the check box labeled "TAR file smart CR/LF conversion". Now when you use WinZip to untar and ungzip the file you downloaded the binary file with extension ".DEM" will have its correct size and can be imported into RiverTools without errors.

This Tech Tip also illustrates why it is important when you are downloading binary and text/ASCII files from FTP sites to make sure that you toggle your FTP program into either BINARY or ASCII mode, as appropriate for the file you are downloading. Programs like WS_FTP let you provide your own list of file extensions that can help the program to automatically determine if a given file is binary or ASCII so it can perform end-of-line (EOL) character conversions, when necessary. Binary files may also need to be "byte-swapped" for compatibility with your computer, since there are two byte order conventions called little-endian and big-endian for binary data files.

If the RiverTools error dialog indicates that the filesize is smaller than expected, then the binary file may have been truncated during download. This can occur either because you ran out of space on the drive you were downloading to, or the connection was dropped for some reason. Connections can be dropped due to timeouts, problems with the server you are downloading from, or a disconnected cable. The solution in this case is to download the file again. In rare cases, the file you are downloading may already be truncated or corrupt on the server.

See the Tech Tip GENERAL4 for related information.


If you have any additional questions or comments, please contact Rivix Technical Support for assistance.

Copyright (C) RIVIX, LLC. 1998-2016