Announcement

Collapse
No announcement yet.

Using GeoTIFF files in TSRE

Collapse
X
Collapse
First Prev Next Last
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • RichardAJensen
    replied
    The Shuttle Radar Topography Mission (SRTM) Collection User Guide (usgs.gov)

    Alas, the resolution of SRTM HGT files is not contained within the file, it is assumed by the file processing app based on the number of rows/columns in the file. If there are 1201 of each, the HGT processor assumes that it's the 3 second data, and if there are 3601 of each, it assumes that it's a one second (~30M) product. I suppose one could write an extension of a HGT processor to calculate the coverage resolution based on the size of the matrix, but I'm not sure how to do that. And at that point, I would also have to figure out extend HGT exporting software to use any size square matrix.

    There are GeoTIFF stub files in the TSRE5 source code, but I don't know enough C to be able to finish the implementation either.

    All my coding experience is with ColdFusion and JavaScript, which is pretty far removed from being able to parse binary files, process their data and then generate new binary files.

    Leave a comment:


  • RichardAJensen
    replied
    Originally posted by eric View Post
    can this be converted into a HGT file for use with TSRE?
    Let me do some checking. If memory serves, I ended up going down this rabbit hole because HGT files have a 'baked in' 30M resolution limit.

    Leave a comment:


  • eric
    replied
    I've used the 1M resolution stuff and it's remarkably accurate in places like Chicago where you have huge cuts and embankments both for the railroads and freeways/roadways. The problem had been availability.

    Taking your idea a step sideways... can this be converted into a HGT file for use with TSRE? I've stopped using Demex because of the memory issues and simplicity of the HGT's with TSRE...

    Leave a comment:


  • RichardAJensen
    replied
    I've created a how-to PDF.

    The document does not use an example route. Rather, there are instructions on each step, and the steps are provided in the order in which they should be executed.

    Route designers will need to decide how they should stitch together their 1x1 and 1M GeoTIFFs, and how they should clip files for export. All I can provide is the basic rule of thumb that files imported into DEMEX should be less than 1GB.

    Leave a comment:


  • RichardAJensen
    replied
    Originally posted by PerryPlatypus View Post
    I've never been able to get the 1M data to work out for exactly the reason you indicated - it would always scale weirdly north-south
    Heh. I'm glad other people are having the exact same problem--I thought I was going nuts there for a while ("Why is this app altering perfectly intelligible latitude coordinates? WHY?")

    I'll try to get some screen caps together.

    Leave a comment:


  • PerryPlatypus
    replied
    Originally posted by RichardAJensen View Post
    ...3) In QGIS, import both the NED and 1M GeoTIFFS. It is helpful to put the NED TIFFs below the 1M TIFFs.

    4) Go to 'Raster > Miscellaneous > Merge'. Select all of the 1M TIFF layers that share a footprint with one NED TIFF (each TIFF will be on a separate layer with the same name as the file). Export this file to temporary directory.

    5) By default, this new image will be placed in a new layer on top of the other layers. Right click on the layer and choose 'Export > Save As'. In the dialog that pops up, you'll need to change the coordinate system to EPSG:4269 - NAD83. This is the same coordinate system as the NED TIFFs. Save this new TIFF to a temporary directory.

    6) Go to 'Raster > Miscellaneous > Merge' again. This time select the layer with the GeoTIFF you just created as well as the layer with the NED GeoTIFF. The new file that you create will be the basis for DEMEX terrain.
    Let me be the first to say, this is really ground-breaking stuff. I've made good use of 1/9th arc DEMs where they are available, but I've never been able to get the 1M data to work out for exactly the reason you indicated - it would always scale weirdly north-south. I'll be giving your method a try soon and seeing if I can make it work. If you are so inclined, any additional detail you can add to steps 3 through 6 would be greatly appreciated. I fully agree that there is a very noticeable improvement going higher resolution than 1/3rd arc, especially in the mountainous areas that I love most

    Leave a comment:


  • RichardAJensen
    replied
    I'm resurrecting this topic..

    I had a few other projects that occupied a lot of my free time over the past year, but I'm back.

    I've had to modify the workflow required to get more precise DEM data into TSRE/Open Rails.

    Here's the step-by-step process:

    0) You'll need to have QGIS installed on your computer.

    1) Go to the National Map downloader TNM Download v2 (nationalmap.gov)​. Download 1/3rd arc second (NED) GeoTIFFs from the USGS website. Each file covers an area one degree square. These files can be imported directly into DEMEX.

    2) Go to the USGS LIDAR explorer USGS Lidar Explorer Map (nationalmap.gov)​. Here you can download 1M resolution GeoTIFFs. These files cannot be imported directly into DEMEX.

    3) In QGIS, import both the NED and 1M GeoTIFFS. It is helpful to put the NED TIFFs below the 1M TIFFs.

    4) Go to 'Raster > Miscellaneous > Merge'. Select all of the 1M TIFF layers that share a footprint with one NED TIFF (each TIFF will be on a separate layer with the same name as the file). Export this file to temporary directory.

    5) By default, this new image will be placed in a new layer on top of the other layers. Right click on the layer and choose 'Export > Save As'. In the dialog that pops up, you'll need to change the coordinate system to EPSG:4269 - NAD83. This is the same coordinate system as the NED TIFFs. Save this new TIFF to a temporary directory.

    6) Go to 'Raster > Miscellaneous > Merge' again. This time select the layer with the GeoTIFF you just created as well as the layer with the NED GeoTIFF. The new file that you create will be the basis for DEMEX terrain.

    7) Go to DEMEX, select your route and the GeoTIFF you just created, and create terrain tiles as normal.

    I could put together a more thorough explanation, if the community is interested.

    --

    Why the complicated process? Because the USGS has bundled 1M LIDAR data into handy TIFFs that use the UTM coordinate system, and converting these TIFFs to NAD83 within QGIS creates files which, for some reason, DEMEX stretches, so that they extend farther north and south than they should. I'm not sure how or why DEMEX does this, because the correct latitudes are unquestionably part of the TIFFs as well as any derivative file created (such as an XYZ).

    The only resolution I've been able to come up with is to merge these GeoTIFFs with a larger GeoTIFF that DEMEX can import with no issues.

    While 1/3rd arc second data, which runs about 10 meters between elevation markers, is a closer match to the 8 meter grid used by MSTS and Open Rails, it is still less accurate than the MSTS grid. Going through this hassle to import one meter data might seem a bit much, but not by as much as you'd think.

    With 10 meter spacing, two elevation points in an MSTS tile will fall between two NED points on a periodic basis and tile elevation points can be as far as four meters from the closest NED elevation. With one meter spacing, no elevation point on an MSTS tile will ever be more than half a meter from a corresponding USGS elevation point.

    This may not be a big deal with routes through, oh, say, Kansas (or huge chunks of eastern South Dakota, for that matter), but it's huge when the terrain is even mildly interesting. I might also add that the 1M data makes a big difference when it comes to rendering railway and highway embankments, etc., even when the overall terrain is fairly flat.

    Leave a comment:


  • RichardAJensen
    replied
    Here's an update...

    So, the .5M USGS geoTIFFs are embedded in a UTM grid which DEMEX does not recognize (when you load one of these geoTIFFs in DEMEX, it reads the easting & northing as longitude and latitude respectively, with problematic results).

    If you convert the geoTIFFs from UTM to WGS84 or NAD83 using QGIS or another app, DEMEX can't align the geoTIFF with the route tiles properly (DEMEX apparently uses UTM as a basis, even though it reports coordinates in lat/long).

    However, I was able to figure out a round-trip that works properly:

    1) I stitch together the .5M geoTIFF files in QGIS
    2) Open the stitched geoTIFF in VTBuilder
    3) Save the geoTIFF as an XYZ file (warning: It will be enormous--possibly over 1GB)
    4) Open the XYZ file in DEMEX (I find it works best to run DEMEX in compatibility mode under Windows XP SP3). It will take quite a while to import the file, but DEMEX was able to import a 2.7GB file without crashing on my 16GB laptop.
    5) In my case, DEMEX places the file in the proper latitude, but the longitude is (for some unknown reason) exactly six degrees offset to the east.

    I'm trying to figure out a way to generate a shapefile for the route tiles that can be imported in QGIS.

    But in any case, I think it's pretty impressive that you can import terrain that is accurate down to ~20"x20" squares into an app that is over 20 years old!

    Click image for larger version

Name:	Screenshot 2022-06-17 193630.jpg
Views:	1
Size:	79.3 KB
ID:	2204115
    Last edited by RichardAJensen; 06-17-2022, 07:52 PM.

    Leave a comment:


  • RichardAJensen
    replied
    Originally posted by Zephyr06 View Post
    I went through the same process, and now I'm realizing the tutorial I referenced didn't headline the fact that you would ultimately be using DEMEX to generate the terrain, not TSRE's "Load Heights". Fortunately, an activation key has been shared in the file library, and DEMEX's features are still extremely useful when creating and modifying route geography.
    I'm an inveterate tinkerer, and I honestly don't mind fiddling with stuff to get it to work.

    I still joke about wanting to buy an old British car so I'll always have something to fix on the weekends.

    Leave a comment:


  • Zephyr06
    replied
    Originally posted by RichardAJensen View Post
    So ... it took me until earlier this afternoon to realize that I don't need to use the "Load Heights" tool in TSRE5.

    And because TSRE5 is MSTS compatible, I could create a route in TSRE5 and then add terrain using DEMEX.
    I went through the same process, and now I'm realizing the tutorial I referenced didn't headline the fact that you would ultimately be using DEMEX to generate the terrain, not TSRE's "Load Heights". Fortunately, an activation key has been shared in the file library, and DEMEX's features are still extremely useful when creating and modifying route geography.

    Leave a comment:


  • RichardAJensen
    replied
    Originally posted by Zephyr06 View Post
    Hi Richard,

    The tutorial by Jerry Sullivan was exactly what I needed to get high-res USGS data manipulated for use in TSRE.

    In the file library:


    -Matt
    So ... it took me until earlier this afternoon to realize that I don't need to use the "Load Heights" tool in TSRE5.

    And because TSRE5 is MSTS compatible, I could create a route in TSRE5 and then add terrain using DEMEX.



    Ah well, live and learn.

    The only thing left on my 'to-do' list is figuring out the proper coordinate system to use with these .5M resolution TIFs:
    This is the Original Product Resolution (OPR) Digital Elevation Model (DEM) as provided to the USGS. This DEM is delivered in the original resolution, with the original spatial reference. These data may be used as the source of updates to the seamless layers of the 3D Elevation Program, which serves as the elevation layer of the National Map. These data can be used by scientists and resource managers for global change research, hydrologic modeling, resource monitoring, mapping and visualization, and many other applications.


    The source files are UTM, and DEMEX doesn't seem to be able to place them correctly.

    Leave a comment:


  • RichardAJensen
    replied
    Originally posted by scottb613 View Post
    Goku had mentioned in passing in a post on Elvas that someone had figured out a way to use or convert Geotiffs for use in TSRE. In the same thread I asked him for more details - he ignored my query.
    I actually went poking around in the TSRE5 source code and found a couple files (geotiff.h & geotiff.cpp) that suggest that functionality was at least anticipated for files covering 1 degree square, and the file naming convention is the same: [N/S][latitude][E/W][longitude].tiff (instead of ([N/S][latitude][E/W][longitude].hgt) But when I stuck .tiff files in the hgts directory, it didn't work.

    Leave a comment:


  • RichardAJensen
    replied
    Hey all,

    So the instructions worked, but I learned a few things trial-and-error that I'm going to post just in case they haven't been written down anywhere else:

    1) If you right-click on any of these legacy executables (train.exe, launcher.exe, demex.exe, etc.) and go to properties, there's a 'compatibility' tab. I set each of these apps to use the Windows version current when they were written (mostly XP). Doing that made Demex far more stable and Train Simulator ran much more quickly.

    2) I don't know if it's stated explicitly, but when you load images in Demex, the image(s) you load need to cover the entire route. At least when you first create the tiles.

    I tried to export just a single tile in Demex, using a TIFF that only covered that tile, but I don't think it worked properly.

    What I'm trying to do is use USGS 1/3rd second GeoTIFFs for the overall landscape, and then use 1M GeoTIFFs for the route tiles.

    I like that you can use the USGS 1/3rd second GeoTIFFs via Demex. It's a bit kludgy, but ironically, I believe that gives you higher res with a 20 year old app than you can get through the Dovetail Games, which I think are still restricted to the 1 sec SRTM data.

    However, it would be nice to be able to use that 1M data. I can't do all the tiles at once because the file size would be gargantuan even by 2022 standards, let alone what apps like Demex were written to handle.

    One other note on the 1M data: Its embedded coordinate system is UTM, so it has to be converted to WGS84. I use QGIS to do this, as well as to create tile sized mosaics from the 1M TIFFs.

    Leave a comment:


  • scottb613
    replied
    Hi Folks,

    Goku had mentioned in passing in a post on Elvas that someone had figured out a way to use or convert Geotiffs for use in TSRE. In the same thread I asked him for more details - he ignored my query.

    Regards,
    Scott

    Leave a comment:


  • Zephyr06
    replied
    Sorry about that. My search query was simply "USGS" and the first file in the results is "dem_instructions.zip"

    Hope this gets you moving in the right direction
    -Matt

    Leave a comment:

Working...
X