Announcement

Collapse
No announcement yet.

Using GeoTIFF files in TSRE

Collapse
X
Collapse
First Prev Next Last
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Using GeoTIFF files in TSRE

    Hi,

    I've been experimenting with both MSTS and TSRE, and I'm in a bit of a quandary.

    TSRE seems much easier to use, but it also seems to be hardwired to use the SRTM datasets. However, the USGS has a significant amount of the lower 48 surveyed with high res LIDAR. The resolution is so good that you can actually pick up the crown on streets and highways.

    I would like to use these files in TSRE. Is this possible?

    I really don't want to battle with the MSTS route builder just to get high res terrain.

    I was rather surprised to see the SD Capitol, from my hometown on the USGS LIDAR landing page: https://www.usgs.gov/calval/lidar

    #2
    Side note: I probably shouldn't be too surprised by the use of the SD Capitol, seeing that the article was written by someone at the EROS data center which is just a few miles north of my current home in Sioux Falls, SD.

    Comment


      #3
      Hi Richard,

      The tutorial by Jerry Sullivan was exactly what I needed to get high-res USGS data manipulated for use in TSRE.

      In the file library:


      -Matt

      Comment


        #4
        Hey Matt,

        Thanks for the response. However, I get a 'no files match your search criteria' error when I click on that link.

        Web programmer note: It seems that the forum software does not create URLs that encode the search string (e.g.: https://www.google.com/search?q=jerry+sullivan+tsre+train+sim), it creates search URLs that refer to a session key or database row that does not necessarily persist (https://www.trainsim.com/vbts/tslib.php?searchid=21497993).

        I experimented a bit with the file search, and discovered a bunch of Milwaukee Road routes by him, which is kind of interesting because my first project is trying to reconstruct a short stretch of abandoned Milwaukee Road track from their Dakota Division.

        Can you give me the file name?

        Thanks a lot!!

        Comment


          #5
          The right way to refer to a file in the library here is by giving the exact zip file name. The next person can then easily use our search engine to find that exact file. Search result links as used in the message above are temporary and are not transferable to another user.

          Comment


            #6
            Sorry about that. My search query was simply "USGS" and the first file in the results is "dem_instructions.zip"

            Hope this gets you moving in the right direction
            -Matt

            Comment


              #7
              Hi Folks,

              Goku had mentioned in passing in a post on Elvas that someone had figured out a way to use or convert Geotiffs for use in TSRE. In the same thread I asked him for more details - he ignored my query.

              Regards,
              Scott
              <a href=https://www.trainsim.com/forums/filedata/fetch?filedataid=80663&type=full title=thumb_80663.png >thumb_80663.png</a>​ My Blender Models

              Comment


                #8
                Hey all,

                So the instructions worked, but I learned a few things trial-and-error that I'm going to post just in case they haven't been written down anywhere else:

                1) If you right-click on any of these legacy executables (train.exe, launcher.exe, demex.exe, etc.) and go to properties, there's a 'compatibility' tab. I set each of these apps to use the Windows version current when they were written (mostly XP). Doing that made Demex far more stable and Train Simulator ran much more quickly.

                2) I don't know if it's stated explicitly, but when you load images in Demex, the image(s) you load need to cover the entire route. At least when you first create the tiles.

                I tried to export just a single tile in Demex, using a TIFF that only covered that tile, but I don't think it worked properly.

                What I'm trying to do is use USGS 1/3rd second GeoTIFFs for the overall landscape, and then use 1M GeoTIFFs for the route tiles.

                I like that you can use the USGS 1/3rd second GeoTIFFs via Demex. It's a bit kludgy, but ironically, I believe that gives you higher res with a 20 year old app than you can get through the Dovetail Games, which I think are still restricted to the 1 sec SRTM data.

                However, it would be nice to be able to use that 1M data. I can't do all the tiles at once because the file size would be gargantuan even by 2022 standards, let alone what apps like Demex were written to handle.

                One other note on the 1M data: Its embedded coordinate system is UTM, so it has to be converted to WGS84. I use QGIS to do this, as well as to create tile sized mosaics from the 1M TIFFs.

                Comment


                  #9
                  Originally posted by scottb613 View Post
                  Goku had mentioned in passing in a post on Elvas that someone had figured out a way to use or convert Geotiffs for use in TSRE. In the same thread I asked him for more details - he ignored my query.
                  I actually went poking around in the TSRE5 source code and found a couple files (geotiff.h & geotiff.cpp) that suggest that functionality was at least anticipated for files covering 1 degree square, and the file naming convention is the same: [N/S][latitude][E/W][longitude].tiff (instead of ([N/S][latitude][E/W][longitude].hgt) But when I stuck .tiff files in the hgts directory, it didn't work.

                  Comment


                    #10
                    Originally posted by Zephyr06 View Post
                    Hi Richard,

                    The tutorial by Jerry Sullivan was exactly what I needed to get high-res USGS data manipulated for use in TSRE.

                    In the file library:


                    -Matt
                    So ... it took me until earlier this afternoon to realize that I don't need to use the "Load Heights" tool in TSRE5.

                    And because TSRE5 is MSTS compatible, I could create a route in TSRE5 and then add terrain using DEMEX.



                    Ah well, live and learn.

                    The only thing left on my 'to-do' list is figuring out the proper coordinate system to use with these .5M resolution TIFs:
                    This is the Original Product Resolution (OPR) Digital Elevation Model (DEM) as provided to the USGS. This DEM is delivered in the original resolution, with the original spatial reference. These data may be used as the source of updates to the seamless layers of the 3D Elevation Program, which serves as the elevation layer of the National Map. These data can be used by scientists and resource managers for global change research, hydrologic modeling, resource monitoring, mapping and visualization, and many other applications.


                    The source files are UTM, and DEMEX doesn't seem to be able to place them correctly.

                    Comment


                      #11
                      Originally posted by RichardAJensen View Post
                      So ... it took me until earlier this afternoon to realize that I don't need to use the "Load Heights" tool in TSRE5.

                      And because TSRE5 is MSTS compatible, I could create a route in TSRE5 and then add terrain using DEMEX.
                      I went through the same process, and now I'm realizing the tutorial I referenced didn't headline the fact that you would ultimately be using DEMEX to generate the terrain, not TSRE's "Load Heights". Fortunately, an activation key has been shared in the file library, and DEMEX's features are still extremely useful when creating and modifying route geography.

                      Comment


                        #12
                        Originally posted by Zephyr06 View Post
                        I went through the same process, and now I'm realizing the tutorial I referenced didn't headline the fact that you would ultimately be using DEMEX to generate the terrain, not TSRE's "Load Heights". Fortunately, an activation key has been shared in the file library, and DEMEX's features are still extremely useful when creating and modifying route geography.
                        I'm an inveterate tinkerer, and I honestly don't mind fiddling with stuff to get it to work.

                        I still joke about wanting to buy an old British car so I'll always have something to fix on the weekends.

                        Comment


                          #13
                          Here's an update...

                          So, the .5M USGS geoTIFFs are embedded in a UTM grid which DEMEX does not recognize (when you load one of these geoTIFFs in DEMEX, it reads the easting & northing as longitude and latitude respectively, with problematic results).

                          If you convert the geoTIFFs from UTM to WGS84 or NAD83 using QGIS or another app, DEMEX can't align the geoTIFF with the route tiles properly (DEMEX apparently uses UTM as a basis, even though it reports coordinates in lat/long).

                          However, I was able to figure out a round-trip that works properly:

                          1) I stitch together the .5M geoTIFF files in QGIS
                          2) Open the stitched geoTIFF in VTBuilder
                          3) Save the geoTIFF as an XYZ file (warning: It will be enormous--possibly over 1GB)
                          4) Open the XYZ file in DEMEX (I find it works best to run DEMEX in compatibility mode under Windows XP SP3). It will take quite a while to import the file, but DEMEX was able to import a 2.7GB file without crashing on my 16GB laptop.
                          5) In my case, DEMEX places the file in the proper latitude, but the longitude is (for some unknown reason) exactly six degrees offset to the east.

                          I'm trying to figure out a way to generate a shapefile for the route tiles that can be imported in QGIS.

                          But in any case, I think it's pretty impressive that you can import terrain that is accurate down to ~20"x20" squares into an app that is over 20 years old!

                          Click image for larger version

Name:	Screenshot 2022-06-17 193630.jpg
Views:	1
Size:	79.3 KB
ID:	2204115
                          Last edited by RichardAJensen; 06-17-2022, 07:52 PM.

                          Comment


                            #14
                            I'm resurrecting this topic..

                            I had a few other projects that occupied a lot of my free time over the past year, but I'm back.

                            I've had to modify the workflow required to get more precise DEM data into TSRE/Open Rails.

                            Here's the step-by-step process:

                            0) You'll need to have QGIS installed on your computer.

                            1) Go to the National Map downloader TNM Download v2 (nationalmap.gov)​. Download 1/3rd arc second (NED) GeoTIFFs from the USGS website. Each file covers an area one degree square. These files can be imported directly into DEMEX.

                            2) Go to the USGS LIDAR explorer USGS Lidar Explorer Map (nationalmap.gov)​. Here you can download 1M resolution GeoTIFFs. These files cannot be imported directly into DEMEX.

                            3) In QGIS, import both the NED and 1M GeoTIFFS. It is helpful to put the NED TIFFs below the 1M TIFFs.

                            4) Go to 'Raster > Miscellaneous > Merge'. Select all of the 1M TIFF layers that share a footprint with one NED TIFF (each TIFF will be on a separate layer with the same name as the file). Export this file to temporary directory.

                            5) By default, this new image will be placed in a new layer on top of the other layers. Right click on the layer and choose 'Export > Save As'. In the dialog that pops up, you'll need to change the coordinate system to EPSG:4269 - NAD83. This is the same coordinate system as the NED TIFFs. Save this new TIFF to a temporary directory.

                            6) Go to 'Raster > Miscellaneous > Merge' again. This time select the layer with the GeoTIFF you just created as well as the layer with the NED GeoTIFF. The new file that you create will be the basis for DEMEX terrain.

                            7) Go to DEMEX, select your route and the GeoTIFF you just created, and create terrain tiles as normal.

                            I could put together a more thorough explanation, if the community is interested.

                            --

                            Why the complicated process? Because the USGS has bundled 1M LIDAR data into handy TIFFs that use the UTM coordinate system, and converting these TIFFs to NAD83 within QGIS creates files which, for some reason, DEMEX stretches, so that they extend farther north and south than they should. I'm not sure how or why DEMEX does this, because the correct latitudes are unquestionably part of the TIFFs as well as any derivative file created (such as an XYZ).

                            The only resolution I've been able to come up with is to merge these GeoTIFFs with a larger GeoTIFF that DEMEX can import with no issues.

                            While 1/3rd arc second data, which runs about 10 meters between elevation markers, is a closer match to the 8 meter grid used by MSTS and Open Rails, it is still less accurate than the MSTS grid. Going through this hassle to import one meter data might seem a bit much, but not by as much as you'd think.

                            With 10 meter spacing, two elevation points in an MSTS tile will fall between two NED points on a periodic basis and tile elevation points can be as far as four meters from the closest NED elevation. With one meter spacing, no elevation point on an MSTS tile will ever be more than half a meter from a corresponding USGS elevation point.

                            This may not be a big deal with routes through, oh, say, Kansas (or huge chunks of eastern South Dakota, for that matter), but it's huge when the terrain is even mildly interesting. I might also add that the 1M data makes a big difference when it comes to rendering railway and highway embankments, etc., even when the overall terrain is fairly flat.

                            Comment


                              #15
                              Originally posted by RichardAJensen View Post
                              ...3) In QGIS, import both the NED and 1M GeoTIFFS. It is helpful to put the NED TIFFs below the 1M TIFFs.

                              4) Go to 'Raster > Miscellaneous > Merge'. Select all of the 1M TIFF layers that share a footprint with one NED TIFF (each TIFF will be on a separate layer with the same name as the file). Export this file to temporary directory.

                              5) By default, this new image will be placed in a new layer on top of the other layers. Right click on the layer and choose 'Export > Save As'. In the dialog that pops up, you'll need to change the coordinate system to EPSG:4269 - NAD83. This is the same coordinate system as the NED TIFFs. Save this new TIFF to a temporary directory.

                              6) Go to 'Raster > Miscellaneous > Merge' again. This time select the layer with the GeoTIFF you just created as well as the layer with the NED GeoTIFF. The new file that you create will be the basis for DEMEX terrain.
                              Let me be the first to say, this is really ground-breaking stuff. I've made good use of 1/9th arc DEMs where they are available, but I've never been able to get the 1M data to work out for exactly the reason you indicated - it would always scale weirdly north-south. I'll be giving your method a try soon and seeing if I can make it work. If you are so inclined, any additional detail you can add to steps 3 through 6 would be greatly appreciated. I fully agree that there is a very noticeable improvement going higher resolution than 1/3rd arc, especially in the mountainous areas that I love most
                              ~Sean Kelly~
                              SP Shasta Route for Open Rails: https://www.trainsimulations.net/sp-shasta
                              MRL Mullan Pass for Open Rails: https://www.trainsimulations.net/mullanpass

                              Comment

                              Working...
                              X