Precise SRTM .hgt file handling

Markus Gpsmax shared this question 22 months ago
Answered

How is Locus Handling the .hgt Files ?

I mean, the SRTM format is same as BIL. Difference: just with defined pixel rows and clumns.

Is Locus able to handle a bigger .hgt file ? Do Locus calculate the pixel-spacing or just use one of the 2 normal possible ?

I want to use a much more detailled height model ! (Lidar Data with 50cm Spacing or 1 m)

Comments (24)

photo
1

Good day Markus,

if you have data prepared, best is to give it a try. I think it should work (and if not, create a bug report and attach one file for test). Only thing I'm little worried is fact, that app load necessary HGT files directly into memory for a lot faster work. 1m files will be probably really huge so there will be need for some automatic optimization and disabling this system. Anyway give it a try and you will see, I've personally never tried/used such huge HGT files.

photo
1

Ok, Thanks for fast Answering

I use GlobalMapper to export my Data to the .bil Format. (Motorola BigEndian Byte-Order) rename it to .hgt

Delete the corresponding File that Locus downloaded and replace it with my "new" one.

When I export my Data as .hgt -File (SRTM 1") i get it working but with the standard resolution. And it seems also to be very close to .bil format. (I am just able to export as .bil in custom Resolution)

I think i should keep the square Area of the Data so that Locus maybe is able to use it. (I don´t know how Locus parses the Data).

So I give it a try the next Days.

photo
1

I'm not sure if simple rename will work, you will see. I remember I worked with bil format some time ago, but it's direct internal support was removed few years ago.

App should anyway work with all HGT files, no matter how precise they are.

photo
1

It is working when i export as .bil file with same Resolution than 1 arc second. (so export as .bil and just rename works)

So it seems now that locus is just using one of the standard resolutions for srtm-files and can´t handle other resolutions.


Maybe if we say all files contain Data for 1 square arc second tiles. So we are able to calculate the point-density.

And locus load just a maximum count of opints to working memory (ram) ? to save memory.


My task is to use it in a mountain Rescue Team, and when we go to unknown terrain sometimes it helps to see in high detailled height model where are holes or steep rocks. or get back height values.


OK i can load a Hillshade-Map with high detailled Raster hillshade. but then i don´t have the right height data.

photo
1

Good day Markus,

I'm checking implementation of this feature and for me it should work, no matter how big are HGT files.

May you please share one file with me for test? Thank you.

photo
1

OK, yesterday i tried again. I exported as 0,1" grided Data. But Locus can´t work with it well.

If i Export as 1" grided Data then it works fine. (same method of Export as .bil, then rename to .hgt) I try to Attach a File with 0,5" Resolution. I hope the Size is not too big.

Are you sure Locus don´t just decide if Data is 1" or 3" ? Because in .hgt Files there is just a Grided Data, not the coordinates for each point. So Locus have to know how the Resolution of the Data is. But Locus also don´t work with Header Files... so... How do Locus find out the Resolution ?

photo
1

For me Locus is really the best mapping app on Android, so i hope my task is solveable. Now I try to generate 0.5" grided Data and attach. I don´t calculated but i think the Filesize is around 200MB for a 1 degree cell. I try it with my Region N47E009.

photo
1

Good day Markus,

our help desk most probably refuses upload of such huge file, so please use any Google Drive or similar service, thanks.

And app detects resolution of HGT file thanks to filesize. Size has to be always the same for the same resolutions.

photo
1

ok, i found out where the Mistake was..

My Data is in Meter accuracy, so exported them to grid resolutions for example: i tried 3m to 3m grid resolution.

But Locus Needs a binary dividable raster resolution of 1 arc Second. So possible is: 0.5 arc second grid spacing, 0.25" spacing, 0.125" spacing, 0.0625, ..., so ist not possible to make exactly 10m Resolution, but something close to.


So now it is working fine. thanks. I get really precise heights and also a Hillshade. (I see little rivers and also roads on Hillshade)

My next Task is to find a Software that generates better .bil data. (=.hgt data with Motorola Byte Order)Because Global-Mapper start produce Little stepps when use resolutions finer than 0.25 arc seconds as Export.

Or Maybe i let this data at a Resolution of 0.125" (close to 3m Resolution) and use a Map-Overlay (Multiplied) with Grayscale shaded Image of Hillshade. (I also use this on my Garmin Device).

photo
1

You found a problem with your generated files: perfect!

I'm glad it works for you now. If there will be something I may help with, feel free to ask.

photo
1

And yes... I didn't use the right Resolution.

After you told me, that Locus uses the File-Size to "calculate" the Resolution, it was all clear. I have to use square resolutions in divided parts of 1arc second. Thank you much.

photo
photo
1

Hello

Now i have a question. When i plan a new Route manually (line of sight). And then look at the routes Height- Graph. Locus do not "interpolate" along the Route-Line, while viewing the graph ? Locus just usrs heights of the route points ?


Is it possible somehow to get a path profile ?


I think Garmin is doing it like: when you are in path-profile window, they "interactive" take tighter height points "below" the line of sight, while you zoom in.

photo
1

Hello Markus,

when you enable manual drawing that Locus should interpolate between the start/end points of the line for usage in Route planner. Usually, it should generate points every 100m. You see at the bottom chart only straight line? You may even slide up the bottom chart and then move with start/end point to see changes in the chart immediately! let me know if this works for you.

photo
photo
1

Ok, every 100m a point, so Locus don't show meters deep stream ways, when not hit by the 100m raster. Is there possibility to improofe this ?

photo
1

Understand. Quite a specific task, but let's try it ...

photo
1

OK, I tested it. And YES, much better. I tested it by making a manual Route uphill out of a quarry. Let me describe: its a hole where are roads around in circle and Hill-upwards in steps.

And now i see the Steps of the Roads like in a real Cut of the Terrain. Just because of the limitation to 5m i sometimes don´t see one of the road-steps. (I need to play around with the 2nd Route point so that the Interpolation hits also this road)

So is there a reason why the minimum Value is limited to 5 m ?

I think its an Option, because allways changeable back to 5m or 100m.


If you give me the option to go down to 1 m interpolation, I try to make even better height Model of my Region.

At the moment i have about 3m stepping model, OK, locus is a little bit slower in panning around (when Hillshade is turned on), but i also switch off Hillshading sometimes, because i use LoMaps overlayed by a Grayscale-Hillshade-Map (more steep = more dark). As Overlay Method I use Multiply or Darken. So I normally don´t need the Hillshade Option in my Region. And Locus is really fast in overlaying this 2 maps. Panning and zooming is so REALLY FAST.

I need the .hgt data just for getting good Profiles and for measuring Height differences exactly. My Basis-Data is normally 30 cm exact in height. (75% of all the points, most of other points are 50cm exact)

At all now Locus is really close to be GREAT for my Task. Thank you really much for this Tool.

Because I want to show Friends in the Mountain-Rescue-Team what Great Maps are possible today, I have sometimes special tasks. But now most of them are done great.

(Now we see where to get out best, where is a possible route in Offroad-Terrain,...)

photo
1

Hello Markus,

glad we have some progress. I choose 5 meters limit as the compromise and also some kind of security mechanism. The too low number may cause slow-down (in real-time interpolation when you move with one line point in Route planner), so I wanted to prevent setting of too low number. Anyway ok, let's try it to next version with the 1m option.

I'm surprised to read about elevation data with such huge accuracy mainly in mountains. Anyway, I'm always glad to read that Locus Map seems to be a helpful companion in useful professions.

photo
1

Hello

OK, Now i get really realistic profiles. I attached some Screenshots. And its also very good to get fast access to switch overlay on or off, to switch Hillshade on and off,... Yes Locus is really Good for my tasks.

Hm... Just some little thing is...

Ok, when there is a Road and i plan an automatically Route with Walking-Profile, i get a Height-Profile and its also the same when i open the saved route-details and go to diagramm (profile)

BUT not when i save my mannually planned route. When go to details and profile, i get just the profile of my manally set route Points.

My Idea was first: if the 1m thing works well... why don´t we make it one step ahead.. and make the interpolation density dependable on the x-Axis zoom of the Height-Profile screen. (so interactively get the data from background .hgt data).

So that we get really more detailled when zooming in the x-axis. and less detailled when zooming out. So the problems with too much points is also solved. Maybe change the new option in Expert-Mode-Settings to a Factor-Variable, so with this factor we can choose the relation between x-Axis and interpolation-density.

photo
1

Why is it working well in a automatically calculatet Route-Profile ?

But not in manually created.

Maybe you can helpe me with this also.

photo
1

Hi Markus,

in case of the automatically computed route, the route itself is made from many trackpoints which are then also visible on the chart. So there is no interpolation between points. In case of manually draw route, in the final route are really saved only manually defined shaping points and interpolation we talked about before, is lost.

I had a similar request on old forum a few days ago, related to applying correct line style also to these manually drawn lines, which is the exact same problem.

Saving manually draw segments interpolated with 1 m is little overkill for huge lines. Understand your use case and agree that what you see in the route planner should be visible in saved track as well. I'm just not sure how to simply technically do it. Because it is more complicated to do and because it currently has a low priority, I'm sorry for now, but can't help here now.

photo
1

ok, I think when going to route-details and to profile Page, there decide if its a Route or a recorded Track. (or in other words, if the route have recorded height-data or not). If its a Route without recorded height Data, then lets say get 800 height-Points of the whole "Route" and display it in the Diagramm. If you zoom in: then also take 800points but just of the visible Part of the Route (visible on Diagram x-Axis). So we get more detailled while zooming in. And Just if you export a Route to a Track, you can decide if you want to fill height data in or let it without height data. So its close to the thing when you analyze a segment of the Route. (Shift Start and End point along the Route while Zooming and Panning on the Height-Profile)

Just my ideas to this.

photo
1

I think, the easiest way at Moment: Save all the interpolated Points. (Standard-Setting is to 100m, so no Problem)

Because: if i record a Track with Settings to 1m or 1s i also get much Points. And as you told, when planning a Route by any Kind of automatic Route-Calculation, locus also use many Points (all Points of the Road-Line ?).

I think it is no Problem to save also, for manually created routes, all interpolated Points for the Height-Profile. (Users can Always set the Interpolation-Step back to 100m or so)

photo
1

One Other Question:

Can you add also a Parameter for "Heicht data multiplicator" as Standard set to 1.

My Problem: The height data Format is in 16-Bit integer values. So without making a Little "trick" my Software is just able to create Height-Grid in steps of 1m.

But, now i Exported my Data in "dm" Units. so i get 1000m instead of 100.0 m.

So Route-Calculated-Time is wrang and also other Parameters.

But if you add a factor to height-Data, then it will be perfect.

Thanks

Greets from Austria

photo
1

Good day Markus,

give a try next beta version, it will be improved little bit and interpolated points will be saved in final generated route. We will see how this will work.

About your second request: sorry, but such special one-user options will only overcomplicate UI of app. Thanks for understanding.

photo
1

ok, sure i understand. hm, i think its just a multiplication-facror of the height-data. if set to 1, there is not much cpu-extra load. And opens the possibility to use smooth height-data for everyone. In near future there will be available a worldwide high-precision height model.... ok, your argument for over-complication.

So thanks anyway. I can use 2 Files. 1 with Meters height data. (for standard use and for better time-calculation in routes) And one with deci-meter height-Values. (really smoothe display and smoothe %Uphill/%Downhill Graph)

photo
1

Hello Menion

Thanks for adding this Parameter.

When I make a Manual-Route (Line of Sight) and i make shorter Route-Point distances than 100m, I don´t get the interpolated Height-Profile of this section.

But this is really neccessary, because when i plan a Manual Route i have often route-Point distances smaller than 100m, (And so no Interpolation between them)

So in the Software there is somewhere an additional Parameter which is fixed to 100m. (Minimum route Point distance for Interpolation)

Maybe you can set this Parameter equal to the "new" Parameter in Expert-Mode. (Interpolation-step)Reason: I think this can be same than route-Interpolation-step, because if someone wants tighter Interpolation steps, then the Minimum route-Point-distance also should be smaller.

At the Moment i Always have to make longer route Segments than 100m to get detailled Height-Profile. But in Terrain this is unusable. (I want to draw my Walking way down somewhere and get the height-Profile detailled)

Thanks

photo
1

Hello Markus,

you are correct. Hard-coded 100m parameter found. Will be fixed in next version 3.37.1, thanks.

photo
photo
1

Thanks for sharing of your shaded SQLite map and HGT file. Both downloaded and correctly displayed in app. And now what to do with it :).

Firstly, a quick test in route planner with visible SQLite map and drawn short profile show over some huge terrain changes show no problem to me.


The application uses bicubic interpolation from 16 interpolated points around the required location. In this interpolation is most probably not an issue because otherwise, it will be clearly visible. So the problem really may be in some shift of computed row and column indexes of elevation values.


The whole algorithm is little complex and because if there is an issue, it is really tiny, I cannot invest a day to finding exact reason of possible small shift of interpolated elevations, sorry to say it. We anyway planned to optimize shading in the app in the near future, so I'll keep this in my mind! Thanks for understanding.

photo
1

Thanks, for me Locus is a really good app. And maybe I manage somehow to shift my .hgt Data some Meters.

I see the Shift best when i plan a manual route, open Route height profile and tap on the Top of a falling Edge, then i see the cursor shifted some meters away from the cliff. Also when i plan a auto-route along a street. (The Street is exactly drawed to the lidar data in openstreetmap). Or by just gwtting heights from cursor position near a cliff.

photo
1

Hello,

I tried what you describe: "plan a manual route, open Route height profile and tap on the Top of a falling Edge". I tried this maybe on five places and on all it looks correctly. Give me please some start > end coordinates of line, where this will be clearly visible, thanks.

photo
1

Hello

I work with disabled Hillshade but enabled Overlay "ESG-Hillshade" multiplied mode. (100% non Transparent).As Map I use "Freizeitkarte ALPS+East" or also LoMaps-Austria.

I Start a Manual Route...

Start at Position : N 47°18.209' ; E 009°40.001'

End at Position: N47°18.172' ; E 009°40.022'


Then I tap on the Height-Profile Icon (bottom-right Corner).I tap on the middle of the first flat Region above bottom. (middle of the "Road")

The Cursor on the Map is then shifted. (because the Height-data are shifted to top-left. (About 5.5m to Top and 3m to Left(nearby 0.125 arc-Seconds = the Resolution of my Height-Data = one Pixel)

Or I see it better when I enable the Display of Height-Values at Cursor-Position. And move to an Edge, then the Values start rise at shifted Position.

Now I shifted just my .hgt Data (my Hillshade-Map "ESG-Hillshade" is the same as bevore).Its much better now. For the shifting i had to download the N47E008 Tile and the N48E009 Tile and shift them all. Then cut the Data to the exact Window of the original N47E009. (so I got additional Data at Top and Left but cutted Data from Right and Bottom = shifted Height-Data to Bottom-Right because Locus takes the new Data as N47E009)

I uploaded the New Data to my Google-Drive.

https://drive.google.com/file/d/1QYYcqnRP55ywqjw8W_X2IvbamoTSiXcc/view?usp=sharing

photo
1

Thanks Markus for the precise description ....

So after maybe two hours of testing (and half of hour of sleep after falling asleep during thinking :) ), I have this result for your two coordinates.

/969c29a2b92cc179624a51034d5d10ac

Better right? :). Seems to match your generated map a lot better.

photo
1

Ah back, sorry. This my "fix" was incorrect and instead, it created another issue.

We here compare the result of Locus Map vs result of some program you use and I tried to "fix" Locus Map to match data from you. Is there any other option how to test, which program gives correct results? Because I'm just unable to find a mistake in my old algorithm. All seems to match!

photo
1

Hm, yes. There is a projection problem. Its not just a shift. I exported with exactly x and y square 0.125 arcsecond resolution. And a wgs84 projection. I try to check this in my projection header files. The Thing is: when i open both Data in the professional GIS Sodtware it fits cm exact. (Hillshade and .hgt) But Difference is: my GIS Software uses the additional header files. And locus calculate from Filesize

photo
2

So i check what is inside the additional Files

photo
1

Ok, found a problem in my Data. I used the bounding-box from the original 1" .hgt file, to cut my Data to. But with my higher Resolution, the box don't hit the right points. So my .hgt had additional data, and locus did a stretch or push

Solution was: to cut Data to exact N47 and N48...

photo
1

Oki ... so at least I've confirmed that my algorithm is correct.

Menion

photo
1

Thanks for the Work on Locus. And sorry. My GIS-Software did open the .hgt-File always with the additional Projection files. So there it looked perfect matched. And my Idea to use the Bounding-Box from an original 1" .hgt File to cut my Data, was a mistake.