Support for zipped routing files on external SD card in Addon GraphHopper
Gathering feedback
The GraphHopper offline routing add-on currently uses the internal storage for routing files. It also requires the routing files to be unzipped prior to use. The England zipped file is 179 MB; the unzipped version takes 319 MB. As there is also a restriction on where the files are stored, this is very limiting.
Would it be possible to add support for storing the files on the external SD card, preferably without needing to unzip them. I realise having them zipped up would probably slow the routing down a little, but space is more limited for me than time!
Thanks for this extension and I'm looking forward to more developments!
I like this idea 
+1
btw. any news about GraphHopper (with Locus)
(have no link to last thread)
+1
btw. any news about GraphHopper (with Locus)
(have no link to last thread)
Good day Alan,
using zipped files is really impossible idea. I'm not a developer of GraphHopper engine, but can't imagine, this should work (anyway this is more on developer of GraphHopper, then on me as a developer of Locus).
Second part, optional directory is another story that depends only on me. Anyway as gynta correctly asked - "any news"? Currently not, really sorry, but I do not know what to do sooner :).
Good day Alan,
using zipped files is really impossible idea. I'm not a developer of GraphHopper engine, but can't imagine, this should work (anyway this is more on developer of GraphHopper, then on me as a developer of Locus).
Second part, optional directory is another story that depends only on me. Anyway as gynta correctly asked - "any news"? Currently not, really sorry, but I do not know what to do sooner :).
Hi Menion,
Thanks for the reply. I've done a bit more investigation; regarding zipped files, according to the "Maps" section of the graphhopper android page (points 5 & 6), graphhopper supports compressed data. It might just be a case of renaming the .zip files to .ghz (although possibly without the subdirectory inside the zip file). I can't try it with the existing Locus GraphHopper add-on at the moment as my android phone has died, but I will do so as soon as I have a new one!
Anyway, really looking forward to more development on this add-on: for me offline routing is the one essential feature that Locus is missing. I understand you probably have lots of other priorities, but I thought I'd just put my opinion in!
Thanks!
Al
Hi Menion,
Thanks for the reply. I've done a bit more investigation; regarding zipped files, according to the "Maps" section of the graphhopper android page (points 5 & 6), graphhopper supports compressed data. It might just be a case of renaming the .zip files to .ghz (although possibly without the subdirectory inside the zip file). I can't try it with the existing Locus GraphHopper add-on at the moment as my android phone has died, but I will do so as soon as I have a new one!
Anyway, really looking forward to more development on this add-on: for me offline routing is the one essential feature that Locus is missing. I understand you probably have lots of other priorities, but I thought I'd just put my opinion in!
Thanks!
Al
Hi Menion,
I haven't got as far as the ghz files yet, but I've been looking at external storage options. I downloaded your graphhopper project from bitbucket and made a couple of commits to:
It seems to work in that it finds files stored on the sd card in Android/data/com.asamm.locus.addon.graphhopper/files/graphHopper
However, I'm having problems getting it to do routing and I suspect this is because of my ignorance, so I was hoping you could help. I created a simple project in Android Studio that includes the locusAPI, the locusAPI_android and the locus-addon-graphhopper. I couldn't get it to compile until I removed slf4j-api-1.7.7.jar from the libs directory. With that file included, Android Studio complained:
With it removed, it compiles fine, but Locus can't do routing and the log shows:
At this point, I feel a little stuck! Do you have any ideas where I might go next?
My repository is here: https://bitbucket.org/abudden/locus-add-on-graphhopper/commits/all
Thanks,
Al
Hi Menion,
I haven't got as far as the ghz files yet, but I've been looking at external storage options. I downloaded your graphhopper project from bitbucket and made a couple of commits to:
It seems to work in that it finds files stored on the sd card in Android/data/com.asamm.locus.addon.graphhopper/files/graphHopper
However, I'm having problems getting it to do routing and I suspect this is because of my ignorance, so I was hoping you could help. I created a simple project in Android Studio that includes the locusAPI, the locusAPI_android and the locus-addon-graphhopper. I couldn't get it to compile until I removed slf4j-api-1.7.7.jar from the libs directory. With that file included, Android Studio complained:
With it removed, it compiles fine, but Locus can't do routing and the log shows:
At this point, I feel a little stuck! Do you have any ideas where I might go next?
My repository is here: https://bitbucket.org/abudden/locus-add-on-graphhopper/commits/all
Thanks,
Al
Good day Alan,
such interest from user(s) is what I like most :). More and more people seems to be interested in working solution in Locus. Do you know if roundabouts and one-way roads are already correctly working?
Anyway to your issue ... I tried also update to latest graphHopper library and result was same. So I removed log4j lib as you, refreshed whole project in Android Studio (refresh of gradle and also rebuild of project) and then it correctly compile. I then generated new map for my area from PBF file (you cannot use older files from Locus forum here) and all seems to work correctly for me.
So I suggest to clear everything, it usually helps.
Good day Alan,
such interest from user(s) is what I like most :). More and more people seems to be interested in working solution in Locus. Do you know if roundabouts and one-way roads are already correctly working?
Anyway to your issue ... I tried also update to latest graphHopper library and result was same. So I removed log4j lib as you, refreshed whole project in Android Studio (refresh of gradle and also rebuild of project) and then it correctly compile. I then generated new map for my area from PBF file (you cannot use older files from Locus forum here) and all seems to work correctly for me.
So I suggest to clear everything, it usually helps.
Hmmm... clearing it doesn't seem to have helped: same error message.
I had generated my own -gh folders, but it's possible it was done with a different version of graphhopper to the one embedded in my project repository (I used the github repository version), so I'll try again with the embedded one. I might have to go back to square one and start from scratch. It may be something with my build set up or an issue with the fact I'm generating a Great Britain -gh folder from the Great Britain pbf file rather than separate ones for England and Wales (I live very near the English-Welsh border). If it is an issue with different versions of graphhopper, it would probably be a good idea to publish the specific jar file for graphhopper with each locus-add-on-graphhopper release so those who feel able to can generate their own routing data.
According to the graphhopper issue tracker, roundabouts are sorted, so hopefully the issues are sorted, but I'm not sure.
Incidentally, the new version of graphhopper also supports a profile for motorcycles as an alternative to cars (favouring fun roads over straight ones). I couldn't see how to integrate this into Locus as it isn't available in VALUE_RTE_TYPE_* in ExtraData.java.
Maybe this is optimistic, but is there any chance you could send me your whole locus-graphhopper project in case there's something I've set up wrong? I've got a top-level project which includes the bitbucket project linked above along with locusAPI and locusAPI_android as described on your wiki page, but it could be that there is some setting in the top level project that is causing issues... obviously if it's got anything in there you'd rather not share then I completely understand.
Hmmm... clearing it doesn't seem to have helped: same error message.
I had generated my own -gh folders, but it's possible it was done with a different version of graphhopper to the one embedded in my project repository (I used the github repository version), so I'll try again with the embedded one. I might have to go back to square one and start from scratch. It may be something with my build set up or an issue with the fact I'm generating a Great Britain -gh folder from the Great Britain pbf file rather than separate ones for England and Wales (I live very near the English-Welsh border). If it is an issue with different versions of graphhopper, it would probably be a good idea to publish the specific jar file for graphhopper with each locus-add-on-graphhopper release so those who feel able to can generate their own routing data.
According to the graphhopper issue tracker, roundabouts are sorted, so hopefully the issues are sorted, but I'm not sure.
Incidentally, the new version of graphhopper also supports a profile for motorcycles as an alternative to cars (favouring fun roads over straight ones). I couldn't see how to integrate this into Locus as it isn't available in VALUE_RTE_TYPE_* in ExtraData.java.
Maybe this is optimistic, but is there any chance you could send me your whole locus-graphhopper project in case there's something I've set up wrong? I've got a top-level project which includes the bitbucket project linked above along with locusAPI and locusAPI_android as described on your wiki page, but it could be that there is some setting in the top level project that is causing issues... obviously if it's got anything in there you'd rather not share then I completely understand.
Agree about distribution of jar for generating, may be useful.
Thanks for info about roundabouts. If this really works, it brings me to long awaited moment - continue in work on this add-on.
About your issue: hard to say where should be a problem. Unfortunately i cannot share whole project with you, because I have around 30 packages together with GraphHopper add-on, together with Locus itself, in one. I've just generated release of 0.1.4 ...
https://drive.google.com/file/d/0B_Nr4CT8osmtVkVYWnZIOVdLZ1E/view?usp=sharing
... try it. For me it works correctly. If this won't work for you, there is probably something with generated routing data. I've used latest script from github to generating.
Agree about distribution of jar for generating, may be useful.
Thanks for info about roundabouts. If this really works, it brings me to long awaited moment - continue in work on this add-on.
About your issue: hard to say where should be a problem. Unfortunately i cannot share whole project with you, because I have around 30 packages together with GraphHopper add-on, together with Locus itself, in one. I've just generated release of 0.1.4 ...
https://drive.google.com/file/d/0B_Nr4CT8osmtVkVYWnZIOVdLZ1E/view?usp=sharing
... try it. For me it works correctly. If this won't work for you, there is probably something with generated routing data. I've used latest script from github to generating.
The distribution of the jar file might not be that helpful actually: I did some more digging and it looks like the one that's packaged up for inclusion with the add-on doesn't have the import class that's needed to perform the conversion.
Completely understand about not sharing the project: I wondered if that might be the case.
I've had a go with your new version and it works fine with my generated -gh folders, so it must be either (a) something wrong with my project or (b) something wrong with the code changes I made to make it work with the SD card.
To try and figure out what's wrong with my project, I guess one option would be for you to compile my bitbucket repository code in your project and we could see whether that works (then we'll know if it's the code or my project). Having said that, if you're planning to do some more development on this add-on, then you'll probably do it WAY better than I have anyway, so my project is arguably pointless!
Thanks again.
The distribution of the jar file might not be that helpful actually: I did some more digging and it looks like the one that's packaged up for inclusion with the add-on doesn't have the import class that's needed to perform the conversion.
Completely understand about not sharing the project: I wondered if that might be the case.
I've had a go with your new version and it works fine with my generated -gh folders, so it must be either (a) something wrong with my project or (b) something wrong with the code changes I made to make it work with the SD card.
To try and figure out what's wrong with my project, I guess one option would be for you to compile my bitbucket repository code in your project and we could see whether that works (then we'll know if it's the code or my project). Having said that, if you're planning to do some more development on this add-on, then you'll probably do it WAY better than I have anyway, so my project is arguably pointless!
Thanks again.
I though about whole pack with shell script, not just a jar. But not important now ...
Your project has two important benefits: you learned something new :), and you gave me small kick so things can start move. I'll try during next week dig a little more into this topic. Also I already asked my colleague last month for new data for GraphHopper add-on, but he probably forget. We should at least now generate data with a new generator for 0.5 version.
So if you do not have any special personal plans with extending an add-on, leave it as is and I'll improve it and publish on forum during next week (in work case during next 14 days). If you HAVE any plans with it, we may try to find out a reason why this happen, but to be true, I have currently no idea :(
I though about whole pack with shell script, not just a jar. But not important now ...
Your project has two important benefits: you learned something new :), and you gave me small kick so things can start move. I'll try during next week dig a little more into this topic. Also I already asked my colleague last month for new data for GraphHopper add-on, but he probably forget. We should at least now generate data with a new generator for 0.5 version.
So if you do not have any special personal plans with extending an add-on, leave it as is and I'll improve it and publish on forum during next week (in work case during next 14 days). If you HAVE any plans with it, we may try to find out a reason why this happen, but to be true, I have currently no idea :(
Hi Alan,
I've started some smaller changes in GraphHopper add-on and first bigger change is support for compressed ghz files. Anyway after import I get an error that file cannot be loaded. So I'm searching GH source code and look at this: https://github.com/graphhopper/graphhopper/blob/0.4/core/src/main/java/com/graphhopper/GraphHopper.java#L712
On line 712 starts code that handle zipped files and first step, is extract of this directory to directory from where this file comes. So it seems that routing data are not read directly from ghz file, but file is firstly extracted. Then support for ghz is not a big help.
Hi Alan,
I've started some smaller changes in GraphHopper add-on and first bigger change is support for compressed ghz files. Anyway after import I get an error that file cannot be loaded. So I'm searching GH source code and look at this: https://github.com/graphhopper/graphhopper/blob/0.4/core/src/main/java/com/graphhopper/GraphHopper.java#L712
On line 712 starts code that handle zipped files and first step, is extract of this directory to directory from where this file comes. So it seems that routing data are not read directly from ghz file, but file is firstly extracted. Then support for ghz is not a big help.
Hi Menion,
Oh. That's a shame. Ah well, never mind: it was a nice thought.
Have your smaller changes included support for external SD card storage? If so and you'd like someone to test it out I'd be very happy to have a go and let you know how I get on with it!
Thanks
Al
Hi Menion,
Oh. That's a shame. Ah well, never mind: it was a nice thought.
Have your smaller changes included support for external SD card storage? If so and you'd like someone to test it out I'd be very happy to have a go and let you know how I get on with it!
Thanks
Al
Replies have been locked on this page!