This object is in archive! 

GC Offlinizer - skip already downloaded

Wolfgang shared this idea 7 years ago
Completed

Hi,


I'd like to have a checkbox to skip already downloaded pictures for the GC Offlinizer.


The reason is, before I leave (for holidays) I have a lot of caches imported. But sometimes the download crashes and so I'd like to start a new run.


Thanks Wolfgang

Replies (9)

photo
1

Good day Wolfgang,

this is quite complicated task. How should Locus know, if all images are already downloaded?

- if you use "Live API", Locus firstly check, which images download over internet, and then download it. So it's almost impossible task here.

- if you anyway do not use "Live API", Locus only check required images that was found in listing. So if they are already downloaded, this check should be quick without big delay even for a huge list of caches.


Is if working fast for You in case, you do now use "Live API"?

photo
1

Hi Menion,


I thought setting a flag for a cache would make it easy to determine if already caches have been downloaded or not.


Because I think the live api, with no checkbox at spoiler, is the best which takes all images I used this setting mostly like that. But sometimes I had caches in the listing on which one it stopped/crashed downloading or maybe the WLAN disappeared because of some other reason. And so I stopped using it because of these problems. A friend of mine reported similar problems.


I just did a test approx 100 caches got fetched in 8 minutes.


But there is another problem:

I have e.g. 5000 caches in my locus DB and I fetched all images. A few days later I update this DB, always with delete before import. Reason I found 10 caches and there are 10 new caches in the area. I don't want to keep the found ones and need the new data from my GSAK export.

So now I have to fetch all images once again. And so on. -> I see the idea with the flag wont work here any more :(


That means you should know if there are any images already downloaded before you check the online listing of that cache. So we need no additional flag for the cache, but you need to identify from the GC number the corresponding local images.

Hmm but what's with caches without images. For them could maybe written a dummy file, so we know there aren't any images for that listing.


I don't know how the Offlinizer actually works and handles the images. I'm afraid of a total rewrite. I don't think this would it be worth it.


The test is still running now after 20 minutes 280 caches.

Not just regarding the time it takes I also think at the Groundspeak Servers - I generate a lot of requests.


I currently also got some errors - but it's still running. I don't know the reason of this download errors. To get them maybe fixed I have to do a complete run once again.


So because of these reasons for me the GC Offlinizer is a great feature, but only useful for some few filtered caches. Not for the whole DB.


Menion, please don't understand me wrong. I'm not complaining I just want to show you my problems. I like the whole locus - it's the best I every used the last 10 years of Geocaching. I only have some ideas/problems.


Thanks Wolfgang

photo
1

Hello Wolfgang,

understand, thanks for explanation. No problem and no worry about your feedback!

Main problem with "Live API" is that it is slow by intent. Between every request is few second break to be even slower. Reason for this, is limit on geocaching.com server, where in case of too quick requests, server stop respond and block next requests, crazy.

Anyway few notes:

1. from my point of view, you should check if using "Live API" is really benefit for You. Because except few rare cases, there should be no difference. Difference is made when using Live API also also "Spoilers" option.

2. consider if you really need all images for 5000 caches. I'm using offlinizer when I'm going somewhere, but I always sort caches in folder by distance to current map center (which is where I wants to go) and use "offlinizer" for first X (100, 200 max) caches. No reason to download them all .. wasting of my time and "their" servers

3. errors during download are usually caused by already invalid link to image (not existing anymore)

I'll keep this problematic in my mind during next developing.

photo
1

Hi Menion,


taking your order:


1. ok you'll say usually there is no difference between grabbed images if I use API version or not. In most cases only the spoilers are sufficient, but I think this option only works with images that have the file description "spoiler", but that's not allways the case...


2. my problem is I do may day plannings very spontaneous from my holiday hotel / B&B or whatever. So I put all caches of the region in locus and decide in the morning what to do.

E.g. this summer I was in Ireland I had half of the caches of Ireland on my device. Or when I stay at the Lake of Constance over a weekend I have almost 8000 Caches with me. I can't plan which one I'll need. I do not always have internet on may accommodation.


3. good to know


Thanks for keeping in mind this things for next developing :D


Wolfgang

photo
1

Understand.


Consider as implemented. You will see in next version. It should work anyway if cache is "offlinized" will be based only on fact, that there are already downloaded images or not. If owner change listing and add some images, this method won't detect it.

photo
1

>Consider as implemented.

i expect there is a checkbox where user can decide what he want to do !

photo
1

I'm late to this conversation, but I think I want the opposite of what the OP wants.


I had real problems with the offlinizer one day: it was stalling, failing to load anything at all. So I gave up. At one particular cache later in the day, a cache which I'd attempted to offlinize over breakfast (and which failed), the Images tab contained 1) a broken-image icon, and 2) a tiny horizontal sliver of an image, useless. All my attempts to re-offlinize this in the field failed. Probably because, it already offlinized, why should it do it again? I can tell you why, because it failed and only thinks it succeeded.

photo
1

Hmm understand. I'm not anyway sure how often you should need to force Locus to remove all downloaded data and download them again. Such rare cache may be done by manually delete directory in Locus/data/geocaching/... with downloaded data.

photo
1

BIG THANKS for realizing !

photo
1

Better thank me later when you will be able to confirm that it works as expected :).

photo
1

Merry Christmas Menion !

You're right :)


Maybe two more things ;)


Looks like already downloaded pics are mentioned as "GC...Finished , nothing to download" in the list. Would it be possible to mention "skipped, already downloaded" ?


How am I able to delete (probably all) downloaded images? There is a menu in the settings -> miscellaneous -> clear data. But there is not mentioned if images are deleted too.


Thanks. Wolle

photo
1

Hello Wolfgang,

oki, in case of "skipped" cache, there will appropriate text.


Clearing images > two methods.

a) automatically in settings > misc > clear data > Unused geocaching data ... this clear all downloaded images that are not anymore needed (cache is not anymore in Locus)

b) manually by deleting folder with images. For example cache GCABCDE has a directory Locus/data/geocaching/E/D/GCABCDE


Hope this helps.

photo
1

I just stumbled over this thread, cause I wasn't sure about the option "Skip downloaded" of GC Offlinizer, since it's missing in the manual. Could you please add it's behaviour to the manual?


I'm always using GCO with "Use Live API" active to get each image. I thougt that the option "Skip downloaded" would be to save time and data to don't load already downloaded images again? I tried the behaviour of an already downloaded Cache again, now with if "Skip downloaded" INACTIVE - but the messege stays "... Finished. Nothing to download". So the same behaviour i would get with "Skip dowanloaded" active where it says "... Skipped. Nothing to download". So what's the advatage of this option since Locus seems to check anyway if an image has been downloaded before?


And finally a small suggestion: on top of GC offlinizer is written:

"Do you want to pre-download all data (images in listings, ... ) for selected x caches?"

Since GCO is JUST for predownloading images and not for any other Cachedata it would be better to write: "pre-download Listing-images for selected x caches"

so people don't confusing it with "update caches" or update Logs, etc.


And a last suggestion ;-)

Could you mention within the dialog that if people want to download all images including SPOILER-images, they have to activate "Use Live API". Maybe with a tiny grey comment below that option. Or simply by renaming it to "Additionally load Spoiler-images"

photo
1

Hello sonny,

difference: try it with 1000 caches with API access and you will see. For every cache for which app downloads images, creates additional HTML file in Locus/data/geocaching directory.

Detailed description: app firstly tests necessary images in listing (or directly on gc.com if you use API), then downloads them and then into mentioned HTML write information about downloaded images. If you enable "skip downloaded", in case this HTML file exists, this cache is skipped. This is huge time saver mainly if you use API.

About your suggestions: thanks, I'll look at it.

Replies have been locked on this page!