Exported FIT files have missing time_created since v4.23.0
Hello,
I am having issues with FIT files exported from Locus Map.
It looks like this started happening after either Locus Map v4.22.0 or v4.23.0 was released - I suspect v4.23.0 as I notice that there was a fix included with description "issues with FIT files(export to Garmin watches)". I suspect this change introduced another bug. I can't be sure as I didnt create/save any FIT files between 4.22.and 4.23.
My workflow is to export the track from Locus in FIT format with FIT mode = "Course". I then copy this to my USB connected Garmin watch into folder Garmin/NewFiles, then disconnect the watch which then automatically processes the new file into Garmin/Courses and it is then visible in the Watch itself.
Prior to V4.22.0 I could load several tracks onto my watch in this manner, the FIT files either coming from Locus or any other source.
After V4.23.0 what happens is that the first file from Locus loads OK, but any subsequent files from Locus get rejected by the watch. After I copy the file into NewFiles, then disconnect the watch, the track does not appear on the watch and is not present in Garmin/Courses. I cannot see any information on the watch why this happened (no log or similar). It simply rejects and deletes it. Note that FIT files from other sources, or from Locus prior to v4.22.0 continue to load OK.
I have analysed what is happening using tools:
FIT file viewer: (https://www.fitfileviewer.com/)
The Garmin FIT SDK CSV Tool
Fit File tools: https://www.fitfiletools.com/
I can see that exported FIT files from v4.23.0 onwards have invalid timestamps in some records:
File_ID : time_created is invalid - missing in fact. It is present in the Definition record but always absent from the Data record.
Lap: timestamp and start_time are sometimes missing. Not sure why only sometimes.
Event (both types "start" and "stop_disable_all") : timestamp sometimes missing.
After some experimenting it seems that the Garmin device rejects any input FIT file which has the same time_created value as any previously loaded file. If time_created is missing then I guess it takes it as zero, hence it accepts the first one then rejects any further such files. I cannot prove this is the cause as obviously I don't have visibility of the actual logic used by Garmin but it seems highly likely.
Please would you look into this and ensure that valid times/dates are written wherever they should be present
Thanks
Alan
The same problem 
Hi,
Some more information.
1/ It seems that:-
For the "LAP" and "Event" records the date_time fields are missing if the track was created using the Locus app route planner.
If the track was imported from an external GPX and then exported as a FIT then those fields are present in the FIT. The GPX I'm using has <time> values for every trackpoint and waypoint.
For the "File_ID" record the time_created field is always missing
2/ One workaround is to post-process the FIT file with the "Fit File Tools" tool - Time Adjuster.
3/ The Garmin FIT SDK documentation makes it clear that all these date_time fields are mandatory.
File_ID: https://developer.garmin.com/fit/file-types/
Others: https://developer.garmin.com/fit/file-types/course/
which also says:
Timestamps
Many of the messages used in Course files have date_time fields eg. timestamp, start_time, etc. The purpose of the date_time fields is to provide relative time offsets in between points or markers along the course. The start_time value is an arbitrary value and can be any valid date_time. What is important is that all date_time values in the file are relative to the start_time.
----
I expect that if these fields are missing then it is likely that there could be various unpredictable results when loaded onto Garmin devices.
Hi,
Some more information.
1/ It seems that:-
For the "LAP" and "Event" records the date_time fields are missing if the track was created using the Locus app route planner.
If the track was imported from an external GPX and then exported as a FIT then those fields are present in the FIT. The GPX I'm using has <time> values for every trackpoint and waypoint.
For the "File_ID" record the time_created field is always missing
2/ One workaround is to post-process the FIT file with the "Fit File Tools" tool - Time Adjuster.
3/ The Garmin FIT SDK documentation makes it clear that all these date_time fields are mandatory.
File_ID: https://developer.garmin.com/fit/file-types/
Others: https://developer.garmin.com/fit/file-types/course/
which also says:
Timestamps
Many of the messages used in Course files have date_time fields eg. timestamp, start_time, etc. The purpose of the date_time fields is to provide relative time offsets in between points or markers along the course. The start_time value is an arbitrary value and can be any valid date_time. What is important is that all date_time values in the file are relative to the start_time.
----
I expect that if these fields are missing then it is likely that there could be various unpredictable results when loaded onto Garmin devices.
Also found this :
The file_id message identifies the format/content of the FIT file (type field) and the combination of fields provides a globally unique identifier for the file. Each FIT file should contain one and only one file_id message. If the combination of type, manufacturer, product and serial_number is insufficient, for example on a device supporting multiple device files, the time_created or number fields must be populated to differentiate the files.
----------------
All FIT files from Locus appear to have same values for type, manufacturer, product and serial_number, so time_created must be set to differentiate them.
Also found this :
The file_id message identifies the format/content of the FIT file (type field) and the combination of fields provides a globally unique identifier for the file. Each FIT file should contain one and only one file_id message. If the combination of type, manufacturer, product and serial_number is insufficient, for example on a device supporting multiple device files, the time_created or number fields must be populated to differentiate the files.
----------------
All FIT files from Locus appear to have same values for type, manufacturer, product and serial_number, so time_created must be set to differentiate them.
Note: version referred to be above should be v4.23.1
I've also tested v4.22.2. date_time fields in this version, in the example I created, are:
File_id: time_created is PRESENT.
Lap: timestamp and start_time both absent
Event: timestamp absent.
Course and Record rows: timestamp missing.
So it seems the problem arose in V4.22.x , there was a partial fix in v4.23.x but this also cause time_created to be omitted.
Note: version referred to be above should be v4.23.1
I've also tested v4.22.2. date_time fields in this version, in the example I created, are:
File_id: time_created is PRESENT.
Lap: timestamp and start_time both absent
Event: timestamp absent.
Course and Record rows: timestamp missing.
So it seems the problem arose in V4.22.x , there was a partial fix in v4.23.x but this also cause time_created to be omitted.
Hi Alan,
thanks for your thorough report. The bug has been fixed. The new app version is to be released next week.
Hi Alan,
thanks for your thorough report. The bug has been fixed. The new app version is to be released next week.
Thanks for the quick action. It partly fixes the issue but not entirely:-
I notice that the date/time used to fill the FIT file file_id time_created field is that of the export of the FIT file itself and not that of the creation (or update) of the actual track, as recorded in the time_created (or time_updated) column in the Locus tracks.db track table.
This means that if several Locus tracks are selected together and exported at the same time there is a likelihood that more than one will be given the same FIT file_id time_created value, as this time is to the nearest second. If these are then loaded to a Garmin device the same issue will arise - duplicates get rejected. (I tried this and it did happen).
If the actual Locus track creation (or update) date/time was instead used this would be extremely unlikely to occur.
(An alternative would be to put a unique distinguishing value into the FIT file_id "number" field, which currently is not set by Locus)
For myself I can remember to export them one by one, but others may not realise this and might encounter the same issue in future.
Thanks for the quick action. It partly fixes the issue but not entirely:-
I notice that the date/time used to fill the FIT file file_id time_created field is that of the export of the FIT file itself and not that of the creation (or update) of the actual track, as recorded in the time_created (or time_updated) column in the Locus tracks.db track table.
This means that if several Locus tracks are selected together and exported at the same time there is a likelihood that more than one will be given the same FIT file_id time_created value, as this time is to the nearest second. If these are then loaded to a Garmin device the same issue will arise - duplicates get rejected. (I tried this and it did happen).
If the actual Locus track creation (or update) date/time was instead used this would be extremely unlikely to occur.
(An alternative would be to put a unique distinguishing value into the FIT file_id "number" field, which currently is not set by Locus)
For myself I can remember to export them one by one, but others may not realise this and might encounter the same issue in future.
Hi Alan,
we'll be releasing one more bugfix tomorrow so please test the issue with it.
Hi Alan,
we'll be releasing one more bugfix tomorrow so please test the issue with it.
Hi,
Thanks. I tested it and I think that solves the immediate problem as stated in the original issue.
(Note: Garmin Connect also uses the course creation date/time in exported FIT files so I think it is the correct thing to do)
--------
FYI for completeness and in case you want to do anything more in this area I'd better also add the following:-
When testing I realized that with this change the same issue could arise in some other rarer circumstances. For example:
1/ If in Locus a track is copied, then adapted, the create time is also copied , so if the two tracks were both exported as FITs they would clash if loaded to a Garmin device.
2/ If GPXs, with timestamps on the trackpoints, are imported to Locus then the Locus track time_created value is taken from the timestamp of the first trackpoint. I know of at least one GPX plotter/exporter site (plotaroute) which sets that timestamp to the start of day. This means if you plot several tracks in plotaroute on the same day, transfer the GPXs to Locus then export as FITs then these FITs would clash. I realised this as that is what I sometimes do ! (though I'm trying to move to using Locus to plot routes)
There might be other unforeseeable scenarios.
Maybe the only certain way to be sure of no clashes, if you want to do that, is to also put some unique number in either the "serial_number" (uint32) or "number" (uint16) field (or both) of the FIT file_id record.
I had a look what Garmin Connect does and its sets the serial_number value to a numeric ID (currently 9 decimal digits), unique for that course and it seems to set "number" always to "1". Currently Locus sets serial_number to fixed value "1701" and doesn't set "number".
So I suggest if you want to avoid these FIT clashes 100% in future then generating a unique ID for the FIT file into serial_number could be the way to do it.
Hi,
Thanks. I tested it and I think that solves the immediate problem as stated in the original issue.
(Note: Garmin Connect also uses the course creation date/time in exported FIT files so I think it is the correct thing to do)
--------
FYI for completeness and in case you want to do anything more in this area I'd better also add the following:-
When testing I realized that with this change the same issue could arise in some other rarer circumstances. For example:
1/ If in Locus a track is copied, then adapted, the create time is also copied , so if the two tracks were both exported as FITs they would clash if loaded to a Garmin device.
2/ If GPXs, with timestamps on the trackpoints, are imported to Locus then the Locus track time_created value is taken from the timestamp of the first trackpoint. I know of at least one GPX plotter/exporter site (plotaroute) which sets that timestamp to the start of day. This means if you plot several tracks in plotaroute on the same day, transfer the GPXs to Locus then export as FITs then these FITs would clash. I realised this as that is what I sometimes do ! (though I'm trying to move to using Locus to plot routes)
There might be other unforeseeable scenarios.
Maybe the only certain way to be sure of no clashes, if you want to do that, is to also put some unique number in either the "serial_number" (uint32) or "number" (uint16) field (or both) of the FIT file_id record.
I had a look what Garmin Connect does and its sets the serial_number value to a numeric ID (currently 9 decimal digits), unique for that course and it seems to set "number" always to "1". Currently Locus sets serial_number to fixed value "1701" and doesn't set "number".
So I suggest if you want to avoid these FIT clashes 100% in future then generating a unique ID for the FIT file into serial_number could be the way to do it.
Hello Alan,
thanks a lot for the time you invest into this problem. As I do not personally use Garmin or FIT files, it is a little harder to cover such edge cases.
I've studied best practices for the usage of "serial number" and "number" values. Seems that the "serial number" variable is a good adept for a random number."number" should be set to "1" as Garmin does. So I'll improve it to the next app version. Thanks.
Hello Alan,
thanks a lot for the time you invest into this problem. As I do not personally use Garmin or FIT files, it is a little harder to cover such edge cases.
I've studied best practices for the usage of "serial number" and "number" values. Seems that the "serial number" variable is a good adept for a random number."number" should be set to "1" as Garmin does. So I'll improve it to the next app version. Thanks.
Hi,
In v4.25.1 it looks like the serial_number in the FIT file_id record is now absent - it was previously set to "1701". the "number field is now set to "1".
I believe the intention was to put a random number in the serial_number field , so it looks like there may be some issue with that.
Please would you check if you also see this.
(To test it I exported both a route created before 4.25.1, and one after 4.25.1 , then viewed the FIT file contents using an online FIT file viewer tool, and also the Garmin SDK FITtoCSV tool)
Hi,
In v4.25.1 it looks like the serial_number in the FIT file_id record is now absent - it was previously set to "1701". the "number field is now set to "1".
I believe the intention was to put a random number in the serial_number field , so it looks like there may be some issue with that.
Please would you check if you also see this.
(To test it I exported both a route created before 4.25.1, and one after 4.25.1 , then viewed the FIT file contents using an online FIT file viewer tool, and also the Garmin SDK FITtoCSV tool)
Hi Alan,
sorry, there still was a small glitch. So check it again in the next app version, please. Thanks.
Hi Alan,
sorry, there still was a small glitch. So check it again in the next app version, please. Thanks.
Replies have been locked on this page!