FIT Activity - no data for 1 second

Username		Password		Remember Me?	forgot password?

06 May 2015 10:20 AM

MaximumTrainer Sr. Member

Total Posts: 123

Joined 2013-10-07

PM

I record an indoor cycling ride with 1sec data point precision.

At each second, I will receive up to 4 metric per second (Power, Cadence, Speed, HR)
At each second, I don't know what point I could receive, it could be only power, or a mix of all the above metrics.

What happen is that if I receive data for power for a second and nothing for the next, FIT seems to record "0" for that data even if I didn't record a "0", no data doesn't mean 0 in my case. so what I would like is that it doesn't put a 0 and make a continuous line from the previously received data point to the next.

Here is a file example (have a look at power and speed) :
https://connect.garmin.com/activity/765929748

A better example (what I would like) with less zero (data point were received for each 1sec so it doesn't cause problem here)
https://connect.garmin.com/activity/762160232

Code for reference :
This function only get called when there is at least one data point (Power or Cadence or Speed or HR is > 0)

void FitActivityCreator::writeRecord(int timeNow,
                                     double averageHr1sec, double averageCadence1sec, double averageSpeed1sec, double averagePower1sec) {

    if (!fileIsOpen)
        return;

    fit::RecordMesg recordMesg;
    recordMesg.SetTimestamp(secStartedWorkout+timeNow);


    if (averageHr1sec != -1) {
        recordMesg.SetHeartRate((int)(averageHr1sec + 0.5));
    }

    if (averageCadence1sec != -1) {
        recordMesg.SetCadence((int)(averageCadence1sec + 0.5));
    }

    if (averageSpeed1sec != -1) {

        qDebug() << "##Writing speed:" << averageSpeed1sec;

        double speedms = averageSpeed1sec*convertKphToMs;
        recordMesg.SetSpeed(speedms);

        // Compute Accumulated Distance
        if (averageSpeed1sec > 0) {
            int timeSec = timeNow - timeSecLastWriteRecord;
            double timeHrs = timeSec/3600.0;
            double distanceM = averageSpeed1sec * timeHrs *1000;
            accumulatedDistance += distanceM;
            recordMesg.SetDistance(accumulatedDistance);
        }
        else {
            recordMesg.SetDistance(accumulatedDistance);
        }
        timeSecLastWriteRecord = timeNow;
    }

    if (averagePower1sec != -1) {

        qDebug() << "##Writing power:" << averagePower1sec;
        recordMesg.SetPower((int)(averagePower1sec + 0.5));
    }

    //% right pedal - TO TEST
    //    recordMesg.SetLeftRightBalance(0.60);

    encode.Write(recordMesg);
} 

Here is the FIT analysed that shows the "0"
https://www.dropbox.com/s/rvirsn9ubafaae8/AnalyseGC.png?dl=0

Looking for ways to fix this, I don't think putting the last known received value if I get a "0" would be a good solution, since it will change the data. Thanks!

Signature

——————————————————
Free Indoor Cycling Software - https://maximumtrainer.com

08 May 2015 03:10 PM #1

edge-python Jr. Member

Total Posts: 44

Joined 2013-04-07

PM

Unfortunately, you have not attached the activity-fit-file.

The question is: why some values are 0 ?

e.g. Cadence: because it was not pedaling or because the signal of the cadence-sensor has been lost?

case A:
because it was not pedaling
-> the value in the activity-fit-file is 0

case B:
because the signal was lost from the cadence-sensor
-> you have to change the data-message
-> you need an new message-definition

I have an example in the appendix.
It is an excerpt of a decoded activity-fit file.

You can see the different cases:
-> local message 5: all values
-> e.g. 2012-08-29-17-50-04 it was not pedaling -> local message 5 with cadence value = 0
-> local message 6: GPS (pos lat/long) is lost
-> local message 7: cadence is lost

maybe help my instructions to you, but perhaps the problem also has a different cause

Thank you.
edge-python
(sorry, english is not my native language)

File Attachments

activity-fit_decoded.txt (File Size: 30KB - Downloads: 1069)

08 May 2015 04:36 PM #2

MaximumTrainer Sr. Member

Total Posts: 123

Joined 2013-10-07

PM

The reason is quite simple, I write record every 1sec (software write record in real-time), I write data for 4 metrics every 1sec, but sometimes some metric have not been received for 1 second. it doesn't mean the signal has been lost or the user stopped pedaling, just that no data has been received for that time-span.

For example, if no HR data is present for 1 second, the HR doesn't drop to 0, I would like the same behavior with Cadence, Power, Speed. If I want a 0, I would write a record with a 0 value.

Thanks for your example, will try to find a good solution because right now the graph shows a lot of 0. the 0 are not written by the software, it it caused by not writing Power data for 1sec, the FIT file seems to assume that it dropped to 0, that is not the case.

Signature

——————————————————
Free Indoor Cycling Software - https://maximumtrainer.com

08 May 2015 05:29 PM #3

edge-python Jr. Member

Total Posts: 44

Joined 2013-04-07

PM

an addition to my post
see the appendix

Thank you.
edge-python
(sorry, english is not my native language)

File Attachments

example_v2.pdf (File Size: 230KB - Downloads: 679)

09 May 2015 02:24 AM #4

old_man_biking Sr. Member

Total Posts: 296

Joined 0

PM

So what happens, when you don't set any of the record message properties besides date/time and e.g. distance?
Will it be encoded as a record message with just date/time and distance fields? Or will it have additional fields that are set to 0 or invalid (0xFFFF)?
I'd assume that it should have just date/time and distance - why should any unset fields be encoded?
If not, you could try a workaround and set the properties to float.NaN. I didn't have a look at the encoding methods, but I'd assume that this should set the fields to invalid.

Cheers,
OMB

13 May 2015 07:56 AM #5

MaximumTrainer Sr. Member

Total Posts: 123

Joined 2013-10-07

PM

Thanks all for your help resolving this.

Here is a concrete example of a 5sec data span that is causing problem:

1sec update Data DataWorkout HR: 90.5 CAD: 69 Speed: 14.5 Power: 43
##Writing speed: 14.5
##Writing power: 43
2sec update Data DataWorkout HR: 92 CAD: 78 Speed: 17 Power: 56
##Writing speed: 17
##Writing power: 56
3sec update Data DataWorkout HR: 93 CAD: 84 Speed: -1 Power: -1
4sec update Data DataWorkout HR: 94.5 CAD: 86 Speed: -1 Power: -1
5sec update Data DataWorkout HR: 95 CAD: 87 Speed: 19 Power: 104
##Writing speed: 19
##Writing power: 104

In this example, I receive Power data and speed data at 0:01 and 0:02, but at 0:03 and 0:04, I just receive HR data and no speed and power, but I still want to write the HR data in my file. By writing the HR data and not writing Power and Speed at 0:03, this will cause "0" to be written in the FIT file for Power and Speed.

old_man_biking: I don't write a record if no metric is received for a specific second, I need at least one metric to write a Record (HR || Power || Speed || Cadence)

Here is the FIT file created by the app
Here on garmin connect
Here is the workout done in the app (no 0 shown on the curves here) :

This workout is done on a KICKR, a lot of 0 seems to appear on the KICKR since it's receiving ANT Message every second to set the appropriate load on the trainer and has less time to broadcast metrics (speed,power).

Signature

——————————————————
Free Indoor Cycling Software - https://maximumtrainer.com

13 May 2015 08:18 AM #6

old_man_biking Sr. Member

Total Posts: 296

Joined 0

PM

I still can't believe that the encoding methods create fields (with 0 value) for values that you haven't explicitely set. I'd assume that unset values just will be missing from the message.
So in my previous post I had suggested that you just TRY OUT, what happens if and when you create record messages with a minimal set of explicitely set values (date/time and e.g. distance).

Besides, there's my suggestion to set undefined values fo float.NaN and see what happens.

Cheers,
OMB

13 May 2015 08:36 AM #7

MaximumTrainer Sr. Member

Total Posts: 123

Joined 2013-10-07

PM

I checked with decode.exe and it's not setting "0" values but some other values (see in bold below) when I set nothing on it, Thanks for your suggestion will try it now :

So the Fit file has some default values that are not "0", this is okay I guess. The problem is maybe with the software that decodes the Fit file and shows "0" for theses values.

OnMesg: Received Mesg with global ID#20, its name is Record
Field0 Index0 ("Timestamp" Field#253) Value: 800458944 (raw value 800458944)
Field1 Index0 ("HeartRate" Field#3) Value: 91 (raw value 91)
Field2 Index0 ("Cadence" Field#4) Value: 69 (raw value 69)
Field3 Index0 ("Speed" Field#6) Value: 4.028 (raw value 4028)
Field4 Index0 ("Distance" Field#5) Value: 4.03 (raw value 403)
Field5 Index0 ("Power" Field#7) Value: 43 (raw value 43)
Field6 Index0 ("EnhancedSpeed" Field#73) Value: 4.028 (raw value 4028)
OnMesg: Received Mesg with global ID#20, its name is Record
Field0 Index0 ("Timestamp" Field#253) Value: 800458945 (raw value 800458945)
Field1 Index0 ("HeartRate" Field#3) Value: 92 (raw value 92)
Field2 Index0 ("Cadence" Field#4) Value: 78 (raw value 78)
Field3 Index0 ("Speed" Field#6) Value: 4.722 (raw value 4722)
Field4 Index0 ("Distance" Field#5) Value: 8.75 (raw value 875)
Field5 Index0 ("Power" Field#7) Value: 56 (raw value 56)
Field6 Index0 ("EnhancedSpeed" Field#73) Value: 4.722 (raw value 4722)
OnMesg: Received Mesg with global ID#20, its name is Record
Field0 Index0 ("Timestamp" Field#253) Value: 800458946 (raw value 800458946)
Field1 Index0 ("HeartRate" Field#3) Value: 93 (raw value 93)
Field2 Index0 ("Cadence" Field#4) Value: 84 (raw value 84)
Field3 Index0 ("Speed" Field#6) Value: 65.535 (raw value 65535)
Field4 Index0 ("Distance" Field#5) Value: 4.294967E+07 (raw value 4294967295)
Field5 Index0 ("Power" Field#7) Value: 65535 (raw value 65535)
Field6 Index0 ("EnhancedSpeed" Field#73) Value: 65.535 (raw value 65535)
OnMesg: Received Mesg with global ID#20, its name is Record
Field0 Index0 ("Timestamp" Field#253) Value: 800458947 (raw value 800458947)
Field1 Index0 ("HeartRate" Field#3) Value: 95 (raw value 95)
Field2 Index0 ("Cadence" Field#4) Value: 86 (raw value 86)
Field3 Index0 ("Speed" Field#6) Value: 65.535 (raw value 65535)
Field4 Index0 ("Distance" Field#5) Value: 4.294967E+07 (raw value 4294967295)
Field5 Index0 ("Power" Field#7) Value: 65535 (raw value 65535)
Field6 Index0 ("EnhancedSpeed" Field#73) Value: 65.535 (raw value 65535)

Signature

——————————————————
Free Indoor Cycling Software - https://maximumtrainer.com

13 May 2015 08:45 AM #8

MaximumTrainer Sr. Member

Total Posts: 123

Joined 2013-10-07

PM

deleted. [wanted to edit my previous post..]

Signature

——————————————————
Free Indoor Cycling Software - https://maximumtrainer.com

13 May 2015 09:11 AM #9

old_man_biking Sr. Member

Total Posts: 296

Joined 0

PM

I just decoded the .FIT file that you've attached a few posts above.
Bottom line: I can't see any record messages with cadence=0, however some without cadence.

Here's part of the decoding of one record message - you see that it has just date/time and heart rate.

Data message: Record
  DateTime:                   13.05.2015 13:44:13
  Distance:                   -
  Position:                   -
  Elevation:                  -
  Speed:                      -
  Heart rate:                 131 bpm
  Cadence:                    -
  Power:                      -
  Temperature:                - 

So your .FIT files contain what you want them to contain. Obviously your way of looking at them has skewed things.

Cheers,
OMB

13 May 2015 09:17 AM #10

MaximumTrainer Sr. Member

Total Posts: 123

Joined 2013-10-07

PM

The problem in my file was with Power and Speed, not cadence.
You are right, it doesn't write 0 values but another default value (see above post, values in bold)

And this default value is interpreted by a "0" by all the decoding software/website out there, that is why it has "skewed" my view like you said.
Just a note, if you don't want to help or want to be rude, just don't reply, I can find help elsewhere, thank you

Signature

——————————————————
Free Indoor Cycling Software - https://maximumtrainer.com

13 May 2015 09:47 AM #11

old_man_biking Sr. Member

Total Posts: 296

Joined 0

PM

blaisminator - 13 May 2015 09:17 AM
... if you don't want to help or want to be rude ...

Hello blaisminator,
sorry, but I don't have the slightest idea why you accuse me of being not helpful or even rude.
All I wanted to say is: your file is perfect, it's just your tools that you use for looking at them, which makes the file look wrong.

I didn't have any intention of being rude. And as you can see I've invested my time to try and help you.
What makes you call me "rude"?

Btw, I didn't find any zero values of speed and power in you .FIT file either.

Cheers,
OMB

13 May 2015 11:38 AM #12

MaximumTrainer Sr. Member

Total Posts: 123

Joined 2013-10-07

PM

Sorry english is not my native language, probably misunderstood the meaning of "skewed"..

The file is perfect and the problem is more with the tools that decode the file.
The tools shows "0" when there is no data. Is there really no data? Power shows ("Power" Field#7) Value: 65535 (raw value 65535) in my example above when for that second I didn't write a value.

I'll try to look for workaround because as you can see the tools produce graph that becomes meaningless with all theses 0.
Thanks for investing your time on this, really appreciated.
Max

Signature

——————————————————
Free Indoor Cycling Software - https://maximumtrainer.com

14 May 2015 07:07 AM #13

edge-python Jr. Member

Total Posts: 44

Joined 2013-04-07

PM

@ blaisminator

your dropbox link to "exampleWorkoutFit.png works" for me
but your dropbox link to "2015-05-1309-42-23" works not for me

(this forum has the option to attach a file directly at your post ...)

my post #1

The decoded file example is a fit file from a "Garmin edge 500" (and not my own creation).
I think it is better to use different message definitions for different data records.
I think the use of invalid values is not a good idea.
(a signal lost or a signal not in time is the same problem)
It is not trivial to write a fit file (especially if the data is not linear), it is easier to decode a fit file.

Thank you.
edge-python
(sorry, english is not my native language)

14 May 2015 08:02 AM #14

MaximumTrainer Sr. Member

Total Posts: 123

Joined 2013-10-07

PM

@edge-python

I updated the link in my previous post, should be good now

So I could create different message definition for messages that contains different data inside.
I have 4 metrics and mixing all the possibilities of incoming metric gives me 15 possibilities (HR, CAD, SPD, PW)
Right now, I just create the recordMesg this way:

fit::RecordMesg recordMesg;
    recordMesg.SetTimestamp(secStartedWorkout+timeNow);
    if (averageHr1sec != -1) 
        recordMesg.SetHeartRate((int)(averageHr1sec + 0.5));
    if (averageCadence1sec != -1) 
        recordMesg.SetCadence((int)(averageCadence1sec + 0.5));
    ... 

I know I could input the last known received value for a metric if I don't receive it for a second to workaround this problem, but this would change the data. I would prefer to have a continuous line from the previous received point to the next, like you posted in your example (example_v2.pdf)

Thank you!

Signature

——————————————————
Free Indoor Cycling Software - https://maximumtrainer.com

ANT Forum