Just throwing this out there in case it helps anyone. A Python library for parsing FIT files and some utilities built on top of that. These are currently part of a training diary called Choochoo, but the FIT library is conceptually separate - if necessary I could pull it out into a separate package (but I'm not going to bother right now unless someone really needs it that way).
There's little documentation of the Python API, but the tools documentation will give you the flavour of what is possible:
https://andrewcooke.github.io/choochoo/fit-cookbook - a bunch of examples
https://andrewcooke.github.io/choochoo/fit-files - the tool for display FIT files
https://andrewcooke.github.io/choochoo/fix-fit - the tool for fixing FIT files
Some more technical background: The library is only for reading, not writing (but from the examples above with "fixing" files it can help with a minimal amount of manipulation). Although it's written in Python, and so never going to be super-efficient, the library is designed to parse the file lazily. So breaking a FIT file down into records is quite fast (since each record does not need to be parsed completely). The parsing data are extracted from Profile.xslx (the spreadsheet) directly and then stored as pickled Python code (so there is no actual "source file" describing the different messages). The parsing is incomplete - I don't handle optional developer fields right now (anyone have any test data?) - but does include accumulating fields.
Finally, the github repo -
https://github.com/andrewcooke/choochoo