Hi Paul,
The ANT protocol has a frequency tolerance requirement of ±50ppm. So for ANT applications on the nRF51 you are limited to the 20, 30 and 50 ppm accuracy options (Refer to the nRF51422 Product Specification Document). I would assume the time difference between those accuracy options is minimal, do you have very specific timing/battery requirements for your design?
For reference, the N5 ANT SoC Module from Dynastream Innovations uses a 50ppm 32k crystal.