Migrate to using de440.bsp for consistency?

[akoumjian]

Would ASSIST be open to using de440.bsp instead of linux_p1550p2650.440 ?

I ask because Asteroid Institute has a package for DE440 (naif-de440) and is also in the process of publishing sb441-n16.bsp to a package jpl-small-bodies-de441-n16. This package is still awaiting the file size limit increase.

If ASSIST moved to using the .bsp format for both files, it would mean users could install all the ephemeris files as simple Python packages, rather than having to download them separately.

If you're open to it, I'd be happy to put in a PR with tests.

[hannorein]

Maybe I'm not up-to-date with all the python packaging stuff, but I'd be cautious to install large data files as python packages. (You already mention that you're waiting for a file size limit to be increased.)

Another reason: I currently have different directories where I use slightly different versions of ASSIST for different projects. Each project is using its own virtual environment. But they all use the same shared data files. I wouldn't want to have multiple copies of the datafiles.

[akoumjian]

In this case, the data packages provide an option for those who want to be able to setup their environments (particularly in automated build scenarios) that don't require running custom download scripts or extra bash commands. It has the added benefit of being reproducible and hash verified.

For what it's worth, pip by default uses a global cache (even across virtual environments) that avoids having to re-download packages. Although you will end up installing multiple copies to each virtualenv you install it to.

In the case of ASSIST, using these data packages would look like this:

import assist
from naif_de440 import de440
from jpl_small_bodies_de441_n16 import de441_n16

ax = assist.Ephem(planets_path=de440, asteroids_path=de441_n16)

Where de440 and de441_n16 are simply paths. So no material change to how ASSIST is pointed to files and users can re-use the same files in many different environments if they choose.

[akoumjian]

To clarify, the only real ask here is whether ASSIST is open to using the .bsp format of de440, since it's already using that format for the asteroids. The data packages component is simply explaining our personal interest in the change.

[hannorein]

Thanks. Got it. I'll keep downloading the files manually then. 😉

I don't know enough about bsp vs de440. Is the bsp one considered the "new" one?

[akoumjian]

I believe the difference is that the SPICE .bsp files provide Chevyshev polynomial coefficients over separate periods of time, which can then be used to interpolate a precise position and velocity. I think linux_p1550p2650.440 contains a binary format of pre-computed positions and velocities at some regular interval and I believe ASSIST is likely doing a simpler polynomial or linear interpolation over those.

[matthewholman]

No, they are simply different formats of the same content. ASSIST is reads blocks of Chebyshev coefficients and uses those to obtain precise interpolated positions and velocities of the Sun, planets, moon, and other perturbers.

There are a few more details here:
https://ssd.jpl.nasa.gov/planets/eph_export.html

@cylon359, could you comment on whether it would be straightforward to use the .bsp files rather than the linux versions?

[hannorein]

@matthewholman, that was my understanding too.

One concern I have is that the SPICE bsp format, as far as I understand it, is very flexible. We can't possible support all of its options without an external library (which I would strongly advise against). So at best, ASSIST could read a very specific sub-set of compatible bsp files. I don't know enough to say whether this would be worth it or not.

[akoumjian]

Is assist already reading the spice files for the asteroids? Or is there other data being pulled from the planets file that will make it more complicated?

[hannorein]

I think ASSIST can only read a very specific bsp file for asteroids.

ASSIST also reads other constants from the planets file (masses, J2, C, etc).

[cylon359]

@matthewholman

We probably could use the BSP file instead of the other file, but the constants would not be available (asteroid masses for example?). I may be forgetting if there were any other differences.

As to SPK being very flexible, if an end user were to provide a sample file of what they want to use, we should be able to modify the routines to be generic enough to handle it.

[hannorein]

@cylon359 Looking through this presentation I feel a bit overwhelmed by the complexity!

[akoumjian]

I've been working on this a bit. The constants are indeed present in the BSP, but they are in a free form comment record. I've got a version working that is mostly successful at reading those constants in. I do wonder on the utility of reading the constants in from either file, however, since they appear quite specific to those files and will not until a whole new DE is created. One could actually hard code these values and not lose any flexibility.

[hannorein]

@akoumjian We actually had the same discussion when we first wrote ASSIST. I see a strong argument for reading them from the file rather than hard coding them. Whenever a new DE comes out, we want to support it, but we also want to keep supporting the older one so people can choose when to update their pipelines. If we hard-code the constants, that would mean hard coding multiple versions of each constant. We will probably have to change the code to read in the new DE, but I prefer having multiple "read file" functions over multiple hard coded constants. FWIW, this also works in favour of you providing a new "read file" function to read the BSP format. That function can then deal with the constants in whatever way is best (I'd read them in, but up to you!).

[cylon359]

I will take a stab at seeing how involved it is to read and use the constants from the BSP files, but this will have to wait until bright time.

[akoumjian]

I am close to offering a pull request on this soon for discussion, in case @cylon359 or anyone else has been working on it.