meson: explicitly configurable options

[orlitzky]

As discussed in various places over the past two years, it is useful to be able to override the automagic detection of libraries to explicitly enable or disable a feature. This commit adds meson.options at the top level to do that, and to make the documentation optional as well.

Example:

$ meson setup -Dsirocco=disabled

will disable the cython module sage.libs.sirocco, even if I have sirocco installed. This is nice for a few reasons:

• If you are building from source (or using a source-based distro), this is the only way to achieve accurate dependency information.
• Now that we are adding meson subprojects, there is again a risk of vendoring libraries that you already have installed if (say) its pkg-config check is flaky. Specifying -Dfoo=enabled makes that an error rather than quietly doing the wrong thing.
• It makes it possible to easily test the # needs whatever tags on non-Windows systems (i.e. to test that the things we say are optional are really optional).
• On a system where the corresponding libraries happen to be installed, you can speed up the build by disabling the features you aren't currently working on or using.

And of course, the defaults are all "auto," and should not change anything for anyone else.

I was finally motivated to do this because the doc build fails on my machine, and otherwise I have to edit out the line in src/meson.build and be careful not to commit it to all of my PRs. This implementation is the first thing I came up with after a few minutes of reading the docs, but it works surprisingly well.

[dimpase]

@tobiasdiez - looks good to me.

[tobiasdiez]

I can certainly see that these options can be helpful in certain situations, and the maintenance overhead is quite small. So I'm fine with going forward with this PR.

If you feel like it, you can also add a bit of documentation on how to toggle those features. Should be enough to point to meson introspect --buildoptions for the full list of options (https://mesonbuild.com/Commands.html#introspect).

The CI is failing due to some small formatting issues.

Two small remarks:

If you are building from source (or using a source-based distro), this is the only way to achieve accurate dependency information.

Not sure if I understand this comment correctly, but the dependency info can be found in builddir\meson-info\intro-dependencies.json and in meson-logs.

Now that we are adding meson subprojects, there is again a risk of vendoring libraries that you already have installed if (say) its pkg-config check is flaky. Specifying -Dfoo=enabled makes that an error rather than quietly doing the wrong thing.

In this case, it's better to use --wrap-mode (https://mesonbuild.com/Subprojects.html#commandline-options) to skip all submodules.

[kiwifb]

We had those discussion before in other threads/foras. It would be best if stuff like sirocco was just split like giac is. That part is a slight regression in the meson build compared to the previous state. A growing pain no doubt.
But as a general rule for package maintainers, automagic detection is evil and prevents you from figuring why stuff succeeds on one machine and fails on another. Controlling it has my vote.
In other news I will start to be more active again. Hopefully it will last.

[orlitzky]

If you feel like it, you can also add a bit of documentation on how to toggle those features. Should be enough to point to meson introspect --buildoptions for the full list of options (https://mesonbuild.com/Commands.html#introspect).

Done, but with with meson configure instead since it looks a lot nicer.

If you are building from source (or using a source-based distro), this is the only way to achieve accurate dependency information.

Not sure if I understand this comment correctly, but the dependency info can be found in builddir\meson-info\intro-dependencies.json and in meson-logs.

That will give you the dependency info after the fact, but it may not be what you wanted to achieve. Essentially we want users to be able to say e.g. "build me a sage that doesn't need sirocco," and then we want our distro package to know that sirocco is not a dependency of sage from that point on. With explicit config this is pretty easy to do, we just take the user's enabled/disabled flag for the sirocco feature, and then pass it to the sage build system. This guarantees that sagelib is built with/without sirocco according to the user's wishes, and that we know ahead of time whether or not the result will depend on sirocco, so that we can pull it in as a dependency and record it in the package manager's database iff we need to do that.

Now that we are adding meson subprojects, there is again a risk of vendoring libraries that you already have installed if (say) its pkg-config check is flaky. Specifying -Dfoo=enabled makes that an error rather than quietly doing the wrong thing.

In this case, it's better to use --wrap-mode (https://mesonbuild.com/Subprojects.html#commandline-options) to skip all submodules.

Fair enough, I added that locally.

[orlitzky]

We had those discussion before in other threads/foras. It would be best if stuff like sirocco was just split like giac is. That part is a slight regression in the meson build compared to the previous state.

I personally agree but each of those will require a few days of work to split off cleanly. Until then this PR will prevent things from regressing too badly when the setuptools packages are removed.

[antonio-rojas]

Works fine here, tested no configuration (everything is built) and disabling a couple of modules while still keeping the dependencies installed.

[dimpase]

how about mentioning --wrap-mode=... in the docs?

[orlitzky]

how about mentioning --wrap-mode=... in the docs?

done

[tobiasdiez]

CI is still falling. Could you please run the update meson script locally and commit it?

[orlitzky]

Sure, it's only complaining about whitespace but that should fix it.

[tobiasdiez]

Okay, then let's get this in!

[vincentmacri]

Out of curiosity, does this provide a way to change the compilation flags passed to cythonize/gcc/etc. when building Sage? If not, would it be possible to expand upon this in a future PR to add that?

[orlitzky]

Out of curiosity, does this provide a way to change the compilation flags passed to cythonize/gcc/etc. when building Sage? If not, would it be possible to expand upon this in a future PR to add that?

This should already be possible. Meson supports the standard environment variables like CFLAGS, and has invented(?) others like CYTHONFLAGS:

• https://mesonbuild.com/Reference-tables.html#compiler-and-linker-flag-environment-variables

I don't use CYTHONFLAGS, but can confirm that my CFLAGS and LDFLAGS are used during the build.

[github-actions]

Documentation preview for this PR (built with commit 3811277; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[orlitzky]

It looks like eclib is optional now, so I'll add a flag for that too.

[tobiasdiez]

That was actually not a very conscious change on my side, but it's probably a good idea to have it optional indeed. Could you please move eclib also to the optional dependencies in 'pyproject.toml' and perhaps check that there are disable flags for exactly all optional deps listed there. (I can also do this after my conference...)

[orlitzky]

• Moved eclib and brial to optional-host-requires in pyproject.toml
• Bumped the version bound on eclib (I'm almost certain this is required anyway) so that we can rely on pkg-config
• Added the explicit option for eclib

[orlitzky]

(The list of feature flags does now match the list of optional deps.)

[antonio-rojas]

The meataxe description doesn't seem accurate, Sage doesn't use any of the meataxe programs, but rather its multiplication tables (via the libmtx library) to optimize matrix operations over finite fields

[orlitzky]

The meataxe description doesn't seem accurate, Sage doesn't use any of the meataxe programs, but rather its multiplication tables (via the libmtx library) to optimize matrix operations over finite fields

Better now?

I tried to word them all in a way that avoids "description rot" if the implementation details change, but I don't know meataxe very well and wound up on a webpage that wasn't much help the first time around.

[dimpase]

is meataxe really used anywhere apart from defunct p-group cohomology package?

[antonio-rojas]

is meataxe really used anywhere apart from defunct p-group cohomology package?

yes, https://github.com/sagemath/sage/blob/develop/src/sage/matrix/matrix_gfpn_dense.pyx

[dimpase]

Yes, that's a Cython module split out from the now defunct package by Simon King, when he worked on refactoring his package, something that hasn't been completed.

[antonio-rojas]

Yes, that's a Cython module split out from the now defunct package by Simon King, when he worked on refactoring his package, something that hasn't been completed.

which is used by default if present as parent for matrices over fields of small order

[dimpase]

are there meson options allowing to control tools discovery things, e.g. getting pybind11 and cython from a prescribed location, rather than getting whatever found in /usr/bin etc?

[orlitzky]

are there meson options allowing to control tools discovery things, e.g. getting pybind11 and cython from a prescribed location, rather than getting whatever found in /usr/bin etc?

For cython, PATH should work. When building extension modules, meson will use whatever python knows about, so you could probably prepend to PYTHONPATH for something like pybind11.