build/pkgs/sagelib: Invoke install cleaner explicitly, remove it from pkgs/sagemath-standard/setup.py

[mkoeppe]

Unless ./configure --enable-editable is used, the Sage library is installed directly into site-packages using setup.py install, and at the end of sage_setup's custom sage_install_and_clean invokes the installation cleaner, which removes "stale" installed files.

The installation cleaner is problematic when we start to install optional modules in the sage.* namespace via separate distributions such as sagemath-tdlib (#29864).

• pkgs/sagemath-standard/setup.py will no longer enable optional Cython modules based on which optional packages are installed
• but the installation cleaner needs to know about these optional Cython modules so it does not remove them right after their installation.

In this ticket we solve this problem by removing the use of the installation cleaner from pkgs/sagemath-standard/setup.py. Instead we invoke it separately at the end of the installation script build/pkgs/sagelib/spkg-install.

(An alternative solution, replacing this installation procedure by a standard installation procedure using wheels, as proposed in #32874, would make sage -b much slower. Also, the idea of just making ./configure --enable-editable the default, #32406, has met skepticism.)

Depends on #32874

CC: @kiwifb @jhpalmieri

Component: build

Work Issues: Do not build_ext

Branch/Commit: u/mkoeppe/pkgs_sagemath_standard_setup_py__move_distributions_logic_to_a_new_package_sagemath_optional @ edd0ea1

Reviewer: Kwankyu Lee

Issue created by migration from https://trac.sagemath.org/ticket/32927

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,3 +1,3 @@
 ... which generates `install-requires` on `setup.py egg_info` (or `prepare_metadata_for_build_wheel`/`prepare_metadata_for_build_editable`).
 
-
+As an alternative, we could also have `configure` generate a `requirements.txt` with @ links into `SAGE_ROOT/pkgs/`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,3 +1,7 @@
 ... which generates `install-requires` on `setup.py egg_info` (or `prepare_metadata_for_build_wheel`/`prepare_metadata_for_build_editable`).
 
-As an alternative, we could also have `configure` generate a `requirements.txt` with @ links into `SAGE_ROOT/pkgs/`
+(As an alternative, we could also have `configure` generate a `requirements.txt` with @ links into `SAGE_ROOT/pkgs/`.)
+
+We also move the installation cleaner from `pkgs/sagemath-standard/setup.py` to this package.
+
+

[mkoeppe]

Dependencies: #31386, #32899

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,7 +1,2 @@
-... which generates `install-requires` on `setup.py egg_info` (or `prepare_metadata_for_build_wheel`/`prepare_metadata_for_build_editable`).
+We move the installation cleaner from `pkgs/sagemath-standard/setup.py` to a new location. It is only to be used by sage-the-distribution.
 
-(As an alternative, we could also have `configure` generate a `requirements.txt` with @ links into `SAGE_ROOT/pkgs/`.)
-
-We also move the installation cleaner from `pkgs/sagemath-standard/setup.py` to this package.
-
-

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,2 +1,5 @@
 We move the installation cleaner from `pkgs/sagemath-standard/setup.py` to a new location. It is only to be used by sage-the-distribution.
 
+This is also a preparation for namespace packages. 
+The installation cleaner should only be run once, when all namespace packages filling the namespace `sage.*` are installed.
+

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,5 +1,11 @@
-We move the installation cleaner from `pkgs/sagemath-standard/setup.py` to a new location. It is only to be used by sage-the-distribution.
+Unless `./configure --enable-editable` is used, the Sage library is installed directly into `site-packages` using `setup.py install`, and at the end of `sage_setup`'s custom `sage_install_and_clean` invokes the installation cleaner, which removes "stale" installed files.
 
-This is also a preparation for namespace packages. 
-The installation cleaner should only be run once, when all namespace packages filling the namespace `sage.*` are installed.
+The installation cleaner is problematic when we start to install optional modules in the `sage.*` namespace via separate distributions such as **sagemath-tdlib**. 
+- `pkgs/sagemath-standard/setup.py` will no longer enable optional Cython modules based on which optional packages are installed
+- but the installation cleaner needs to know about these optional Cython modules so it does not remove them right after their installation.
 
+In this ticket we solve this problem by removing the use of the installation cleaner from `pkgs/sagemath-standard/setup.py`. Instead we invoke it separately at the end of the installation script `build/pkgs/sagelib/spkg-install`.
+
+
+(An alternative solution, replacing this installation procedure by a standard installation procedure using wheels, as proposed in #32874, would make `sage -b` much slower.)
+

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Description changed:

--- 
+++ 
@@ -7,5 +7,5 @@
 In this ticket we solve this problem by removing the use of the installation cleaner from `pkgs/sagemath-standard/setup.py`. Instead we invoke it separately at the end of the installation script `build/pkgs/sagelib/spkg-install`.
 
 
-(An alternative solution, replacing this installation procedure by a standard installation procedure using wheels, as proposed in #32874, would make `sage -b` much slower.)
+(An alternative solution, replacing this installation procedure by a standard installation procedure using wheels, as proposed in #32874, would make `sage -b` much slower. Also, the idea of just making `./configure --enable-editable` the default, #32406, has met skepticism.)
 

[mkoeppe]

Branch: u/mkoeppe/pkgs_sagemath_standard_setup_py__move_distributions_logic_to_a_new_package_sagemath_optional

[mkoeppe]

Commit: 00d5f59

[mkoeppe]

Last 10 new commits:

77c957f	src/doc/en/developer/packaging_sage_library.rst: Put hierarchy section one level higher
9375ef8	pkgs/sagemath-standard/tox.ini: Use SAGE_VENV or venv symlink to find wheels
c6fc111	Prettier diagram
7d6a44c	Use :mod: as markup for packages/modules
f83d424	Improve ABC example
817a8d0	src/doc/en/developer/packaging_sage_library.rst: More :mod: and :class: markup
b555735	src/doc/en/developer/packaging_sage_library.rst: Link to pypi.org and to documentation of packaging metadata
81e9c9a	Merge #32899
ecc0886	Merge #31386
00d5f59	build/pkgs/sagelib: Invoke install cleaner explicitly, remove it from pkgs/sagemath-standard/setup.py

[mkoeppe]

comment:9

Stalled in needs_review or needs_info; likely won't make it into Sage 9.5.

[sagetrac-git]

Changed commit from 00d5f59 to 2d08b69

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

2d08b69

Merge tag '9.5.rc3' into t/32927/pkgs_sagemath_standard_setup_py__move_distributions_logic_to_a_new_package_sagemath_optional

[mkoeppe]

Changed dependencies from #31386, #32899 to none