sagemathgh-39161: Add hyperelliptic curves using the smooth model
    
## Overview

This PR includes a whole new module into schemes which implements
hyperelliptic curves over the smooth model, and focuses on implementing
decent arithmetic for Jac(H) for all reasonable cases, something which
is not done properly in the current implementation due to limitations of
the curve model used.

The idea is that `hyperelliptic_curves_sm` will be available with the
old model for some time, until someone decides we should depreciate the
old model in favour of this new implementation.

## Motivation

The current implementation of hyperelliptic curves in SageMath uses the
projective plane model. Although this works nicely enough for imaginary
curves with only one point at infinity, it is not descriptive enough for
the real models. My gut feeling is the projective plane model was used
as in early Sage days this was easier to do and allowed hyperelliptic
curves to be supported at all. Mathematically, I believe it makes much
more sense to use a weighted projective model to have this new
description but it requires more tools which we only have thanks to the
work of others in sage since it was released.

This PR is a total rewrite of the hyperelliptic curve classes to instead
use the smooth model for hyperelliptic curves which facilitates
implementing arithmetic of Jacobians of hyperelliptic curves for
(almost) all cases. In particular, now if one tries to perform
arithmetic on Jac(C) one either gets the correct value or a well-handled
error instead of just returning something which is wrong.

The hope is that with this new model for the curves, more recent
research in the area of hyperelliptic curves and their jacobians can be
more easily integrated. As a personal example, being able to compute
arithmetic in Jac(H) for the real model unconditionally would help with
the implementation of genus two isogenies which are "in vogue" right now
in the cryptography world.

### Content of PR

This PR looks WAY bigger than it is, as it duplicates ALL code from
`hyperelliptic_curves` and then has modifications to allow this to work
in the new curve model. I believe the best thing to do in the long run
is to depreciate the old hyperelliptic curve impl and have this replace
it, but this will take years(?) so I think we'll just have to have both
side by side for a while.

This PR still needs a lot of work including:

- more tests
- better comments and docstrings
- potentially refactoring of which files belong where

but the code has sat at this stage for a long time without much more
progress from the three of us, so I think the best thing to do is open
up the PR and try and get some extra attention on getting this code
ready to be included.

One benefit is that all this code is "new" in that it should not
conflict with anyone else's work, so we can get this code into a good
position and keep adding to it with more interesting maths once the base
layer is in place

### Example of better arithmetic

For example, this code fixed the following issue:

sagemath#32024

```py
sage: R.<x> = QQ[]
sage: f = 144*x^6 - 240*x^5 + 148*x^4 + 16*x^3 - 16*x^2 - 4*x + 1
sage: H = HyperellipticCurveSmoothModel(f)
sage: J = Jacobian(H)
sage: P = J(H(0,1))-J(H(0,-1))
sage: (5*P).is_zero()
False
sage:
sage: H = HyperellipticCurve(f)
sage: J = Jacobian(H)
sage: P = J(H(0,1))-J(H(0,-1))
sage: (5*P).is_zero()
True
```

and is also related to the issues
sagemath#37109
sagemath#37093
sagemath#37101
sagemath#37626

### Help!

This code needs more work, and lots of time spent on the review which I
don't think is a reasonable thing to do for one person. So if this area
interests you any help would be appreciated (either in the review of
some of the code or some extra commits which tidy up aspects)
    
URL: sagemath#39161
Reported by: Giacomo Pope
Reviewer(s): Frédéric Chapoton, Giacomo Pope, grhkm21, sabrinakunzweiler, Vincent Macri

Author: Release Manager

src/doc/en/reference/arithmetic_curves/index.rst

@@ -110,22 +110,33 @@ Hyperelliptic curves
    sage/schemes/hyperelliptic_curves/hyperelliptic_finite_field
    sage/schemes/hyperelliptic_curves/hyperelliptic_padic_field
    sage/schemes/hyperelliptic_curves/hyperelliptic_rational_field
+   sage/schemes/hyperelliptic_curves/hyperelliptic_g2
 
+   sage/schemes/hyperelliptic_curves/invariants
    sage/schemes/hyperelliptic_curves/mestre
-
    sage/schemes/hyperelliptic_curves/monsky_washnitzer
    sage/schemes/hyperelliptic_curves/hypellfrob
 
+   sage/interfaces/genus2reduction
+
+Jacobians of hyperelliptic curves
+------------------------------------------------
+
+.. toctree::
+   :maxdepth: 1
+
    sage/schemes/hyperelliptic_curves/jacobian_generic
-   sage/schemes/hyperelliptic_curves/jacobian_g2
-   sage/schemes/hyperelliptic_curves/jacobian_homset
-   sage/schemes/hyperelliptic_curves/jacobian_morphism
-   sage/schemes/hyperelliptic_curves/jacobian_endomorphism_utils
+   sage/schemes/hyperelliptic_curves/jacobian_homset_generic
+   sage/schemes/hyperelliptic_curves/jacobian_homset_ramified
+   sage/schemes/hyperelliptic_curves/jacobian_homset_split
+   sage/schemes/hyperelliptic_curves/jacobian_homset_inert
 
-   sage/schemes/hyperelliptic_curves/hyperelliptic_g2
-   sage/schemes/hyperelliptic_curves/invariants
-   sage/schemes/hyperelliptic_curves/kummer_surface
+   sage/schemes/hyperelliptic_curves/jacobian_g2_generic
+   sage/schemes/hyperelliptic_curves/jacobian_g2_homset_ramified
+   sage/schemes/hyperelliptic_curves/jacobian_g2_homset_split
+   sage/schemes/hyperelliptic_curves/jacobian_g2_homset_inert
+
+   sage/schemes/hyperelliptic_curves/jacobian_morphism
 
-   sage/interfaces/genus2reduction
 
 .. include:: ../footer.txt

src/doc/en/reference/references/index.rst

@@ -2984,6 +2984,10 @@ REFERENCES:
                50(2):259-273, 1995.
                :doi:`10.1006/jcss.1995.1022`
 
+.. [Gal2018] Steven D Galbraith. *Mathematics of Public Key Cryptography*.
+             Version 2.0, October 31, 2018.
+             https://www.math.auckland.ac.nz/~sgal018/crypto-book/main.pdf
+
 .. [Gallai] \T. Gallai, *Elementare Relationen bezueglich der
             Glieder und trennenden Punkte von Graphen*, Magyar
             Tud. Akad. Mat. Kutato Int. Kozl. 9 (1964) 235-236
@@ -3071,6 +3075,10 @@ REFERENCES:
 .. [GHJV1994] \E. Gamma, R. Helm, R. Johnson, J. Vlissides, *Design
               Patterns: Elements of Reusable Object-Oriented
               Software*. Addison-Wesley (1994). ISBN 0-201-63361-2.
+              
+.. [GHM2008] \S. Galbraith, M. Harrison, D. Mireles Morales,
+             *Efficient hyperelliptic arithmetic using balanced representation for divisors*,
+             Algorithmic Number Theory: 8th International Symposium, ANTS-VIII Banff, Canada, May 17-22, 2008 Proceedings 8.
 
 .. [Gil1959] Edgar Nelson Gilbert. *Random Graphs*, Annals of Mathematical
              Statistics. 30 (4): 1141-1144, 1959.
@@ -5245,6 +5253,10 @@ REFERENCES:
              LexDFS, LexUP and LexDown orderings*. (2017)
              :arxiv:`1701.00305`
 
+.. [Mireles2008] David J. Mireles Morales, *Efficient Arithmetic on Hyperelliptic Curves With Real Representation*,
+                 PhD thesis, University of London, UK, 2008.
+                 https://www.math.auckland.ac.nz/~sgal018/Dave-Mireles-Full.pdf
+
 .. [MirMor2009] \R. Miranda, D.R. Morrison, "Embeddings of Integral Quadratic Forms"
                 http://www.math.ucsb.edu/~drm/manuscripts/eiqf.pdf .
 
@@ -5460,6 +5472,9 @@ REFERENCES:
             Number Theory" (ed. Y. Motohashi), London Math. Soc. Lecture Notes
             247 (1997), 313-320, Cambridge Univ. Press.
 
+.. [Mue2010] Jan Steffen Mueller, *Explicit Kummer surface formulas for arbitrary characteristic*.
+             LMS Journal of Computation and Mathematics, Volume 13, 47--64, 2010.
+
 .. [Mul2004] Siguna Muller, *On the Computation of Square Roots in
              Finite Fields*, in Designs, Codes and Cryptography,
              Volume 31, Issue 3 (March 2004)

src/sage/schemes/all.py

@@ -47,3 +47,5 @@
 from sage.schemes.cyclic_covers.all import *
 
 from sage.schemes.berkovich.all import *
+
+from sage.schemes.weighted_projective.all import *

src/sage/schemes/curves/projective_curve.py

@@ -807,19 +807,6 @@ def plot(self, *args, **kwds):
             Graphics object consisting of 1 graphics primitive
             sage: C.plot(patch=1)
             Graphics object consisting of 1 graphics primitive
-
-        A hyperelliptic curve::
-
-            sage: # needs sage.plot
-            sage: P.<x> = QQ[]
-            sage: f = 4*x^5 - 30*x^3 + 45*x - 22
-            sage: C = HyperellipticCurve(f)
-            sage: C.plot()
-            Graphics object consisting of 1 graphics primitive
-            sage: C.plot(patch=0)
-            Graphics object consisting of 1 graphics primitive
-            sage: C.plot(patch=1)
-            Graphics object consisting of 1 graphics primitive
         """
         # if user has not specified a favorite affine patch, take the
         # one avoiding "infinity", i.e. the one corresponding to the
@@ -2391,9 +2378,11 @@ def random_element(self):
 
         Sampling from points on a hyperelliptic curve::
 
-            sage: R.<x,y> = GF(13)[]
-            sage: C = HyperellipticCurve(y^2 + 3*x^2*y - (x^5 + x + 1))
-            sage: P = C.random_element(); P  # random
+            sage: R.<x> = GF(13)[]
+            sage: f = x^5 + x + 1
+            sage: h = 3*x^2
+            sage: C = HyperellipticCurve(f, h)
+            sage: P = C.random_point(); P  # random
             (0 : 1 : 0)
             sage: P in C
             True

src/sage/schemes/curves/weighted_projective_curve.py

@@ -34,7 +34,12 @@
 # ****************************************************************************
 
 from sage.schemes.curves.curve import Curve_generic
-from sage.schemes.weighted_projective.weighted_projective_space import WeightedProjectiveSpace_ring
+from sage.schemes.projective.projective_subscheme import (
+    AlgebraicScheme_subscheme_projective,
+)
+from sage.schemes.weighted_projective.weighted_projective_space import (
+    WeightedProjectiveSpace_ring,
+)
 
 
 class WeightedProjectiveCurve(Curve_generic):
@@ -49,6 +54,7 @@ class WeightedProjectiveCurve(Curve_generic):
         sage: C = Curve(y^2 - x^5 * z - 3 * x^2 * z^4 - 2 * z^6, WP); C
         Weighted Projective Curve over Rational Field defined by y^2 - x^5*z - 3*x^2*z^4 - 2*z^6
     """
+
     def __init__(self, A, X, *kwargs):
         if not isinstance(A, WeightedProjectiveSpace_ring):
             raise TypeError(f"A(={A}) is not a weighted projective space")
@@ -94,12 +100,108 @@ def projective_curve(self):
         from sage.schemes.projective.projective_space import ProjectiveSpace
 
         WP = self.ambient_space()
-        PP = ProjectiveSpace(WP.dimension_relative(), WP.base_ring(), WP.variable_names())
+        PP = ProjectiveSpace(
+            WP.dimension_relative(), WP.base_ring(), WP.variable_names()
+        )
         PP_ring = PP.coordinate_ring()
-        subs_dict = {name: var**weight for (name, var), weight in
-                     zip(WP.gens_dict().items(), WP.weights())}
+        subs_dict = {
+            name: var**weight
+            for (name, var), weight in zip(WP.gens_dict().items(), WP.weights())
+        }
 
         wp_polys = self.defining_polynomials()
         pp_polys = [PP_ring(poly.subs(**subs_dict)) for poly in wp_polys]
 
         return PP.curve(pp_polys)
+
+    def affine_patch(self, i, AA=None):
+        r"""
+        Return the `i`-th affine patch of this projective curve.
+
+        INPUT:
+
+        - ``i`` -- affine coordinate chart of the projective ambient space of
+          this curve to compute affine patch with respect to
+
+        - ``AA`` -- (default: ``None``) ambient affine space, this is constructed
+          if it is not given
+
+        .. TODO:
+
+            Implement this directly for weighted projective space, this code currently
+            just computes the related projective model and continues the computation from
+            there.
+
+        OUTPUT: a curve in affine space
+
+        EXAMPLES::
+
+            sage: WP.<x,y,z> = WeightedProjectiveSpace([1, 1, 1], QQ)
+            sage: C = WP.curve(x^3 - x^2*y + y^3 - x^2*z)
+            sage: C.affine_patch(1)
+            Affine Plane Curve over Rational Field defined by x^3 - x^2*z - x^2 + 1
+        """
+        from .constructor import Curve
+
+        projective_curve = self.projective_curve()
+        return Curve(
+            AlgebraicScheme_subscheme_projective.affine_patch(projective_curve, i, AA)
+        )
+
+    def riemann_surface(self, **kwargs):
+        r"""
+        Return the complex Riemann surface determined by this curve.
+
+        .. TODO:
+
+            Implement this directly for weighted projective space, this code currently
+            just computes the related projective model and continues the computation from
+            there.
+
+        OUTPUT: a :class:`~sage.schemes.riemann_surfaces.riemann_surface.RiemannSurface` object
+
+        EXAMPLES::
+
+            sage: WP.<x,y,z> = WeightedProjectiveSpace([1, 1, 1], QQ)
+            sage: C = WP.curve(x^3 + 3*y^3 + 5*z^3)
+            sage: C.riemann_surface()
+            Riemann surface defined by polynomial f = x^3 + 3*y^3 + 5 = 0,
+            with 53 bits of precision
+        """
+        return self.affine_patch(2).riemann_surface(**kwargs)
+
+    def plot(self, *args, **kwds):
+        """
+        Plot the real points of an affine patch of the associated projective
+        plane curve.
+
+        INPUT:
+
+        - ``self`` -- an affine plane curve
+
+        - ``patch`` -- (optional) the affine patch to be plotted; if not
+          specified, the patch corresponding to the last projective
+          coordinate being nonzero
+
+        - ``*args`` -- (optional) tuples (variable, minimum, maximum) for
+          plotting dimensions
+
+        - ``**kwds`` -- optional keyword arguments passed on to ``implicit_plot``
+
+        EXAMPLES:
+
+        A hyperelliptic curve::
+
+            sage: # needs sage.plot
+            sage: P.<x> = QQ[]
+            sage: f = 4*x^5 - 30*x^3 + 45*x - 22
+            sage: C = HyperellipticCurve(f)
+            sage: C.plot()
+            Graphics object consisting of 1 graphics primitive
+            sage: C.plot(patch=0)
+            Graphics object consisting of 1 graphics primitive
+            sage: C.plot(patch=1)
+            Graphics object consisting of 1 graphics primitive
+        """
+        projective_curve = self.projective_curve()
+        return projective_curve.plot(*args, **kwds)