Make IndexedFreeModuleElement compatible with collections.abc, change method support to return a SupportView

[mkoeppe]

This is the element class of a CombinatorialFreeModule.

Its implementation of __iter__, __contains__, __len__ are not compatible with anything in the collections.abc protocols. Specifically,

sage: F = CombinatorialFreeModule(QQ, ['a','b','c'])
sage: B = F.basis()
sage: f = B['a'] + 3*B['c']; f
sage: import collections.abc
sage: isinstance(f, collections.abc.Collection)
True

but __iter__ating over this collection gives objects that are not __contain__ed in it.

To improve interoperability, we propose to make the following changes:

• Deprecate the __contains__ method (currently testing whether a given index is in the support). After its removal, the class would no longer be considered by Python to be a subclass of collections.abc.Collection, but merely a Sized Iterable.
• As a replacement for this functionality, revise the method support to return an instance of a new class SupportView, which provides a fast __contains__ method, instead of a list.

(Similarly, in #24815, it was proposed to make the elements of free modules from sage.modules a collections.abc.Sequence.)

Part of Meta-ticket #30309: Unify free module elements API: methods dict, monomial_coefficients, etc.

Depends on #34505

CC: @fchapoton @tscrim @mwageringel @nthiery @anneschilling

Component: linear algebra

Author: Matthias Koeppe

Branch/Commit: 2fe3b98

Reviewer: Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/34509

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,8 +1,10 @@
+This is the element class of a `CombinatorialFreeModule`.
+
 Its implementation of `__iter__`, `__contains__`, `__len__` are not compatible with anything in the [collections.abc](https://docs.python.org/3/library/collections.abc.html) protocols.
 
 To improve interoperability, we propose to make changes to make it a [collections.abc.Mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) (= supporting the operations of a `dict` but without mutability):
 - Because a `Mapping` is a `Collection`, we change the `__iter__` method from yielding (key, value) items to yielding keys only. (Thus it will be compatible with the existing `__contains__` method which check for the presence of a key)
 - Add methods `keys`, `items`, `values` with the semantics familiar from `dict`.
 
-(In #24815, it was proposed to make it a [collections.abc.Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence).)
+(Similarly, in #24815, it was proposed to make the elements of free modules from `sage.modules` a [collections.abc.Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence).)
 

[mkoeppe]

Dependencies: #34505

[mkoeppe]

Description changed:

--- 
+++ 
@@ -4,7 +4,7 @@
 
 To improve interoperability, we propose to make changes to make it a [collections.abc.Mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) (= supporting the operations of a `dict` but without mutability):
 - Because a `Mapping` is a `Collection`, we change the `__iter__` method from yielding (key, value) items to yielding keys only. (Thus it will be compatible with the existing `__contains__` method which check for the presence of a key)
-- Add methods `keys`, `items`, `values` with the semantics familiar from `dict`.
+- Add methods `keys`, `items`, `values` with the semantics familiar from `dict`. If ``self`` is a submodule, they refer to the ambient module.
 
 (Similarly, in #24815, it was proposed to make the elements of free modules from `sage.modules` a [collections.abc.Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence).)
 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -4,7 +4,7 @@
 
 To improve interoperability, we propose to make changes to make it a [collections.abc.Mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) (= supporting the operations of a `dict` but without mutability):
 - Because a `Mapping` is a `Collection`, we change the `__iter__` method from yielding (key, value) items to yielding keys only. (Thus it will be compatible with the existing `__contains__` method which check for the presence of a key)
-- Add methods `keys`, `items`, `values` with the semantics familiar from `dict`. If ``self`` is a submodule, they refer to the ambient module.
+- Add methods `keys`, `items`, `values` with the semantics familiar from `dict`. If `self` is a submodule, they refer to the ambient module (see #34455).
 
 (Similarly, in #24815, it was proposed to make the elements of free modules from `sage.modules` a [collections.abc.Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence).)
 

[tscrim]

comment:5

I am against this change. They are vectors and only on their backend are implemented as a dict, but they are not dictionaries, nor mappings. Likewise, I strongly oppose changing them to iterating over keys. This brings their implementation further from the other free module vectors. The reason we iterate over both is because there is no natural way to order the basis so the sequence of coefficients occurs in a particular order (which will unfortunately be a source of incompatibility). Hence, most of the time when iterating we want to use both the support index and the coefficient. Being vectors, they don’t have keys and values, but support and coefficients. Nor do they have a reasonable notion of containment.

Contrast this to the “usual” free modules, where they have a natural compatibility with sequence as elements in Rⁿ, which are also how we think of finite sequences.

[mkoeppe]

comment:6

Replying to Travis Scrimshaw:

Being vectors, they don’t have keys and values, but support and coefficients. Nor do they have a reasonable notion of containment.

It does define __contains__ currently; no change to that is proposed.

    def __contains__(self, x):
        """
        Returns whether or not a combinatorial object x indexing a basis
        element is in the support of self.
    ...

[mkoeppe]

comment:7

Replying to Travis Scrimshaw:

I strongly oppose changing them to iterating over keys. This brings their implementation further from the other free module vectors.

This makes no sense.

[mkoeppe]

comment:8

Replying to Travis Scrimshaw:

most of the time when iterating we want to use both the support index and the coefficient.

Yes, and there are two interpretations of this when the parent is a submodule.

• one is monomial_coefficients, which refers to the basis of the submodule
• one is the proposed new items, which will refer to the ambient. (This will be compatible with sparse vectors in sage.modules.)

[mkoeppe]

Description changed:

--- 
+++ 
@@ -8,3 +8,4 @@
 
 (Similarly, in #24815, it was proposed to make the elements of free modules from `sage.modules` a [collections.abc.Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence).)
 
+Part of Meta-ticket #30309: Unify free module elements API: methods dict, monomial_coefficients, etc.

[tscrim]

comment:10

Replying to Matthias Köppe:

Replying to Travis Scrimshaw:

I strongly oppose changing them to iterating over keys. This brings their implementation further from the other free module vectors.

This makes no sense.

We want the two implementations to have the same API as much as possible. Rⁿ vectors iterate over coefficients, but iterating over keys/supports here would be the exact opposite of that behavior.

[mkoeppe]

comment:11

Yes, it makes no sense to say that returning (key,value) pairs is closer to being the "same API" as returning values.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,6 +1,16 @@
 This is the element class of a `CombinatorialFreeModule`.
 
-Its implementation of `__iter__`, `__contains__`, `__len__` are not compatible with anything in the [collections.abc](https://docs.python.org/3/library/collections.abc.html) protocols.
+Its implementation of `__iter__`, `__contains__`, `__len__` are not compatible with anything in the [collections.abc](https://docs.python.org/3/library/collections.abc.html) protocols. Specifically, 
+
+```
+sage: F = CombinatorialFreeModule(QQ, ['a','b','c'])
+sage: B = F.basis()
+sage: f = B['a'] + 3*B['c']; f
+sage: import collections.abc
+sage: isinstance(f, collections.abc.Collection)
+True
+```
+but `__iter__`ating over this collection gives objects that are not `__contain__`ed in it.
 
 To improve interoperability, we propose to make changes to make it a [collections.abc.Mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) (= supporting the operations of a `dict` but without mutability):
 - Because a `Mapping` is a `Collection`, we change the `__iter__` method from yielding (key, value) items to yielding keys only. (Thus it will be compatible with the existing `__contains__` method which check for the presence of a key)

[tscrim]

comment:13

Replying to Matthias Köppe:

Yes, it makes no sense to say that returning (key,value) pairs is closer to being the "same API" as returning values.

So actually returning the coefficient as part of the iterator instead of saying you should access it by a completely different method is exactly the same by your logic. You wouldn’t agree that it is a much smaller change to have Rⁿ vector iterators change to return the pairs?

[mkoeppe]

comment:14

Replying to Travis Scrimshaw:

change to have Rⁿ vector iterators change to return the pairs?

That would be a really bad change for dense vectors, which clearly want to be a collections.abc.Sequence

[tscrim]

comment:15

Replying to Matthias Köppe:

Replying to Travis Scrimshaw:

most of the time when iterating we want to use both the support index and the coefficient.

Yes, and there are two interpretations of this when the parent is a submodule.

• one is monomial_coefficients, which refers to the basis of the submodule
• one is the proposed new items, which will refer to the ambient. (This will be compatible with sparse vectors in sage.modules.)

Unfortunately because of different implementation goals (mainly the speed that is needed for the Rⁿ modules), we cannot enforce compatibility for everything. I would rather all vectors follow what CFM vectors do from a more abstract mathematical point of view, but we have to make trade-offs. This one is here very clear-cut.

Also, it isn’t just limited to sparse vectors, right?