test: improve commons module doctests

[bonjourmauko]

Fixes #1250

New Features

• Introduce openfisca_core.types

Bug Fixes

• Fix doctests of the commons module

Documentation

• Complete the documentation of the commons module

[HAEKADI]

Thank you @maukoquiroga! I ran the tests and everything seems to be in order :)

[MattiSG]

Thank you @maukoquiroga, this is impressive documentation work! 😲 👏

How are these tests run in CI?

I would personally need the following to validate that PR:

1. A review by at least one person who has extensive knowledge of the API and can double-check both the given information and that this is isofunctional in terms of tests. I'll ask for a review accordingly.
2. As mentioned in a comment, that all style convention changes are extracted from this PR.
3. That an explanation on how to run these tests and produce the documentation is added, at least to this PR, possibly to the doc itself, so that I can validate the changeset on my machine 🙂

[MattiSG]

How can I confirm this would fix openfisca/openfisca-doc#244? 🙂

[bonjourmauko]

Thanks for your comments @HAEKADI & @MattiSG

How can I confirm this would fix openfisca/openfisca-doc#244? 🙂

Good point, I'll work on it.

[benjello]

@maukoquiroga @MattiSG @HAEKADI : definitely ok to review the docstrings but I might be of better use after decisions are made about docsting format etc.

[bonjourmauko]

How can I confirm this would fix openfisca/openfisca-doc#244? 🙂

I kind of was forced by the magics of test-driven documentation to fix it before openfisca/openfisca-doc#251

The idea would be to reach a point where builds fail in CI when the docs, including doctests, fail to build.

[bonjourmauko]

@maukoquiroga @MattiSG @HAEKADI : definitely ok to review the docstrings but I might be of better use after decisions are made about docsting format etc.

What I intend as a first step is to make whatever exists today work, which is not the case.

That includes roughly two things:

• Doctests, present both in the code documentation and in the markdown documentation. Both can be verified programatically so in dev and CI.

• Documentation, where beyond doctests, you have a style —today we live with three different styles: Google, NumPy, and Shpinx—, and then prose.

  • Changes in Fix documentation openfisca-doc#251 are to be able to build the three styles (minimal possible change)
  • Changes here are to enforce Google style for new documentation
  • I don't intend to add a check for prose.

Of course, I agree with @MattiSG the aim is to automatise as much as we can, which is possible.

[bonjourmauko]

That was some work! So I've:

• Removed all style changes to code
• Added pytest to automatically check that commons doctests run
• Added linters to automatically check that docstrings are valid and up-to-date :) 🙌

@benjello regarding the style of docstrings specifically, do you have any preference? To resume my take on this:

• There are four main styles: Google, NumPy, Sphinx, and PEP
• We currently have all four in the code 😄 and no official guideline as to prefer one over the others
• I like Google because is easier to write and read, and numpydoc could make sense because we use numpy a lot.

In any scenario, this changeset has little to no impact on that because we're adding/completing doc and doctests rather than rewriting it to a style or the other, but as we move on documenting the whole code base (are we?) it could make sense to have a collective community discussion to choose one style and adhere to it, as it'll make contributions and maintenance easier.

[benjello]

@maukoquiroga : I like Google style too.

[benjello]

See my comment.
My fear is the the current typing may force bool and int array conversion to float array.
Otherwise good to go.
Thanks !

[MattiSG]

The only blocking change for me is the addition of a test coverage threshold in a PR presented as improving doctests. I believe that such a change has a potential impact on contributions acceptance processes, and should be subject to a separate discussion 🙂

I'm glad to see we've made a lot progress and are very close to shipping, notably thanks to a direct discussion with @maukoquiroga yesterday. Thanks for that!

[MattiSG]

🚀 😭