build: add pyproject.toml, poetry, and tox

[bonjourmauko]

Technical changes

• Add pyproject.toml to the repository.
• Add poetry to manage dependencies.
• Add tox to manage isolated tests.

Deprecations

• openfisca_core.commons.Dummy => openfisca_core.commons.empty_clone
• openfisca_core.errors.ParameterNotFound => openfisca_core.errors.ParameterNotFoundError
• openfisca_core.errors.VariableNameConflict => openfisca_core.errors.VariableNameConflictError
• openfisca_core.errors.VariableNotFound => openfisca_core.errors.VariableNotFoundError
• openfisca_core.formula_helpers.py => openfisca_core.commons
• openfisca_core.memory_config.py => openfisca_core.experimental
• openfisca_core.parameters.Bracket => openfisca_core.errors.ParameterScaleBracket
• openfisca_core.parameters.ParameterNotFound => openfisca_core.errors.ParameterNotFoundError
• openfisca_core.parameters.ParameterParsingError => openfisca_core.errors.ParameterParsingError
• openfisca_core.parameters.Scale => openfisca_core.errors.ParameterScale
• openfisca_core.rates => openfisca_core.commons
• openfisca_core.simulation_builder => openfisca_core.simulations
• openfisca_core.simulations.CycleError => openfisca_core.errors.CycleError
• openfisca_core.simulations.NaNCreationError => openfisca_core.errors.NaNCreationError
• openfisca_core.simulations.SpiralError => openfisca_core.errors.SpiralError
• openfisca_core.taxbenefitsystems.VariableNameConflict => openfisca_core.errors.VariableNameConflictError
• openfisca_core.taxbenefitsystems.VariableNotFound => openfisca_core.errors.VariableNotFoundError
• openfisca_core.taxscales.EmptyArgumentError => openfisca_core.errors.EmptyArgumentError

[MattiSG]

Exciting! 😃

I understand that this PR brings a few different sets of changes:

1. Refactor of the way requirements are declared: from arrays in the setup.py to separate files in the requirements folder.
2. Pinning down all non-default requirements versions.
3. Supporting constraining the numpy version.
4. Change the supported Python version from 3.6 and 3.7 to 3.7, 3.8 and 3.9.
5. Refactor all CI-specific dependencies to the standard requirements declaration.

If this is correct, then:

• I am very much in support of (5).
• While in favour of it, I don't understand how this changeset enables (4). I believe we should add matrix testing in CI if we want to support multiple Python versions.
• I don't understand the expected usage of (3): if we advertise support for multiple numpy versions, why do we test only against the oldest version?
• I don't understand the added value of (2), while I see the cost of needing to deploy a new Core version whenever a patch is published in one of the dev requirements 😰
• I am not sure I understand the added value of (1). I like how each set of requirements has its own file, but I also like that all requirements are in setup.py. What is the most standard in Python dependency management?

[bonjourmauko]

Thanks for the review!

• I am very much in support of (5).

🙌

• While in favour of it, I don't understand how this changeset enables (4).

You're right, I forgot to add pyupgrade.

I believe we should add matrix testing in CI if we want to support multiple Python versions.

We could. I'm assuming 3.7 compatible code will also be 3.7+, albeit naively maybe.

• I don't understand the expected usage of (3): if we advertise support for multiple numpy versions, why do we test only against the oldest version?

Again, I'm naively assuming that if 1.17 works, and 1.21 works, all in between should work.

• I don't understand the added value of (2)

The idea is to:

1. reduce entropy for contributors —without impact for reusers
2. being able to use pip without the legacy option —currently it is too slow as dep ranges are too broad

while I see the cost of needing to deploy a new Core version whenever a patch is published in one of the dev requirements 😰

It's definitely a trade-off.

My assumption is that we don't need to:

• core dep ranges needs to be broad —we can't know reusers' constraints upfront
• numpy specifically needs also to be retro-compatible, for the same reason
• the rest do not, as dev is done locally, so we can live with outdated dev deps and update them once every 3 months for ex.

Again it is just an assumption.

• I am not sure I understand the added value of (1). I like how each set of requirements has its own file, but I also like that all requirements are in setup.py. What is the most standard in Python dependency management?

Seems to be to use different files (both for requirements and for constraints), similar to what's proposed here.

I did actually borrowed the idea to both https://github.com/opendatateam/udata and https://github.com/jazzband/pip-tools.

[MattiSG]

I'm assuming 3.7 compatible code will also be 3.7+

If that assumption held, then why wouldn't 3.6-compatible code be 3.7-compatible? 🤔

I'm naively assuming that if 1.17 works, and 1.21 works

This sounds reasonable but I wouldn't advertise compatibility explicitly if there is no guaranteed testing.
Also, I'm not sure I understand how the system implemented in this PR checks for both upper and lower bounds. I understand it checks explicitly for the lower bound, and checks for some version matching version constraints, on which we have no control, and we assume it should be the latest but we are not certain (maybe the cache is old and the package manager is satisfied with it).

1. reduce entropy for contributors —without impact for reusers

Do we have documented cases of debugging / contribution being hindered and that would have been solved by pinning?

we can't know reusers' constraints upfront

Exactly, that's why I'm a bit worried with pinning dependencies.

I'd love to have @openfisca/france-contrib-ipp, @guillett, @MaxGhenis… chime in on that. What would be the expected impact of pinning down dependencies for your teams?

[bonjourmauko]

Thanks @MattiSG, I'll split this PR so to move incrementally while reaching consensus on the controversial bits 👍

[bonjourmauko]

Well I did a couple of changes, it looks better now :)

[MattiSG]

Thank you very much for your work!

This gets pretty close to shipping. I believe it might be appropriate to use this very impactful change to ship as many breaking deprecations as possible at once, as reusers are in for some adaptation work. I think of removing Dummy, for example. Do you see others?

[bonjourmauko]

@MattiSG ?

[MattiSG]

When I see the impact on country-template (openfisca/country-template#121), I am clearly in favour of getting rid of the system of deprecation and actually using major version breaks. I fail to understand the added value of this delayed way of updating the API, and now we have PRs that are not backwards-compatible yet have no version bump 🙃

[bonjourmauko]

As things have converged toward the use of a lockfile, I'm closing this PR in favour of opening a new one.