Skip to content

Automated Release Management

Jump to bottom
rsheftel edited this page Oct 15, 2022 · 1 revision

This PR introduces the new realease management, following the discussion in #208.

This is a draft because I still need to create a CONTRIBUTING.md file and wait for @rsheftel to set the pypi tokens (see 'more detailed description'). After merging this, I will have to finalize the setup with a couple of commits and a few test-PRs.

New Release Management - Big Picture:

  1. Open PR against the dev branch. This triggers a workflow that runs tests and checks coverage.
  2. Merge PR into dev This triggeres nothing. It is the opportunity to make adjustments and to merge multiple PRs into dev, to make them a single release.
  3. Open PR of dev to main This triggers a workflow that runs tests, and uploads the package to testpypi, to test the upload and installation of the new version.
  4. Merge dev into main This triggers the final workflow, which sets up the new release, by pushing to pypi and creating a release on GitHub

Manual Steps:

  1. Merge all PRs into dev, that should be part of the same release
  2. On the dev branch, fill in .github/config_new_release.yml and make desired adjustments
  3. Open PR of dev to main and merge after tests pass

More Detailed Description:

  • Set up

    • There are four workflows

      • check_branch -> makes sure that only PRs from dev are opened against main
      • test_runner -> runs test suite
      • test_release -> runs tests, checks version, and tests the release on testpypi
      • releaser -> runs tests, prepares files for final release, pushes to pypi, creates release on GitHub

    • There is a token required for both testpypi and pypi. @rsheftel, I have sent you an e-mail about this to the email adress on your github profile, please let me know if you did not receive it.

    • A dev branch needs to be opened.

    • For the coveralls integration, the test_runner must have been run on both branches, otherwise coveralls won't report a coverage change.

  • Open PR against the dev branch (triggers: test_runner)

    • To standardize the whole release management, everything has to go through the dev branch.
    • Only the dev branch can open a PR against main, all others will immediately be closed by a workflow, which will also request the author to reopen it against the dev branch by leaving a comment.
    • All the tests are run with python versions 3.8, 3.9, 3.10 and all three latest OS's

  • Merge PR(s) into dev

    • When dev contains everything for the next release, fill in .github/config_new_release.yml
    • config_new_release.yml has three fields that need to be set:
      • new_version [str]: The new version number, which needs to be higher than what is currently in setup.py
      • change_log [multiline-str]: The list of changes that will be used for the change_log
      • release_body [multiline-str]: Additional text that will be included after the list of changes, when creating the official release on github

  • Open PR of dev to main (triggers test_release)

    • This should be done after config_new_release.yml is filled in
    • If the new_version field in config_new_release.yml is not incremented, relative to the version in setup.py, the test_release workflow will never test the new release on testpypi. The workflow will fail and a comment will be posted if this happens.
    • If it is incremented, the release candidate: new_version + "rc{run_number}.dev{run_attempt}" will be tested on testpypi, where 'run' and 'run_attempt' refer to integers that uniquely identify this workflow run.

  • Merge dev into main [or commit anything else to main] (triggers releaser)

    • Like test_release, if the new_version field is not incremented, this will not do any release work. Although, tests will be run and the merge/commit will be performed.
    • If new_version is incremented:
      • Together with the current date, new_version and change_log from config_new_release.yml will be inserted into docs/change_log.rst.
      • VERSION=... in setup.py and pyproject.toml will be replaced with VERSION=new_version
      • The package will be built, uploaded to pypi, downloaded and tested again
      • examples/usage.ipynb will be converted to rst, updating docs/usage.rst
      • The changes in setup.py, pyproject.toml, docs/change_log.rst and docs/usage.rst are committed to the repository
      • The GitHub Release is created, using new_version, change_log and release_body from config_new_release.yml

Takeaways for manual intervention:

  • Since the releaser will only upload the package to pypi if config_new_release contains an incremented version number, it is possible to commit to master without immediatly creating a new release. Technically, this allows a series of commits to be made with a final commit that increments the version number and only then triggers a new release. Although, I strongly recommend just sticking to the dev branch and testing the release through the standard process.

  • Regarding any changes to setup.py or pyproject.toml, the only thing these workflows rely on is that the version=... definition is at the top of the file. Our version number needs to be the first version number in the file.

Clone this wiki locally