Release 0.18

[BigLep]

Meta

• Release owner: @galargh
• Release reviewer: @ajnavarro
• Expected RC date: 2022-12-08 week of 2022-12-12
• Expected final release date: 2022-12-15 week of 2022-12-19 2023-01
• Accompanying PR for improving the release process: Process Improvement: v0.18.0 #9484

See the Kubo release process for more info.

Kubo 0.18.0 Release

We're happy to announce Kubo 0.18.0!

As usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will not be called out in the release notes. Please make sure to update ASAP. See our security fix policy for details.

🗺 What's left for release

We're pushing to do another release before the end of 2022 because:

1. Get delegated routing with HTTP into a good state (per IPIP-337: Delegated Content Routing HTTP API specs#337)
2. General wrap up to the year

Project board: IPFS Shipyard Team (view)

Required before RC

• Gateways stall after 5 minutes when using reframe/delegated routing #9445
• deps: go-merkledag@0.8.1 #9318
• fix: expose agent-version-suffix via identify protocol #9457
• Update Kubo to new HTTP Delegated Routing #9422
• Add implicit HTTP delegated router for cid.contact network indexer #9454
• [Tracking issue] go-libp2p resource manager critical post release fixes #9442 (complete the "Must complete before 0.18 RC" items")
• go-libp2p 0.24 update
  • WIP: update go-libp2p to v0.24.0 #9423
  • migrations needed after updating go-libp2p v0.24.0 update #9410
  • fix: support /quic-v1 in webui v0.21 #9490
• Enable WebTransport by default #9292 (depends on libp2p 0.24 update)
• New defaults for the Provider Record Republish and Expiration (22h/48h) #9389
  • feat: increase default Reprovider.Interval #9326
  • switch Kubo to go-libp2p-kad-dht release with:
    • feat: increase the max record age to 48h (PUT_VALUE, RFM17) libp2p/go-libp2p-kad-dht#794
    • feat: increase expiration time for Provider Records to 48h (RFM17) libp2p/go-libp2p-kad-dht#793
• (@Jorropo) fs-repo-12-to-13 migration
  • merge and release feat: add fs-repo-12-to-13 fs-repo-migrations#162
  • add fs-repo-12-to-13 to dist.ipfs.tech
  • switch Kubo to new migrations by bumping the hardcoded CID
• Changelog cleanup: Clened up 0.18 changelog for release #9497

Items to during the RC phase before the final release

• Fully close out [Tracking issue] go-libp2p resource manager critical post release fixes #9442
  • Improve the utility of the swarm stats command for determining which limits are being hit #9474
  • fix: Avoid unknown fields on config #9438
• go-libp2p v0.24.1 libp2p/go-libp2p#1940
• fix(gateway): JSON when Accept is a list #9533

Nice to have

• Gateway support for dag-json and dag-cbor responses #8823

🔦 Highlights

< top highlights for this release notes. For ANY version (final or RCs) >

✅ Release Checklist

Checklist:

• Stage 0 - Prerequisites
  • Open an issue against bifrost-infra ahead of the release (example). Idealy, do this multiple days in advance of the RC to give Bifrost the heads up that asks will be coming their way.
    • Spell out all that we want updated - gateways, the bootstraper and the cluster/preload nodes
    • Mention @protocol/bifrost-team in the issue and let them know the expected date of the release
      • Issue link: https://github.com/protocol/bifrost-infra/issues/2221
  • Ensure that the What's left for release section has all the checkboxes checked. If that's not the case, discuss the open items with Kubo maintainers and update the release schedule accordingly.
  • Create docs-release-v0.18.0 branch, open a draft PR and keep updating docs/RELEASE_ISSUE_TEMPLATE.md on that branch as you go.
    • Link it in the "Meta" section above.
  • Ensure you have a GPG key generated and added to your GitHub account. This will enable you to created signed tags.
  • Ensure you have admin access to IPFS Discourse. Admin access is required to globally pin posts and create banners. @2color might be able to assist you.
  • Access to #bifrost channel in FIL Slack might come in handy. Ask the release reviewer to invite you over.
  • Access to #shared-pl-marketing-requests channel in FIL Slack will be required to request social shares. Ask the release reviewer to invite you over.
  • After the release is deployed to our internal infrastructure, you're going to need read access to IPFS network metrics dashboards. Open an access request in https://github.com/protocol/pldw/issues/new/choose if you don't have it yet (example).
  • You're also going to need NPM installed on your system. See here for instructions.
  • Prepare changelog proposal in docs/changelogs/v0.18.md.
    • Skip filling out the ### Changelog section (the one where which lists all the commits and contributors) for now. We will populate it after the release branch is cut.
    • PR link:
  • Ensure the new changelog is linked in the CHANGELOG.md file.
  • Install ZSH (instructions). It is needed by the changelog creation script.
  • Ensure you have kubo checked out under $(go env GOPATH)/src/github.com/ipfs/kubo. This is required by the changelog creation script.
    • If you want your clone to live in a different location, you can symlink it to the expected location by running mkdir -p $(go env GOPATH)/src/github.com/ipfs && ln -s $(pwd) $(go env GOPATH)/src/github.com/ipfs/kubo.
  • Ensure that README.md is up to date.
• Stage 1 - Initial Preparations
  • Fork a new branch (release-v0.18.0) from master.
  • Bump the version in version.go in the master branch to vX.(Y+1).0-dev via a PR (example).
• Stage 2 - Release Candidate - if any non-trivial changes need to be included in the release, return to this stage
  • If it's not a first RC, add new commits to the release-v0.18.0 branch from master using git cherry-pick -x ...
    • Note: release-* branches are protected. You can do all needed updates on a separated branch (e.g. wip-release-v0.18.0) and when everything is settled push to release-v0.18.0
  • Bump the version in version.go in the release-v0.18.0 branch to v0.18.0-rcN.
  • If it's a first RC, create a draft PR targetting release branch if it doesn't exist yet (example).
  • Wait for CI to run and complete PR checks. All checks should pass.
  • Create a signed tag for the release candidate.
    • This is a dangerous operation, as it is difficult to reverse due to Go modules and automated Docker image publishing. Remember to verify the commands you intend to run for items marked with ⚠️ with the release reviewer.
    • ⚠️ Tag HEAD release-v0.18.0 commit with v0.18.0-rcN (git tag -s v0.18.0-rcN -m 'Pre-release 0.18.0-rcn')
    • Run git show v0.18.0-rcN to ensure the tag is correct.
    • ⚠️ Push the v0.18.0-rcN tag to GitHub (git push origin v0.18.0-rcN; DO NOT USE git push --tags because it pushes all your local tags).
  • Add artifacts to https://dist.ipfs.tech by making a PR against ipfs/distributions
    • Clone the ipfs/distributions repo locally.
    • Create a new branch (kubo-release-v0.18.0-rcn) from master.
    • Run ./dist.sh add-version kubo v0.18.0-rcN to add the new version to the versions file (instructions).
      • dist.sh will print WARNING: not marking pre-release kubo v0.18.0-rcN as the current version..
    • Push the kubo-release-v0.18.0-rcn branch to GitHub and create a PR from that branch (example).
    • Ask for a review from the release reviewer.
    • Enable auto-merge for the PR.
      • PR build will build the artifacts and generate a diff in around 30 minutes
      • PR will be merged automatically once the diff is approved
      • master build will publish the artifacts to https://dist.ipfs.io in around 30 minutes
    • Ensure that the artifacts are available at https://dist.ipfs.io
  • Publish the RC to the NPM package by running https://github.com/ipfs/npm-go-ipfs/actions/workflows/main.yml (it happens automatically but it is safe to speed up the process and kick of a run manually)
  • Cut a pre-release on GitHub (instructions, example)
    • Use v0.18.0-rcN as the tag.
    • Link to the release issue in the description.
    • Link to the relevant changelog in the description.
    • Check This is a pre-release.
  • Synchronize release artifacts by running sync-release-assets workflow.
  • Announce the RC
    • Create a new post on IPFS Discourse. (example)
      • Use Kubo v0.18.0-rcn Release Candidate is out! as the title.
      • Use kubo and go-ipfs as topics.
      • Repeat the title as a heading (##) in the description.
      • Link to the GitHub Release, binaries on IPNS, docker pull command and release notes in the description.
    • Pin the topic globally so that it stays at the top of the category.
    • If there is no more important banner currently set on Discourse (e.g. IPFS Camp announcement), make the topic into a banner.
    • Check if Discourse post was automatically copied to:
      • IPFS Discord #ipfs-chatter
      • FIL Slack #ipfs-chatter
      • Matrix https://matrix.to/#/#ipfs-chatter:ipfs.io
    • Mention early testers in the comment under the release issue (example).
• Stage 3 - Internal Testing
  • Infrastructure Testing.
    • Update the issue against bifrost-infra (example).
      • Mention @protocol/bifrost-team in the issue to let them know the release is ready
      • [Optional] Reply under a message about the issue in the #bifrost channel on FIL Slack once the RC is out. Send the message to the channel.
    • Check metrics every day.
      • Compare the metrics trends week over week.
      • If there is an unexpected variation in the trend, message the #bifrost channel on FIL Slack and ask for help investigation the cause.
  • IPFS Application Testing.
    • IPFS Desktop
      • Upgrade to the RC in ipfs-desktop
      • Run npm install to update package-lock.json.
      • Push to a branch (example)
      • Open a draft PR to track through the final release (example)
      • Ensure CI tests pass
    • IPFS Companion
      • Start kubo daemon of the version to release.
      • Start a fresh chromium or chrome instance using chromium --user-data-dir=$(mktemp -d) (macos /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --user-data-dir=$(mktemp -d))
      • Start a fresh firefox instance using firefox --profile $(mktemp -d) (macos /Applications/Firefox.app/Contents/MacOS/firefox --profile $(mktemp -d))
      • Install IPFS Companion from vendor-specific store.
      • Check that the comunication between Kubo daemon and IPFS companion is working properly checking if the number of connected peers changes.
• Stage 5 - Release - ONLY FOR FINAL RELEASE
  • Prepare the release branch.
    • Bump the version in version.go in the release-v0.18.0 branch to 0.18.0.
    • Update the docs/changelogs/v0.18.md with the new commits and contributors.
      • Run ./bin/mkreleaselog twice to generate the changelog and copy the output.
      • The first run of the script might be poluted with git clone output.
      • Paste the output into the ### Changelog section of the changelog file inside the <details><summary></summary></details> block.
      • Commit the changelog changes.
    • Push the release-v0.18.0 branch to GitHub (git push origin release-v0.18.0)
    • Mark the PR created from release-v0.18.0 as ready for review.
      • Ensure the PR is targetting release branch.
      • Ensure that CI is green.
      • Have release reviewer review the PR.
    • Merge the PR into release branch using the Create a merge commit (do NOT use Squash and merge nor Rebase and merge because we need to be able to sign the merge commit).
      • Do not delete the release-v0.18.0 branch.
    • Checkout the release branch locally.
      • Remember to pull the latest changes.
    • Create a signed tag for the release.
      • This is a dangerous operation, as it is difficult to reverse due to Go modules and automated Docker image publishing. Remember to verify the commands you intend to run for items marked with ⚠️ with the release reviewer.
      • ⚠️ Tag HEAD release commit with v0.18.0 (git tag -s v0.18.0 -m 'Release 0.18.0')
      • Run git show v0.18.0 to ensure the tag is correct.
      • ⚠️ Push the v0.18.0 tag to GitHub (git push origin v0.18.0; DO NOT USE git push --tags because it pushes all your local tags).
  • Publish the release.
    • Wait for Publish docker image workflow run initiated by the tag push to finish.
    • Add artifacts to https://dist.ipfs.tech by making a PR against ipfs/distributions
      • Clone the ipfs/distributions repo locally.
      • Create a new branch (kubo-release-v0.18.0) from master.
      • Run ./dist.sh add-version kubo v0.18.0 to add the new version to the versions file (instructions).
      • Push the kubo-release-v0.18.0 branch to GitHub and create a PR from that branch (example).
      • Ask for a review from the release reviewer.
      • Enable auto-merge for the PR.
        • PR build will build the artifacts and generate a diff in around 30 minutes
        • PR will be merged automatically once the diff is approved
        • master build will publish the artifacts to https://dist.ipfs.io in around 30 minutes
      • Ensure that the artifacts are available at https://dist.ipfs.io
    • Publish the release to the NPM package by running https://github.com/ipfs/npm-go-ipfs/actions/workflows/main.yml (it happens automatically but it is safe to speed up the process and kick of a run manually)
  • Cut the release on GitHub (instructions, example)
    • Use v0.18.0 as the tag.
    • Copy the relevant changelog into the release description.
    • Keep the release notes as trim as possible (e.g. remove top headers where possible, example)
  • Synchronize release artifacts by running sync-release-assets workflow.
  • TODO: https://github.com/protocol/bifrost-infra/issues/2184#issuecomment-1315279257
  • Announce the release
    • Add a link to the release to this release issue as a comment.
    • Create a new post on IPFS Discourse. (example)
      • Use Kubo v0.18.0 Release is out! as the title.
      • Use kubo and go-ipfs as topics.
      • Repeat the title as a heading (##) in the description.
      • Link to the GitHub Release, binaries on IPNS, docker pull command and release notes in the description.
    • Pin the topic globally so that it stays at the top of the category.
    • If there is no more important banner currently set on Discourse (e.g. IPFS Camp announcement), make the topic into a banner.
    • Check if Discourse post was automatically copied to:
      • IPFS Discord #ipfs-chatter
      • FIL Slack #ipfs-chatter
      • Matrix
  • Add a link from release notes to Discuss post (like we did here: https://github.com/ipfs/kubo/releases/tag/v0.17.0)
  • Update the draft PR created for interop to use the new release and mark it as ready for review.
  • Update the draft PR created for IPFS Desktop to use the new release and mark it as ready for review.
  • Update docs
    • Run https://github.com/ipfs/ipfs-docs/actions/workflows/update-on-new-ipfs-tag.yml to generate a PR to the docs repo
    • Merge the auto-created PR in https://github.com/ipfs/ipfs-docs/pulls (example)
  • Get the blog post created
    • Submit a request for blog post creation using the form.
      • Title: Just released: Kubo 0.18.0!
      • Link type: Release notes
      • URL: https://github.com/ipfs/kubo/releases/tag/v0.18.0
    • The post is live on https://blog.ipfs.io
  • Share the link to the GitHub release
    • Twitter (request in Filecoin Slack channel #shared-pl-marketing-requests; example)
    • Reddit
• Stage 6 - Post-Release
  • Merge the release branch back into master, ignoring the changes to version.go (keep the -dev version from master).
  • Create an issue using this release issue template for the next release.
  • Close this release issue.

How to contribute?

Would you like to contribute to the IPFS project and don't know how? Well, there are a few places you can get started:

• Check the issues with the help wanted label in the ipfs/kubo repo
• Join the discussion at discuss.ipfs.tech and help users finding their answers.
• See other options at https://docs.ipfs.tech/community/

[Jorropo]

I don't knows how that will play exactly into the release process but we will have to publish a migration binary to dist.ipfs.io too, that will get downloaded by Kubo, I guess it needs to be released before or when Kubo is released to avoid race conditions where kubo can't find the migration binary.

[BigLep]

2022-12-07 update:

We aren't going to be ready for an RC early Thursday, 2022-12-08.

The previous "nice to haves" have become "required" since some of the "required" items needed a repo migration. Since we were already going to be doing a repo migration anyways, we're taking on all the things (including the previous "nice to haves") for the repo migration.

The issue description is the ultimate source of truth but we have these tracks of work:

1. (@Jorropo @lidel): Repo migrations. This is related to multiple features. We want to publish @Jorropo's (migrations needed after updating go-libp2p v0.24.0 update #9410 ) first and then @lidel will put additional migrations on top.
2. (@Jorropo): go-lipbp2p 0.24 upgrade: WIP: update go-libp2p to v0.24.0 #9423
3. (@lidel unless he defers to @ajnavarro): Review/merge the resource manager docs: docs: libp2p resource management #9468

@Jorropo or @lidel : please callout if there is work that other can help you with like @ajnavarro .

We'll talk about the 0.18 RC date during Kubo standup on 2022-12-08.

In summary @galargh : there isn't anything for you to immediately jump on concerning 0.18 when you start work on 2022-12-08. That said, lets please make sure we've got Bifrost engaged for the deployments we'll want them to do. (I'm hopeful we can do one-box deployments early Friday so we get data over the weekend.)