Release process improvement ideas

[Ejdesgaard]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

23.0.0 got released, so far, so good.
23.0.1 was tagged, built and available, but no release Changelog or warnings to find anywhere.

Here is what I found when digging:

1. Release notes missing on https://nextcloud.com/Changelog
   • Indicating that 23.0.1 was not released
2. 23.0.1 was not to be found under releases on github
   • Indicating that 23.0.1 was not released
3. 23.0.1 was tagged and a PR was made for the release
   • I suspect this triggered a ci-cd for building and publishing the container images
4. 23.0.1 is available on docker.io
   • This makes it a de-facto release, since anyone who deployed eg. 23-apache and restarted the container, would automatically pull the latest 23 point release and thereby upgrade to 23.0.1

Steps to reproduce

1. Follow the release procedure that was used for the 23.0.1 and 23.0.2 releases

Expected behavior

What I would have expected when a bugged release hits the wild is as follows:

1. A process is in place to ensure that another release, in this case 23.0.2 gets released asap, with the fix or revert needed to resolve the critical bug.
2. The changelog https://nextcloud.com/Changelog gets updated with
   1. Release notes covering 23.0.1
   2. When the bug was deemed critical, then a big fat warning should be put on the changelog page, that highlights the issue, what this affects and how to mitigate it, to ensure that everyone who does their due-diligence and reads the changelog prior to an update or upgrade, gets the message.
   3. When 23.0.2 gets released, this changelog should only contain the delta between 23.0.1 and 23.0.2

Installation method

Official Docker image

Operating system

RHEL/CentOS

PHP engine version

No response

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

No response

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

No response

List of activated Apps

N/A

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

https://nextcloud.com/Changelog
#30840 (comment)
https://github.com/nextcloud/server/tree/v23.0.1
https://github.com/nextcloud/server/tree/v23.0.2
https://hub.docker.com/layers/nextcloud/library/nextcloud/23.0.1-apache/images/sha256-5b97169f2f986237472b0a0da84bab98b199cc36439e29f9a9b56d17343b50b4?context=explore

[solracsf]

Related #31201

[Ejdesgaard]

https://github.com/nextcloud/server/releases shows that the last release was 23.0.0

[Ejdesgaard]

nextcloud.com: https://nextcloud.com/Changelog/#latest23 lists 23.0.4 as released on 21. march 2022.
github: https://github.com/nextcloud/server/releases shows v23.0.4 as released "yesterday"
hub.docker.com: https://hub.docker.com/_/nextcloud/?tab=tags&page=1&name=23.0 doesn't even offer the 23.0.4 build.

[rawtaz]

generally painting a really bad picture of the organization around this really nice piece of software

This, literally in every part of the sentence (in other words, not just the "negative" part of it).

I have clients (companies) who are highly interested in using open source products like Nextcloud, but are starting to turn away from it (after having it on trial) due to the lack of proper coordination and project management like not even releasing versions properly or keeping changelogs up to date (or at all). It seems that NC is more interested in advertising new features than to actually follow up on them for more than one version. And other things.

I'm not trying to complain, I just don't know how to make the NC organization understand that how the project is currently managed, or more importantly how it's coming across to potential users, is being an actual problem for adoption of NC.

I also fail to see how it can be so utterly hard for an organization like NC to just 1) keep a fricken changelog up to date; 2) publish the darn thing with releases; 3) make proper releases in a coordinated fashion, across the places where they're supposed to be published. For gods sake, you have done this the right way before, so just fix whatever internal problem is causing this for you and start doing it right again.

[tormodvolden]

They were told (nextcloud/nextcloud.com#1634) and are aware of it, it just hasn't much priority. The "organization" is a commercial company. Unsurprisingly and fully understandable, they focus on their own paying customers, not yours. All in all, we should be grateful that some people are being paid for working on this.

[rawtaz]

All in all, we should be grateful that some people are being paid for working on this.

Yeah, I'm 100% with you there, I'm not looking to demand anything in the sense of "get me stuff for free", but I do think there's something to doing things properly if they are to be done in the first place (in particular when it doesn't take any relevant effort to do it right), e.g. releases and changelogs. Especially when it's being pointed out over and over again.

Unsurprisingly and fully understandable, they focus on their own paying customers, not yours.

I think I failed to convey what I meant earlier. The problem is this:

• Companies and similar who are looking at Nextcloud as a potential software to use for their needs, and I mean on a very serious note, starts looking at NC and trialing the software as well, and instead of finding that it's as good as it looks they find for example:
  • A company that fails at the very basics of software management, being seemingly unable to perform the super simple task of keeping a changelog up to date and publishing proper releases (and changelogs with them) in a coherent and controlled fashion. To be blunt, what they see is a mess, a lack of quality, and that's not an overstatement.
  • A product that while it works fine in many ways also has a bit too many issues, and where some issues aren't dealt with in a professional manner.
  • A company that seems to be much more interested in marketing new features that justify new versions of the product, with grand announcements, than to actually follow up on those features. It's easy to state that "we have this new awesome feature coming up" and then delivering a basic version of it, but when you then let it fall apart and/or stop working in subsequent versions, and when users point out that it's not working you still don't do anything about it, then what the heck. Get real and show potential customers that the features you advertise are something that people can count on being more than marketing fluff.

To clarify; I'm talking about potential customers who would be paying for the software, not those who would use it for free (although there's obviously no reason whatsoever to manage changelogs and releases properly for paying customers and not at the same time do that in the free/open source version).

In summary this isn't a matter of who pays and doesn't pay - as long as the fundamentals of the product is showing a severe lack of quality or project management, it doesn't help that you're a paying customer, because you can't really justify going with a product or company that fail at the absolute basics of managing the product. In other words, the issues I mentioned above are issues that undermine the faith and trust in the NC company, and you can't just throw money at that.

Again, I'm not trying to complain, I'm just trying to make some relevant person realize that there are super basic issues that are probably quite easy to fix, and that are currently turning potential paying customers away. But perhaps they aren't interested in more paying customers, because they already have so many due to their marketing :-)

Personally I just 1) wish they'd come to realize this, and 2) fix it, because it should be very easy to fix these issues.

[Ejdesgaard]

I have seen a few interviews with @karlitschek over the years(linked to 2 of them below) and my impression is that he believes that open-source is essential for the success of the company behind Nextcloud, so the premise that #31198 isn't important because it's not hitting the currently paying customers, should be an invalid statement.
If it has merit, then open-source isn't important for the NC company, and if that's the case, then the NC company should just be up-front about it.

I also agree with @rawtaz that it paints a really bad picture, if you can't or woun't handle a coherent release-process, especially since it worked well until some time ago.

IMO everyone is better of if the NC company do anything but what's happening now, such as one of the following:

1. If the release process gets fixed, then everyone can get back to focusing on the functionality and features of Nextcloud.
2. If the release process and OSS approach is scrapped, then no-one outside the NC company cares and Release process improvement ideas #31198 can just be closed with a statement that OSS isn't of any interest, moving forward.

I sincerely hope that the NC company decides the former.

Interview from 2017: https://www.youtube.com/watch?v=UhD3kfK1VH4
Interview from 2019: https://www.youtube.com/watch?v=fOmDW8hE-iA
Others are around on youtube if you search :)

[rawtaz]

Just like previous versions, the current release of Nextcloud, version 25 or Hub3, is missing a proper changelog.

Question to the Nextcloud company; What is the actual problem that is making you unable to produce a proper changelog for your software releases? Everyone else is able to do it, so why aren't you?