Temporarily fix upgrade tests

[davidcassany]

I think we could consider three different test scenarios for toolkit:

• upgrade from a previous version to current head code [this is the current upgrade test]
• upgrade from current head version to an older version [this would be the downgrade test]
• upgrade from current head to current head [this would what downgrade and recovery tests are doing in this current PR]

As soon as we get a v2 released in elemental-toolkit we could easily implement this three tests scenarios. In fact there is only one missing.

[davidcassany]

With the current change in this PR, the upgrade happens to the same exact image, hence the /etc/os-release will not change after the upgrade. In fact to validate upgrades would be better to test state.yaml file. We could consider adding some helpers to fetch this file in SUT.

[anmazzotti]

What I do when manually testing is that I rebuild elemental-toolkit with an arbitrary GIT_COMMIT, then I build a second image using that elemental-toolkit version, and finally I can test upgrade was done by executing elemental version --long (that should include the GIT_COMMIT value).

Maybe in the tests we could follow a similar logic.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.53%. Comparing base (c03dc35) to head (b46dfc4).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1992   +/-   ##
=======================================
  Coverage   72.53%   72.53%           
=======================================
  Files          76       76           
  Lines        8909     8909           
=======================================
  Hits         6462     6462           
  Misses       1913     1913           
  Partials      534      534           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[davidcassany]

What I do when manually testing is that I rebuild elemental-toolkit with an arbitrary GIT_COMMIT, then I build a second image using that elemental-toolkit version, and finally I can test upgrade was done by executing elemental version --long (that should include the GIT_COMMIT value).

Maybe in the tests we could follow a similar logic.

The intention was to downgrade and upgrade to known and meaningful states for a basic compatibility check. So that upgrading from current stable (v1.1.x) and downgrading to v2 (assuming head is >= 2.0.0). Version 2 is still not tagged, but as soon as it is I'd just stick to that.

Then, what I think that could be also relevant is upgrading to the same head of the PR, this is likely to unveil some upgrade issues introduced by the PR itself that are not exercised with the upgrade and downgrade tests, unlikely but sill possible. Probably an even more complete approach could be downgrade first, reboot to active, reboot to fallback and upgrade from fallback to current head (which is indeed the same OS version). Probably this the closet we can get to test upgrades to an eventual higher upgrade.

Also something we are not doing in this tests and we should is rebooting to fallback. The problem is that in github performance is really poor and this will make upgrade tests quite longer.

[davidcassany]

Closed in favor of #2003 and #2004