Refactor ci flow

[yanganto]

• CI will use MSRV from Rust manifest (Cargo.toml)
• Reduce usage of actions-rs, which are no longer maintained after Oct 13, 2023.
• Set up dependency upgrade bot
• Refactor lint, test, build with Nix
• Allow codecov upload failure

[yanganto]

@dr-orlovsky
Please be kindly reminded.
This PR will go into v0.11 not master, and solve the trivial CI issues.

[dr-orlovsky]

I have zero experience with nix and have a hard time reviewing this change.

Overall I am not happy switching from actions-rs to third-party DeterminateSystems. I have experience with such orgs being abandoned breaking CI all across everything.

Can we start with a trivial fix of MSRV and introduce nix a bit later once I finish with other urgent stuff related to v0.11 completion?

[yanganto]

Sure. I will open another quick-fix one if that is easier.

I will be pleased to introduce Nix to you, once you have time.
This Nix idea is the same as RGB.

In sta file of StrictType/RGB, we lock all the dependencies for VM by source code, not the version number

Dependency: 5teCJyjMWaxbQK8vdga2soWh2U7waERS3ev8KHShJcgv#trumpet-scratch-pelican
Dependency: 7EpM1uymEteG4g4xmF5ntKYX1wXXpbQj5iLiDreH4jWa#protein-donald-cool
Dependency: C5EbVby4rry8esHwoPrGUfhe7yqJgoyA1DBcyuBRf2rZ#granite-target-table
Dependency: DVtm25LRKU4TjbyZmVxPhvCmctZ6vKkPKqfpU2QsDNUo#exodus-axiom-tommy
Dependency: DzTvt9gGhPUKD8Dkkjk9PDBhkJ4gtWxXWQjxnmUYLNrs#voyage-kimono-disco
Dependency: HX2UBak8vPsTokug1DGMDvTpzns3xUdwZ7QJdyt4qBA9#speed-atlanta-trilogy
Checksum-SHA256: a3022416091ac3a073678a3245cd0fb4b7d26084b73dca970dd29fa34bf1f9eb

And what Nix doing is locking every dependency also by source code for the dev shell or the build system

zy9k80zhhd7alsd5g1wi206rdsb5mpz7-sudo-1.9.13p3.tar.gz.drv
zy9swy7w7w1k386smnwsqbmxg8d9cijm-perl5.36.0-TimeDate-2.33
zya2yh6ggs74r28ncnmnq8pgqzxr50sy-virtualenv-20.15.1.tar.gz.drv
zyaagsqqnpkbdfdzhmdkb0bk30d2p5y7-xmltex-0.8-tex.drv
zyax4jj64n8jk6r53mc23ld4211vqks4-npth-1.6.drv
zyb78qgwahs9497m9qg08ya4hg812znv-vo-aacenc-0.1.3

And these are pined by https://github.com/nixos/nixpkgs and we use flake.lock to pin the commit of nixpkgs in used.

Then, we use nix develop .#{shell-name} -c command-as-usual to run the command under a reproducible nix shell, I use msrv for the shell name here, and the exact version of Rust will read from the Rust manifest file Cargo.toml.
Also, it is easy to fall back by removing nix develop .#{shell-name} -c to use everything provided by ubuntu, not nix.

I am happy to maintain the Nix CI system here because I use NixOS, so no matter whether this is merged or not. I still need a Nix dev shell on my local, or else I can not develop this project. (The flake.nix needs to be git added whenever developing, it keeps a developer from using the latest one, but I need to be remove it in every commit not to push it)
If all good, I hope this can merge, and it will not making burden for other developers, people still can use their old way.

[dr-orlovsky]

Few questions:

• do we need something similar for other CI tasks?
• how we are sure in DeterminateSystems continuing support for its GitHub actions?

[yanganto]

actions-rs was marked as archived by an administrator on Oct 13, 2023. It is no longer maintained. They clearly announced, so we should find another solution.

Nix is a good solution to pin the Rust. No matter DeterminateSystems is continuing support, we have a lot of choices. We need a Nix daemon and use an existing tool to install it, DeterminateSystems is not the only provider.

There is more than one GitHub action that can install the Nix daemon
We can use

• https://github.com/cachix/install-nix-action and get the same result.
• Or we can start with the script sh <(curl -L https://nixos.org/nix/install) from any Linux.

The Nix system will build things from scratch or download them from the cache.
They try to make a cache in some place so it will be quicker, and we always use a script and remove the cache provided by them and the CI job still works.

Also, there are other cache services I have used before, we can use them to avoid using a young github action project.
https://garnix.io/
This cache will need the project admin to enable the Github Apps integration with them.

[yanganto]

We can remove DeterminateSystems if this will be much easier to maintain in the future. I do not do a deep survey on DeterminateSystems, because it is just one of Nix daemon installers I can use, and DeterminateSystems is not required.

[dr-orlovsky]

I am failing to understand: do we have all of checks running on NIX - or just toolchains? Can we have all cargo-based checks migrated to NIX?

[yanganto]

Yeah, other actions are updated, and this PR is not MSRV anymore.
By the way, It allows failure to upload code coverage, we can set it back when the token is ready.

[dr-orlovsky]

ACK b94bc06