neither V1 or V2 or OCI manifest returned by registry

[anggutie-dev]

When using Artifact Registry we get this message:

neither V1 or V2 or OCI manifest returned by registry: manifest unknown: Manifest has media type \"application/vnd.oci.image.index.v1+json\" but client accepts [\"application/vnd.oci.image.manifest.v1+json\" \"application/vnd.docker.distribution.manifest.v1+prettyjws\" \"application/vnd.docker.distribution.manifest.v2+json\" \"application/vnd.docker.distribution.manifest.list.v2+json\"]"

I'm not sure if this is actually causing an issue or not since updater says it has 27 images considered. Does this affects the way updater works?

Right now if I push an image to GAR, updater doesn't detects there is a new image. I'm using the latest strategy so it should pick up the change, commit it to git and eventually update the image in K8s

This is what I can see in the logs (DEBUG mode)

time="2023-01-12T06:18:44Z" level=debug msg="Image 'us-central1-docker.pkg.dev/platform-dev-342301/development/frontend:568ecf2b6a3f9e51813ae90b76646c712ab5ac91' already on latest allowed version" alias=frontend application=frontend-dev image_name=platform-dev-342301/development/frontend image_tag=568ecf2b6a3f9e51813ae90b76646c712ab5ac91 registry=us-central1-docker.pkg.dev

This says that it is the latest version available but I have 3 newer versions!

I'm using the latest image updater version along with Argo CD 2.4.11+ version.

[Michael-Burke]

What do you get if you run docker manifest inspect <IMAGE> is there a manifest attached? Also are you using the allowed-tags function to filter the images?

Edit: also did you use an automated workflow to build the image, like with docker buildx through docker/setup-buildx-actions@v2? We noticed a new version of buildx was released (v0.10.0) which changed provenance settings and the OCI layout (https://github.com/docker/buildx/releases/tag/v0.10.0).

[anggutie-dev]

this is exactly the issue, I'm using buildx which just released 4 days ago a newer release. This is when everything started

My workaround for now was to use the previous release v0.9.x.

With buildx v0.10 the mediatype is set to:

and with buildx v0.9.x is set to:

Does updater supports this out of the box or do we need to configure buildx at the Github Actions side?

[gmemstr]

It seems that the updater doesn't yet support this specific mediaType -- I'm unsure what the specific changes between the two are besides the type string changing. For now I've locked our buildx version to 0.9.1 instead of latest, but I'm sure our devs would love to be using the latest release. It seems this functions specifically will need to be updated - the new mediaType is present in the opencontainers/image-spec package, which (I assume) means the distribution package also supports it.

[anggutie-dev]

@gmemstr yeah, I had to do pretty much the same for our pipelines. I guess we will have to wait until image updater fully support it.

[jannfis]

Related: #522

[ahasna]

@anggutie-dev @gmemstr I tried your approach in GitHub Actions. Unfortunately, with no luck. Both v0.9.1 and v0.10.0 ended up causing the same error in argocd-image-updater.

[jannfis]

We just released v0.12.2 which should fix this issue. If it doesn't, feel free to reopen, please.