[index] Add OCI index detection for Nydus alternatives

Implement index detection feature that automatically discovers Nydus alternative manifests within OCI index manifests, similar to what SOCI snashotter is able to do. This enables transparent fallback to optimized Nydus images when available while keeping the original index manifest OCI compliant as non-nydus client can simply pull the regular OCI image.
The feature is heavily based on the already existing ReferrerDetect feature which looks for nydus manifests using the referrer API. However, the referrer API is not supported by all registries and as it relies on changes happening outside the manifest being pulled it's a bit dangerous from a supply chain perspective. On the other hand, this new IndexDetect feature is supported everywhere and does not have the supply chain issue as everyting is packed in a single manifest. Building a manifest compatible with this feature can be easily done with the `--merge-platform` flag in `nydusify convert`.

Key changes:
- Add EnableIndexDetect config option to experimental features
- Implement index package with manifest finding logic. Very similar to referrer package. The detection is based on the `platform.os.feature` (nydus.remoteimage.v1) or the `artifactType` (application/vnd.nydus.image.manifest.v1+json) field in the index manifest
- Result is cached to avoid multiple API calls for the same digest
- Integrate index detection into filesystem and snapshot layers. The index detection is done before the referrer detection: if both exist, we prefer to use the index detection for the supply chain issues mentionned above.
- Add e2e test for index detection
- Add documentation for the new feature

Author: Fricounet

.github/workflows/k8s-e2e-run.yml

@@ -19,3 +19,9 @@ jobs:
     uses: ./.github/workflows/k8s-e2e.yml
     with:
       auth-type: kubeconf
+
+  index_detect:
+    uses: ./.github/workflows/k8s-e2e.yml
+    with:
+      auth-type: kubeconf
+      index-detect: true

.github/workflows/k8s-e2e.yml

@@ -6,6 +6,9 @@ on:
       auth-type:
         required: true
         type: string
+      index-detect:
+        required: false
+        type: boolean
 
 env:
   DOCKER_USER: testuser
@@ -28,7 +31,8 @@ jobs:
           cache-dependency-path: "go.sum"
       - name: Test
         run: |
-          AUTH_TYPE='${{ inputs.auth-type }}'
+          export AUTH_TYPE='${{ inputs.auth-type }}'
+          export INDEX_DETECT='${{ inputs.index-detect }}'
           ./tests/helpers/kind.sh
       - name: Dump logs
         if: failure()

config/config.go

@@ -120,6 +120,7 @@ const (
 type Experimental struct {
 	EnableStargz         bool        `toml:"enable_stargz"`
 	EnableReferrerDetect bool        `toml:"enable_referrer_detect"`
+	EnableIndexDetect    bool        `toml:"enable_index_detect"`
 	TarfsConfig          TarfsConfig `toml:"tarfs"`
 	EnableBackendSource  bool        `toml:"enable_backend_source"`
 }

config/config_test.go

@@ -30,6 +30,7 @@ func TestLoadSnapshotterTOMLConfig(t *testing.T) {
 		Experimental: Experimental{
 			EnableStargz:         false,
 			EnableReferrerDetect: false,
+			EnableIndexDetect:    false,
 		},
 		CleanupOnClose: false,
 		SystemControllerConfig: SystemControllerConfig{

docs/index_detection.md

@@ -0,0 +1,89 @@
+# Index Detection
+
+## Overview
+
+Index Detection is a feature that automatically discovers Nydus alternative manifests within OCI index manifests. This enables transparent fallback to optimized Nydus images when available while keeping the original index manifest OCI compliant, allowing non-Nydus clients to pull regular OCI images.
+
+## Motivation
+
+The Index Detection feature addresses several limitations of the existing Referrer API-based detection:
+
+- **Registry Support**: Not all registries support the Referrer API
+- **Supply Chain Security**: Referrer API relies on external changes to the manifest, creating potential supply chain risks
+- **Universal Compatibility**: Index detection works with all OCI-compliant registries
+
+Unlike referrer-based detection, Index Detection packages everything into a single manifest, eliminating supply chain concerns while maintaining universal registry support.
+
+## How It Works
+
+### Detection Process
+
+1. **Index Manifest Parsing**: When a manifest digest is encountered, the system fetches and parses the original OCI index manifest
+2. **Platform Matching**: The system finds the original manifest descriptor within the index
+3. **Nydus Alternative Search**: It searches for platform-compatible manifests that contain:
+   - `platform.os.features` containing `nydus.remoteimage.v1`, or
+   - `artifactType` set to `application/vnd.nydus.image.manifest.v1+json`
+4. **Validation**: The found manifest is validated to ensure it's a valid Nydus manifest with metadata layers
+5. **Caching**: Results are cached to avoid repeated API calls for the same digest
+
+Both `platform.os.features` and `artifactType` are looked at because index manifests built using `merge-platform` before acceleration-service: v0.2.19 or nydus: v2.3.5 will have `platform.os.features` configured while images built after will have `artifactType` configured.
+
+### Detection Priority
+
+When both Index Detection and Referrer Detection are available, Index Detection takes priority due to its superior supply chain security properties.
+
+## Configuration
+
+Index Detection is controlled by the `EnableIndexDetect` configuration option in the experimental features section:
+
+```toml
+[experimental]
+enable_index_detect = true
+```
+
+## Building Compatible Images
+
+To create images compatible with Index Detection, use the `nydusify convert` command with the `--merge-platform` flag:
+
+```bash
+nydusify convert --merge-platform <source-image> <target-image>
+```
+
+This creates an OCI index manifest containing both the original image and the Nydus alternative:
+
+```json
+{
+  "schemaVersion": 2,
+  "manifests": [
+    {
+      "mediaType": "application/vnd.oci.image.manifest.v1+json",
+      "digest": "sha256:a63dfddecc661e4ad05896418b2f3774022269c3bf5b7e01baaa6e851a3a4a23",
+      "size": 2320,
+      "platform": {
+        "architecture": "amd64",
+        "os": "linux"
+      }
+    },
+    {
+      "mediaType": "application/vnd.oci.image.manifest.v1+json",
+      "digest": "sha256:bb82dd8ee111bfe5fdf12145b6ed553da9c15de17f54b1f658d95ff26a65a01c",
+      "size": 3229,
+      "platform": {
+        "architecture": "amd64",
+        "os": "linux"
+      },
+      "artifactType": "application/vnd.nydus.image.manifest.v1+json"
+    }
+  ]
+}
+```
+
+## Comparison with Referrer Detection
+
+| Aspect | Index Detection | Referrer Detection |
+|--------|----------------|-------------------|
+| Registry Support | Universal | Limited |
+| Supply Chain Security | Secure (single manifest) | Potential risks (external refs) |
+| OCI Compliance | Full compliance | Requires Referrer API |
+| Cache Behavior | Immutable (digest-based) | May require invalidation |
+| Detection Priority | Higher | Lower |

misc/snapshotter/config.toml

@@ -3,7 +3,7 @@ version = 1
 root = "/var/lib/containerd/io.containerd.snapshotter.v1.nydus"
 # The snapshotter's GRPC server socket, containerd will connect to plugin on this socket
 address = "/run/containerd-nydus/containerd-nydus-grpc.sock"
-# The nydus daemon mode can be one of the following options: multiple, dedicated, shared, or none. 
+# The nydus daemon mode can be one of the following options: multiple, dedicated, shared, or none.
 # If `daemon_mode` option is not specified, the default value is multiple.
 daemon_mode = "dedicated"
 # Whether snapshotter should try to clean up resources when it is closed
@@ -27,7 +27,7 @@ pprof_address = ""
 nydusd_config = "/etc/nydus/nydusd-config.fusedev.json"
 nydusd_path = "/usr/local/bin/nydusd"
 nydusimage_path = "/usr/local/bin/nydus-image"
-# The fs driver can be one of the following options: fusedev, fscache, blockdev, proxy, or nodev. 
+# The fs driver can be one of the following options: fusedev, fscache, blockdev, proxy, or nodev.
 # If `fs_driver` option is not specified, the default value is fusedev.
 fs_driver = "fusedev"
 # How to process when daemon dies: "none", "restart" or "failover"
@@ -112,6 +112,10 @@ enable_stargz = false
 # The option enables trying to fetch the Nydus image associated with the OCI image and run it.
 # Also see https://github.com/opencontainers/distribution-spec/blob/main/spec.md#listing-referrers
 enable_referrer_detect = false
+# Whether to enable index detection support
+# The option enables trying to fetch the Nydus image present in the same OCI index as the original OCI image and run it.
+# Also see https://github.com/containerd/nydus-snapshotter/blob/main/docs/index-detection.md
+enable_index_detect = false
 # Whether to enable authentication support
 # The option enables nydus snapshot to provide backend information to nydusd.
 enable_backend_source = false
@@ -134,4 +138,4 @@ max_concurrent_proc = 0
 # - "image_block": generate a raw block disk image with tarfs for an image
 # - "layer_block_with_verity": generate a raw block disk image with tarfs for a layer with dm-verity info
 # - "image_block_with_verity": generate a raw block disk image with tarfs for an image with dm-verity info
-export_mode = ""
+export_mode = ""

pkg/converter/constant.go

@@ -7,9 +7,10 @@
 package converter
 
 const (
-	ManifestOSFeatureNydus   = "nydus.remoteimage.v1"
-	MediaTypeNydusBlob       = "application/vnd.oci.image.layer.nydus.blob.v1"
-	BootstrapFileNameInLayer = "image/image.boot"
+	ManifestOSFeatureNydus    = "nydus.remoteimage.v1"
+	ManifestArtifactTypeNydus = "application/vnd.nydus.image.manifest.v1+json"
+	MediaTypeNydusBlob        = "application/vnd.oci.image.layer.nydus.blob.v1"
+	BootstrapFileNameInLayer  = "image/image.boot"
 
 	ManifestNydusCache = "containerd.io/snapshot/nydus-cache"
 

pkg/filesystem/config.go

@@ -9,6 +9,7 @@ package filesystem
 
 import (
 	"github.com/containerd/nydus-snapshotter/pkg/cache"
+	"github.com/containerd/nydus-snapshotter/pkg/index"
 	"github.com/containerd/nydus-snapshotter/pkg/manager"
 	"github.com/containerd/nydus-snapshotter/pkg/referrer"
 	"github.com/containerd/nydus-snapshotter/pkg/signature"
@@ -60,6 +61,17 @@ func WithReferrerManager(rm *referrer.Manager) NewFSOpt {
 	}
 }
 
+func WithIndexManager(im *index.Manager) NewFSOpt {
+	return func(fs *Filesystem) error {
+		if im == nil {
+			return errors.New("index manager cannot be nil")
+		}
+
+		fs.indexMgr = im
+		return nil
+	}
+}
+
 func WithTarfsManager(tm *tarfs.Manager) NewFSOpt {
 	return func(fs *Filesystem) error {
 		if tm == nil {

pkg/filesystem/fs.go

@@ -15,22 +15,22 @@ import (
 	"os"
 	"path"
 
+	"github.com/containerd/containerd/v2/core/snapshots"
+	"github.com/containerd/containerd/v2/core/snapshots/storage"
 	snpkg "github.com/containerd/containerd/v2/pkg/snapshotters"
+	"github.com/containerd/log"
 	"github.com/mohae/deepcopy"
 	"github.com/opencontainers/go-digest"
 	"github.com/pkg/errors"
 	"golang.org/x/sync/errgroup"
 
-	"github.com/containerd/containerd/v2/core/snapshots"
-	"github.com/containerd/containerd/v2/core/snapshots/storage"
-	"github.com/containerd/log"
-
 	"github.com/containerd/nydus-snapshotter/config"
 	"github.com/containerd/nydus-snapshotter/config/daemonconfig"
 	"github.com/containerd/nydus-snapshotter/pkg/cache"
 	"github.com/containerd/nydus-snapshotter/pkg/daemon"
 	"github.com/containerd/nydus-snapshotter/pkg/daemon/types"
 	"github.com/containerd/nydus-snapshotter/pkg/errdefs"
+	"github.com/containerd/nydus-snapshotter/pkg/index"
 	"github.com/containerd/nydus-snapshotter/pkg/label"
 	"github.com/containerd/nydus-snapshotter/pkg/manager"
 	racache "github.com/containerd/nydus-snapshotter/pkg/rafs"
@@ -46,6 +46,7 @@ type Filesystem struct {
 	enabledManagers      map[string]*manager.Manager
 	cacheMgr             *cache.Manager
 	referrerMgr          *referrer.Manager
+	indexMgr             *index.Manager
 	stargzResolver       *stargz.Resolver
 	tarfsMgr             *tarfs.Manager
 	verifier             *signature.Verifier

pkg/filesystem/index_adaptor.go

@@ -0,0 +1,64 @@
+/*
+ * Copyright (c) 2025. Nydus Developers. All rights reserved.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+package filesystem
+
+import (
+	"context"
+	"fmt"
+
+	snpkg "github.com/containerd/containerd/v2/pkg/snapshotters"
+	"github.com/containerd/log"
+	"github.com/opencontainers/go-digest"
+	"github.com/pkg/errors"
+)
+
+func (fs *Filesystem) IndexDetectEnabled() bool {
+	return fs.indexMgr != nil
+}
+
+// CheckIndexAlternative attempts to find a nydus alternative manifest in the original OCI index manifest
+func (fs *Filesystem) CheckIndexAlternative(ctx context.Context, labels map[string]string) bool {
+	if !fs.IndexDetectEnabled() {
+		return false
+	}
+
+	ref, ok := labels[snpkg.TargetRefLabel]
+	if !ok {
+		return false
+	}
+
+	manifestDigest := digest.Digest(labels[snpkg.TargetManifestDigestLabel])
+	if manifestDigest.Validate() != nil {
+		return false
+	}
+
+	log.G(ctx).WithField("ref", ref).WithField("digest", manifestDigest.String()).Debug("attempting index-based nydus detection")
+	if _, err := fs.indexMgr.CheckIndexAlternative(ctx, ref, manifestDigest); err != nil {
+		return false
+	}
+
+	return true
+}
+
+// TryFetchMetadataFromIndex attempts to fetch metadata from the nydus index alternative
+func (fs *Filesystem) TryFetchMetadataFromIndex(ctx context.Context, labels map[string]string, metadataPath string) error {
+	ref, ok := labels[snpkg.TargetRefLabel]
+	if !ok {
+		return fmt.Errorf("empty label %s", snpkg.TargetRefLabel)
+	}
+
+	manifestDigest := digest.Digest(labels[snpkg.TargetManifestDigestLabel])
+	if err := manifestDigest.Validate(); err != nil {
+		return fmt.Errorf("invalid label %s=%s", snpkg.TargetManifestDigestLabel, manifestDigest)
+	}
+
+	if err := fs.indexMgr.TryFetchMetadata(ctx, ref, manifestDigest, metadataPath); err != nil {
+		return errors.Wrap(err, "try fetch metadata")
+	}
+
+	return nil
+}