Hash modulo meta by roberth · Pull Request #14686 · NixOS/nix

roberth · 2025-12-01T20:02:11Z

Motivation

meta is lost at the derivation level, but it doesn't need to be that way, thanks to quotient hashing aka hash modulo.

This change makes use of the existing "discrepancy" between derivation hashes and output hashes to allow more information to be stored in derivations without causing rebuilds.

The core of the change is fairly simple; see the hashDerivationModulo hunk.

Changes

Add system feature derivation-meta improving hash quotient logic to support storing meta
Adjust JSON formats to make meta a first class top level attr instead of the ATerm embedding

Notes

Must be opted in to by derivations/expressions, as usual, like ca derivations, structuredAttrs
Not supported in non-structuredAttrs. Didn't seem worth the added complexity. Users should move to structuredAttrs instead and afaik Nixpkgs is on track to do so.

Example

$ nix run github:roberth/nix/hash-modulo-meta -- \
  build --store ~/stores/tmp \
    --extra-experimental-features derivation-meta \
    --impure --expr \
    'with import (builtins.fetchTarball { url = "https://github.com/roberth/nixpkgs/archive/derivation-meta.tar.gz"; }) { config.derivationMeta = true; }; python3'
...
<progress bar, substituting from cache.nixos.org>
...

$ nix run github:roberth/nix/hash-modulo-meta -- derivation show --store ~/stores/tmp --extra-experimental-features derivation-meta --impure --expr 'with import (builtins.fetchTarball { url = "https://github.com/roberth/nixpkgs/archive/derivation-meta.tar.gz"; }) { config.derivationMeta = true; }; python3'
{
  "vcllzyc6d87wxlncjcwxlgy82lna4h4v-python3-3.13.9.drv": {
    "args": ...
    ...
    "meta": {
      "available": true,
      "broken": false,
      "changelog": "https://docs.python.org/release/3.13.9/whatsnew/changelog.html",
...
  }
}

Context

Step towards correctly implementing meta.timeout
Implements, closes Have our cake and eat it too derivation metadata #10780
Incremental change on topic of New derivation-making builtin #9774
Closes perhaps Provide SBOM information in an "queryable" way #13564
Possible solution to More context in context string #4677

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

edolstra · 2025-12-03T16:40:52Z

I didn't see this mentioned explicitly, but I assume __meta is not passed to the builder?

It's not entirely obvious how derivation meta data would be used in practice, especially since derivers are kind of useless (e.g. cache.nixos.org paths do have a deriver, but you have no way to get to them, so you wouldn't be able to get to the metadata). Perhaps in conjunction with #11749, the metadata could be propagated into the provenance records of store paths (which are already JSON).

roberth · 2025-12-03T19:58:43Z

I didn't see this mentioned explicitly, but I assume __meta is not passed to the builder?

Correct, and this is now also covered by a functional test.

not entirely obvious how derivation meta data would be used in practice

You can instantiate your stuff and inspect the metadata.
If you don't have the expression sources for what you've deployed, you probably should(!), but if you don't provenance could connect that back, either by eval reproduction, by reference or by value.

eval reproduction: provenance has a pinned flakeref, and you can instantiate that to acquire derivations
by reference: maybe you actually do have derivations in a store
by value: meta could be duplicated into the provenance structures, and it will benefit from the infrastructure that's set up around this:
- having a reliable interface between Nix and the expression world (ie derivation { __meta })
  (meta attribute is not reliable because not the whole derivation closure occurs as packages that are seen by the CLI)
- that interface getting used, pkgs: Add config.derivationMeta nixpkgs#466932

So this is a good step towards making #11749 more effective.

Store path provenance tracking #11749 -> improvements for when expression sources are not immediately known
Hash modulo meta #14686 -> access the metadata for whole closures

roberth · 2026-03-23T09:05:59Z

Marking as draft.
CA should operate on the stripped derivation too. We shouldn't need distinct traces for this, because meta is not part of the build process.

Realisations use regular drv paths again #15322

roberth · 2026-03-26T23:49:57Z

Still fits the model.

                derivation resolution                            usages
       ┌─────────────────────────────────────┐  ┌────────────────────────────────────────────────────────────────────────┐  



                  ╔════════════╗
                  ║  drvAttrs  ║
                  ╚══════╤═════╝
                         ║
                         ║ instantiate
                         ▼  
              ╔════════════════════╗             ╔════════════════╗
              ║ drvPath derivation ╟┄┄┄┄┄┄┄┄┄┄┄┄▷║  ✨ SBOMs ✨   ║
              ╚══════════╤═════════╝ ⋱          ╚════════════════╝
                         ║              ⋱──(status quo)
                         ║ resolve meta    ⋱
                         ▼                    ↘  ╔════════════════╗   ┌────────────────┐
       ╔═════════════════════════════════════╗┄┄▷║ IA output hash ╟┄┄▷│ .narinfo query │
       ║                                     ║   ╚════════════════╝   └────────────────┘
       ║  derivation for IA substitution     ║
       ║            (and more)               ║   ╔════════════════╗   ┌──────────────────────────────────────────────────┐
       ╚═════════════════╤═══════════════════╝┄┄▷║ hash           ╟┄┄▷│ 💀 CA deep build trace query f.k.a. realisation  │
                         ║                       ╚════════════════╝   └──────────────────────────────────────────────────┘
                         ║ resolve inputs
                         ▼
       ╔═════════════════════════════════════╗   ╔════════════════╗   ┌────────────────────────────┐
       ║  derivation, fully resolved         ╟┄┄▷║ hash           ╟┄┄▷│ Shallow realisation query  │
       ╚═════════════════════════════════════╝   ╚════════════════╝   └────────────────────────────┘

could probably improve some labels, but there ya go

Ericson2314 · 2026-04-01T21:15:16Z

My view is that we can land it now (needs review but idea approved), but before it is stabilized, it should be rolled into a "v2 input addressing" feature, which will no longer involve hashDerivationModulo at all.

Ericson2314 · 2026-05-11T21:50:13Z

+    /* Validate derivation-meta usage */
+    if (drv->structuredAttrs.has_value()) {
+        auto & structuredAttrs = drv->structuredAttrs->structuredAttrs;
+        bool hasMeta = structuredAttrs.contains("__meta");
+        bool hasFeature = usesDerivationMeta(*drv);
+
+        if (hasMeta && !hasFeature) {
+            throw Error(
+                "derivation '%s' has '__meta' attribute but does not require 'derivation-meta' system feature",
+                worker.store.printStorePath(drvPath));
+        }
+
+        if (hasFeature) {
+            experimentalFeatureSettings.require(Xp::DerivationMeta);
+        }
+    }
+


I would not put this here but instead make it part of the derivation options parsing logic. Then we will fail harder, which is good.

Ericson2314 · 2026-05-11T21:50:38Z

+            // Only extract __meta if derivation-meta feature is used
+            if (hasDerivationMetaFeature(structuredAttrs)) {
+                return getObject(structuredAttrs.at("__meta"));
+            }


The error checking for __meta with the feature would go here.

Ericson2314 · 2026-05-11T21:53:18Z

+            if (nix::hasDerivationMetaFeature(structuredAttrs)) {
+                auto metaIt = structuredAttrs.find("__meta");
+                res["meta"] = metaIt->second;
+                structuredAttrs.erase(metaIt);


Hmm this is going to be a bit janky

this crates a "have we parsed the options yet" race condition / state of sorts. We are better off eagerly extracting it at ATerm parse time as we do with structured attrs.

Make the meta field optional in DerivationOptions JSON deserialization to maintain backward compatibility with JSON that predates the derivation-meta feature. Changes: - Use optionalValueAt instead of valueAt for meta field deserialization - Mark meta as optional in derivation-options-v1 schema - Add unit test for backward compatibility

When the derivation-meta experimental feature is enabled, extract the __meta field from structuredAttrs to a top-level meta field in the Derivation JSON format (used by `nix derivation show/add`). This makes the format consistent with DerivationOptions JSON and provides better visibility of metadata without it being buried in structuredAttrs. The implementation: - Creates derivationToJson() helper accepting ExperimentalFeatureSettings - Extracts __meta to top-level meta field when feature is enabled - Requires experimental feature when deserializing JSON with meta field - Reconstructs __meta in structuredAttrs during deserialization - Maintains backward compatibility with JSON lacking meta field Tests verify both serialization and deserialization with mocked experimental feature settings, and functional tests ensure the experimental feature requirement is enforced.

Thank you fricklerhandwerk! Co-authored-by: Valentin Gagarin <valentin@gagarin.work>

It used to defer this to consumers. By performing the transformation immediately upon ingestion we avoid some confusion about the exact meaning of e.g. structuredAttrs at potentially many points in the code. We can now trust that it only has the "true" structured attributes that go into the builder. This may make ill-formed derivations harder to handle, but that is a price worth paying for added clarity going forward. Now, invalid __meta usage (missing experimental feature, unsorted requiredSystemFeatures) is caught immediately when the derivation is parsed or instantiated. To make re-injection into requiredSystemFeatures deterministic, it must be sorted when derivation-meta is used in instantiation. This avoids storing the original array position, and removes unnecessary entropy from derivations. Meta moves from DerivationOptions to BasicDerivation, since it is a derivation-level concept, not a build option.

Old daemons and remote builders don't understand derivation-meta, and would produce hash mismatch errors when receiving and validating such derivations. A new worker protocol feature marks support for ingesting these new derivations. The client checks for this feature in writeDerivation (for daemon) and build-remote (for remote builders) before a hash mismatch would occur, so that it can report a more specific error message.

The 'waiting for another Nix process to finish fetching input' message appears nondeterministically, causing the diff comparison to fail.

roberth · 2026-05-13T20:58:49Z

Implemented the review. Thank you @Ericson2314
I've employed worker protocol features to improve the error message when the daemon is old. (was hash mismatch in output)
Had to include tests: filter 'waiting for' messages in nix-profile test #15848 for the moment.

roberth requested review from Ericson2314 and edolstra as code owners December 1, 2025 20:02

github-actions Bot added documentation with-tests Issues related to testing. PRs with tests have some priority store Issues and pull requests concerning the Nix store labels Dec 1, 2025

roberth mentioned this pull request Dec 1, 2025

pkgs: Add config.derivationMeta NixOS/nixpkgs#466932

Draft

13 tasks

roberth force-pushed the hash-modulo-meta branch 3 times, most recently from 846a9c9 to b2c772b Compare December 2, 2025 06:02

roberth force-pushed the hash-modulo-meta branch from 01d6b9f to 1806b86 Compare December 10, 2025 17:08

roberth mentioned this pull request Dec 25, 2025

More context in context string #4677

Open

roberth mentioned this pull request Feb 6, 2026

store: update build settings for every derivation #15125

Closed

roberth marked this pull request as draft March 23, 2026 09:06

roberth marked this pull request as ready for review March 27, 2026 00:09