Skip to content

fix(rootfs/qemu-static): use direct armhf exec probe instead of broken arch-test gate#9820

Open
iav wants to merge 3 commits into
armbian:mainfrom
iav:fix/native-armhf-detection
Open

fix(rootfs/qemu-static): use direct armhf exec probe instead of broken arch-test gate#9820
iav wants to merge 3 commits into
armbian:mainfrom
iav:fix/native-armhf-detection

Conversation

@iav

@iav iav commented May 14, 2026

Copy link
Copy Markdown
Contributor

Problem

prepare_host_binfmt_qemu_cross_arm64_host_armhf_target (added in 6755e9190, 2024-12-28, for Apple Silicon) gates on arch-test arm. On success → skip qemu-arm import; on failure → write a hand-rolled /usr/share/binfmts/qemu-arm magic with interpreter /usr/bin/qemu-arm-static and enable it.

This is broken on most modern aarch64 hosts:

  • arch-test arm probes ARMv5 EABI. Current toolchains and the kernel's COMPAT layer target armhf (EABI v5+); the two surfaces diverge.
  • On Ubuntu Noble 6.8 / Ampere Altra (Hetzner CAX), arch-test arm returns failure even with CONFIG_COMPAT=y and a fully-working COMPAT path. The gate then unconditionally routes every aarch64 host through the Apple-Silicon import branch — qemu-arm gets registered, mmdebstrap and chroot apt-get / dpkg --configure / customize_image run through qemu-user-static emulation, ~10× slower than native.
  • The unconditional update-binfmts --enable qemu-arm at function entry could re-enable a pre-existing registration even on hosts the operator deliberately wanted to keep without qemu.
  • Hand-rolling the descriptor overwrites a packaged /usr/share/binfmts/qemu-arm (resolute's qemu-user-binfmt ships /usr/bin/qemu-arm without -static) → enable fails → armhf builds break on resolute Apple-Silicon-like hosts.

Empirically: helios4 BUILD_MINIMAL=yes RELEASE=trixie PREFER_DOCKER=yes finishes in 1:07 min on Hetzner CAX21 when qemu-arm stays unregistered (mmdebstrap native via binfmt_elf), vs ~10× longer when emulation routes mmdebstrap.

Fix

Replace the single arch-test arm gate with a 4-tier detection that picks the safest path on each host:

  1. /usr/share/binfmts/qemu-arm exists (packaged by qemu-user-binfmt on resolute, or by qemu-user-static install): update-binfmts --enable qemu-arm — loads the packaged descriptor into the kernel. Preserves the packaged interpreter path. Hard-error if enable fails (descriptor present but broken — host needs operator attention).
  2. /proc/sys/fs/binfmt_misc/qemu-arm exists without descriptor: trust if enabled, hard-error if disabled (no descriptor to re-enable from).
  3. No registration: probe with a direct exec /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 --help (ld-linux-armhf is shipped by gcc-arm-linux-gnueabihf, an armbian aarch64 host build dep). Authoritative test of CONFIG_COMPAT functionality. If succeeds → kernel runs armhf natively → no qemu setup needed.
  4. Native probe fails (Apple Silicon / no COMPAT): hand-roll /usr/share/binfmts/qemu-arm magic with /usr/bin/qemu-arm-static, import, enable — the original behaviour, now only reached when actually needed.

Also removed the unconditional update-binfmts --enable qemu-arm "|| true" at function entry — its job is subsumed by step (1).

Empirical validation

  • Native aarch64 (CAX21 / Ampere Altra, Ubuntu Noble 6.8): probe succeeds, no qemu-arm registered, helios4 trixie minimal builds 0:57 (PREFER_DOCKER=no) / 1:07 (PREFER_DOCKER=yes), mmdebstrap native.
  • No-CONFIG_COMPAT kernel (custom helios64 build with opts_n+=("COMPAT" "COMPAT_VDSO" "ARM64_32BIT_EL0")): ld-linux-armhf.so.3 execs with Exec format error; probe correctly fails → step (4) fires.
  • Pre-existing qemu-arm registration (manually added /usr/share/binfmts/qemu-arm with /usr/bin/qemu-arm-static): step (1) early-returns, descriptor md5 unchanged before/after, kernel entry left enabled.

Test plan

  • Hetzner CAX21 Noble 6.8 — native aarch64 with COMPAT, mmdebstrap native, helios4 image OK.
  • helios64 no-COMPAT kernel — ld-linux-armhf exec → "Exec format error", step (4) reachable.
  • Manual qemu-arm registration → step (1) preserves descriptor + enabled state, no clobber.
  • Resolute aarch64 with qemu-user-binfmt package — covered by inspection (step 1 enables packaged descriptor). No resolute Apple Silicon hardware available; happy to defer to community verification.

Summary by CodeRabbit

  • Bug Fixes

    • More reliable ARM 32‑bit emulation setup: repaired stale registrations, unconditional repair path, faster failure on broken handlers, clearer messages and reinstall/retry guidance.
    • Improved detection and early exit when native or kernel support is usable; added runtime verification to avoid false positives.
  • Refactor

    • Deterministic multi‑path enablement flow that prefers native support or packaged descriptors and falls back to creating/importing descriptors.
    • Consolidated verification and a single fast-fail helper to reduce unnecessary reconfiguration.
  • Style

    • Expanded explanatory header and minor formatting cleanups in an extension script.

@iav iav requested a review from a team as a code owner May 14, 2026 00:19
@iav iav requested review from ColorfulRhino and schwar3kat and removed request for a team May 14, 2026 00:19
@coderabbitai

coderabbitai Bot commented May 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: a3abe001-c4cf-4a07-9dd7-251487782639

📥 Commits

Reviewing files that changed from the base of the PR and between c456ae4 and 31b375e.

📒 Files selected for processing (2)
  • extensions/arm64-compat-vdso.sh
  • lib/functions/rootfs/qemu-static.sh
✅ Files skipped from review due to trivial changes (1)
  • extensions/arm64-compat-vdso.sh

📝 Walkthrough

Walkthrough

Probe for native armhf execution on aarch64 hosts; if unavailable, prefer a packaged /usr/share/binfmts/qemu-arm, try re-enabling the kernel /proc entry, or write/import/enable a qemu-arm descriptor; verify runtime executability via arch-test when available.

Changes

QEMU ARMhf native capability probe and setup

Layer / File(s) Summary
Always invoke armhf setup on aarch64→arm
lib/functions/rootfs/qemu-static.sh
Unconditionally call the armhf setup path when host is aarch64 and target is arm, ensuring repair/probe logic runs even if kernel descriptor appears present.
Probe armhf dynamic loader for native support
lib/functions/rootfs/qemu-static.sh
Snapshot kernel qemu-arm state, optionally disable it, and run ld-linux-armhf.so.3 --help (configurable via PREFER_NATIVE_ARMHF) to detect native COMPAT; return early if native execution works.
Post-probe validation and arch-test fallback
lib/functions/rootfs/qemu-static.sh
If native probing is skipped/unavailable, validate an enabled kernel qemu-arm with arch-test arm; if loader not executable, use arch-test arm as a fallback to skip qemu setup.
Enable packaged qemu-arm descriptor
lib/functions/rootfs/qemu-static.sh
If /usr/share/binfmts/qemu-arm exists, import with update-binfmts, force kernel enable via /proc, and verify runtime execution; abort with reinstall/remove-and-retry instructions on failure.
Re-enable disabled kernel entry when descriptor absent
lib/functions/rootfs/qemu-static.sh
When descriptor is missing but kernel entry exists, write 1 to /proc/sys/fs/binfmt_misc/qemu-arm to re-enable it, then verify and validate execution; abort on persistent failure.
Write, import and enable qemu-arm descriptor
lib/functions/rootfs/qemu-static.sh
Fallback path writes /usr/share/binfmts/qemu-arm, update-binfmts --import and --enable it, optionally runs debug arch-test, then verifies execution and logs the cachehit message.
Verification helper for runtime executability
lib/functions/rootfs/qemu-static.sh
_verify_qemu_arm_executes() runs arch-test arm when available, skips verification if arch-test is missing, and fails fast with a targeted error if armhf execution under qemu does not work.
Header and small formatting edits in extension
extensions/arm64-compat-vdso.sh
Added descriptive header about 32-bit compat vDSO and made minor whitespace/redirect/grep formatting tweaks without behavioral changes.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

I prod ld-linux to see if it will run,
I nudge /proc and pkgfiles till qemu's begun,
If native answers, I quiet the shim,
Else I write or import until the checks stop grim,
A rabbit hops when cross-builds hum. 🐇

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely summarizes the main change: replacing arch-test-based gating with direct armhf execution probing for qemu-arm registration.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added size/medium PR with more then 50 and less then 250 lines 05 Milestone: Second quarter release Needs review Seeking for review Framework Framework components labels May 14, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
lib/functions/rootfs/qemu-static.sh (1)

232-233: 💤 Low value

Hardcode qemu-arm instead of inheriting wanted_arch from caller scope.

This function's name pins the target arch (armhf_target), and the only call site (line 187) enters when wanted_arch == "arm". Relying on the caller's loop variable here couples the helper to a particular invocation context without any guard — a future caller or refactor could silently produce qemu- with no arch suffix. Using the literal string also matches the heredoc above (line 222) which is already arm-specific.

♻️ Suggested change
-	run_host_command_logged update-binfmts --import "qemu-${wanted_arch}"
-	run_host_command_logged update-binfmts --enable "qemu-${wanted_arch}"
+	run_host_command_logged update-binfmts --import "qemu-arm"
+	run_host_command_logged update-binfmts --enable "qemu-arm"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/functions/rootfs/qemu-static.sh` around lines 232 - 233, The helper
function armhf_target currently calls run_host_command_logged with
"qemu-${wanted_arch}", inheriting wanted_arch from caller scope; change these
two calls to use the literal "qemu-arm" instead of interpolating wanted_arch so
the helper is self-contained and matches the arm-specific heredoc and intent.
Update both run_host_command_logged update-binfmts --import and --enable
invocations to use "qemu-arm" and ensure no other references inside armhf_target
rely on wanted_arch.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lib/functions/rootfs/qemu-static.sh`:
- Around line 232-233: The helper function armhf_target currently calls
run_host_command_logged with "qemu-${wanted_arch}", inheriting wanted_arch from
caller scope; change these two calls to use the literal "qemu-arm" instead of
interpolating wanted_arch so the helper is self-contained and matches the
arm-specific heredoc and intent. Update both run_host_command_logged
update-binfmts --import and --enable invocations to use "qemu-arm" and ensure no
other references inside armhf_target rely on wanted_arch.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0f4f347d-a3de-47de-91fe-fa6e149fa4d8

📥 Commits

Reviewing files that changed from the base of the PR and between 761d04e and 4a8209d.

📒 Files selected for processing (1)
  • lib/functions/rootfs/qemu-static.sh

@iav iav force-pushed the fix/native-armhf-detection branch 2 times, most recently from 934a2f0 to d5b76d4 Compare May 14, 2026 02:15

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
lib/functions/rootfs/qemu-static.sh (1)

218-226: 💤 Low value

Probe path is gated solely on the cross-toolchain loader, missing fallback for foreign architecture setups.

The probe at line 222 checks /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 (installed by gcc-arm-linux-gnueabihf). If this cross-toolchain package is absent—possible in pre-prepared hosts or manual foreign architecture setups—the probe fails silently and falls through to write a hand-rolled qemu-arm descriptor and force emulation, defeating this PR's optimization.

In normal builds, gcc-arm-linux-gnueabihf is installed during host preparation when the target includes armhf, so this is not a typical correctness issue. However, hosts with armhf added as a foreign dpkg architecture (without the cross-toolchain) could use the alternative loader path /lib/arm-linux-gnueabihf/ld-linux-armhf.so.3 to detect native capability without additional packages. Consider probing both paths to support this setup, or at minimum document the gcc-arm-linux-gnueabihf dependency at the call site so users debugging slow builds can identify whether they need to install it.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/functions/rootfs/qemu-static.sh` around lines 218 - 226, The armhf kernel
probe relies only on /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 via the
armhf_probe check and will miss hosts that have armhf as a foreign dpkg
architecture where the loader lives at
/lib/arm-linux-gnueabihf/ld-linux-armhf.so.3; update the probe logic used around
the armhf_probe variable to check both paths
("/usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3" and
"/lib/arm-linux-gnueabihf/ld-linux-armhf.so.3") and invoke the same executable
--help test for whichever exists, falling back to the existing qemu-arm
descriptor logic only if neither path yields an executable; alternatively add a
clear comment/log entry (via display_alert) documenting the
gcc-arm-linux-gnueabihf dependency if you prefer not to add the second-path
probe.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lib/functions/rootfs/qemu-static.sh`:
- Around line 218-226: The armhf kernel probe relies only on
/usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 via the armhf_probe check and
will miss hosts that have armhf as a foreign dpkg architecture where the loader
lives at /lib/arm-linux-gnueabihf/ld-linux-armhf.so.3; update the probe logic
used around the armhf_probe variable to check both paths
("/usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3" and
"/lib/arm-linux-gnueabihf/ld-linux-armhf.so.3") and invoke the same executable
--help test for whichever exists, falling back to the existing qemu-arm
descriptor logic only if neither path yields an executable; alternatively add a
clear comment/log entry (via display_alert) documenting the
gcc-arm-linux-gnueabihf dependency if you prefer not to add the second-path
probe.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 688ab1df-44ba-4174-9a39-f917a6a0ebf8

📥 Commits

Reviewing files that changed from the base of the PR and between 934a2f0 and d5b76d4.

📒 Files selected for processing (1)
  • lib/functions/rootfs/qemu-static.sh

@iav iav force-pushed the fix/native-armhf-detection branch from d5b76d4 to 4be6288 Compare May 14, 2026 02:49
@iav

iav commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

Re: nitpick "Probe path is gated solely on the cross-toolchain loader, missing fallback for foreign architecture setups."

Verified: gcc-arm-linux-gnueabihf is added to host_dependencies in lib/functions/host/prepare-host.sh:258-260 for any build with wanted_arch ∈ {armhf, all}, and install_host_dependencies runs (line 77 of prepare_host_noninteractive) before prepare_host_binfmt_qemu (line 98). The probe binary /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 is therefore guaranteed to be in place when this helper fires in the normal flow.

The "absent" case the nitpick raises applies only when the operator sets PRE_PREPARED_HOST=yes (or equivalent), which is an explicit opt-out from armbian's host-dep management — the admin is on the hook for installing the toolchain that matches their target arch. If we wanted to be helpful in that path, a clearer message would be a separate concern (a check around assert_prepared_host rather than in the binfmt helper). Leaving this PR scoped to the original gate-replacement.

@igorpecovnik

Copy link
Copy Markdown
Member

Should we merge this after 05 release?

@igorpecovnik igorpecovnik added 08 Milestone: Third quarter release and removed 05 Milestone: Second quarter release labels May 16, 2026
@iav

iav commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

I think it could be done earlier.
But of course I don't see the whole situation.

@igorpecovnik

Copy link
Copy Markdown
Member

Depends how well is this tested? I want as less surprises as possible when hitting the "build all" button.

@iav

iav commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

Indeed — my testing surface doesn't cover CI or the big builder fleet. Agree on holding the merge until builder hiccups can't threaten a release cut.

@igorpecovnik

Copy link
Copy Markdown
Member

Agree on holding the merge until builder hiccups can't threaten a release cut.

Yep. After release we can afford to have broken CI for days while before its really not needed :)

@iav iav force-pushed the fix/native-armhf-detection branch from 4be6288 to 53cb297 Compare May 20, 2026 00:32
@github-actions github-actions Bot added the 05 Milestone: Second quarter release label May 20, 2026
@iav iav force-pushed the fix/native-armhf-detection branch 4 times, most recently from 765d575 to 1aed440 Compare May 21, 2026 22:20

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
lib/functions/rootfs/qemu-static.sh (1)

261-274: 💤 Low value

Consider adding verification after hand-write path.

Unlike the packaged descriptor path (lines 239-244), there's no verification that qemu-arm actually became enabled in the kernel after import/enable. While set -e catches command failures, update-binfmts --enable could succeed without the kernel entry actually becoming enabled (e.g., if binfmt_misc is in an unexpected state).

♻️ Optional: Add verification for consistency
 	run_host_command_logged update-binfmts --import qemu-arm
 	run_host_command_logged update-binfmts --enable qemu-arm
+	if [[ ! -e /proc/sys/fs/binfmt_misc/qemu-arm ]] ||
+		[[ "$(head -n1 /proc/sys/fs/binfmt_misc/qemu-arm 2> /dev/null)" != "enabled" ]]; then
+		exit_with_error "Hand-written qemu-arm descriptor could not be enabled. Check binfmt_misc state."
+	fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/functions/rootfs/qemu-static.sh` around lines 261 - 274, The hand-written
qemu-arm descriptor path writes /usr/share/binfmts/qemu-arm and runs
update-binfmts --import/--enable but lacks post-enable verification; after
run_host_command_logged update-binfmts --enable qemu-arm add a check using
update-binfmts --display qemu-arm or by reading
/proc/sys/fs/binfmt_misc/qemu-arm (or /proc/sys/fs/binfmt_misc/status) to
confirm the entry is present and marked enabled, and if the check fails retry or
log an error and exit non-zero so the failure is not silently ignored (reference
the created file /usr/share/binfmts/qemu-arm and the commands update-binfmts
--import qemu-arm and update-binfmts --enable qemu-arm).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/functions/rootfs/qemu-static.sh`:
- Around line 276-279: The debug block is passing shell operators as literal
args to arch-test (run_host_command_logged arch-test "||" true); update the
if-body so the shell operator applies to the command exit instead of being
passed to arch-test—call run_host_command_logged arch-test and place || true
after that invocation (i.e., run_host_command_logged arch-test || true), keeping
the existing SHOW_DEBUG check and display_alert call unchanged.

---

Nitpick comments:
In `@lib/functions/rootfs/qemu-static.sh`:
- Around line 261-274: The hand-written qemu-arm descriptor path writes
/usr/share/binfmts/qemu-arm and runs update-binfmts --import/--enable but lacks
post-enable verification; after run_host_command_logged update-binfmts --enable
qemu-arm add a check using update-binfmts --display qemu-arm or by reading
/proc/sys/fs/binfmt_misc/qemu-arm (or /proc/sys/fs/binfmt_misc/status) to
confirm the entry is present and marked enabled, and if the check fails retry or
log an error and exit non-zero so the failure is not silently ignored (reference
the created file /usr/share/binfmts/qemu-arm and the commands update-binfmts
--import qemu-arm and update-binfmts --enable qemu-arm).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 23911c9e-9472-4915-8474-2947dc33f7b5

📥 Commits

Reviewing files that changed from the base of the PR and between 4be6288 and 1aed440.

📒 Files selected for processing (1)
  • lib/functions/rootfs/qemu-static.sh

Comment thread lib/functions/rootfs/qemu-static.sh
@iav iav closed this May 22, 2026
@iav iav deleted the fix/native-armhf-detection branch May 22, 2026 21:55
@iav iav restored the fix/native-armhf-detection branch May 22, 2026 21:56
@iav iav reopened this May 22, 2026
@iav iav force-pushed the fix/native-armhf-detection branch from 1aed440 to 1d6d472 Compare May 22, 2026 21:57
@iav iav force-pushed the fix/native-armhf-detection branch from 42eb08d to c456ae4 Compare May 25, 2026 00:53

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/functions/rootfs/qemu-static.sh`:
- Around line 209-218: In prepare_host_binfmt_qemu_cross_arm64_host_armhf_target
the current ordering runs "${armhf_probe}" --help while qemu-arm may still be
enabled, causing the probe to succeed via qemu and then disable qemu-arm
incorrectly; change the flow so you first disable or mask
/proc/sys/fs/binfmt_misc/qemu-arm (using the same mechanism that writes to
/proc/sys/fs/binfmt_misc/qemu-arm) before invoking armhf_probe, then run
"${armhf_probe}" --help; if the probe fails, restore the previous qemu-arm state
(undo the disable) and continue using qemu; use the existing display_alert calls
(e.g., display_alert "Disabling qemu-arm..." and the "Host kernel can run armhf
natively..." path) and the same return semantics so behavior remains consistent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 477e0c1c-20b7-4958-805f-61729a15aeb1

📥 Commits

Reviewing files that changed from the base of the PR and between 1d6d472 and 42eb08d.

📒 Files selected for processing (1)
  • lib/functions/rootfs/qemu-static.sh

Comment thread lib/functions/rootfs/qemu-static.sh Outdated
@iav iav marked this pull request as draft May 25, 2026 00:55

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
lib/functions/rootfs/qemu-static.sh (1)

209-218: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Disable qemu-arm before probing native armhf.

This still probes ld-linux-armhf.so.3 while /proc/sys/fs/binfmt_misc/qemu-arm may be enabled. In that state the probe can succeed via qemu, and then Line 214 disables qemu-arm and returns as if native COMPAT worked. On a no-COMPAT host that turns a working qemu path into a later failure. Save the old qemu-arm state, disable it before the probe, and restore it if the probe fails; also don’t swallow a failed toggle here.

🔧 Minimal fix sketch
+	local qemu_arm_was_enabled=0
+	if [[ "${prefer_native}" == "yes" ]] && [[ -x "${armhf_probe}" ]]; then
+		if [[ -e /proc/sys/fs/binfmt_misc/qemu-arm ]] &&
+			[[ "$(head -n1 /proc/sys/fs/binfmt_misc/qemu-arm 2> /dev/null)" == "enabled" ]]; then
+			qemu_arm_was_enabled=1
+			display_alert "Disabling qemu-arm to probe native COMPAT for armhf" "temporarily bypassing binfmt_misc" "info"
+			echo 0 > /proc/sys/fs/binfmt_misc/qemu-arm || exit_with_error "Failed to disable qemu-arm before native armhf probe"
+		fi
+
+		if "${armhf_probe}" --help > /dev/null 2>&1; then
+			display_alert "Host kernel can run armhf natively (CONFIG_COMPAT)" "no qemu-arm setup needed" "debug"
+			return 0
+		fi
+
+		if (( qemu_arm_was_enabled )); then
+			echo 1 > /proc/sys/fs/binfmt_misc/qemu-arm || exit_with_error "Failed to restore qemu-arm after native armhf probe"
+		fi
+	fi
-
-	if [[ "${prefer_native}" == "yes" ]] && [[ -x "${armhf_probe}" ]] &&
-		"${armhf_probe}" --help > /dev/null 2>&1; then
-		if [[ -e /proc/sys/fs/binfmt_misc/qemu-arm ]] &&
-			[[ "$(head -n1 /proc/sys/fs/binfmt_misc/qemu-arm 2> /dev/null)" == "enabled" ]]; then
-			display_alert "Disabling qemu-arm to use native COMPAT for armhf" "≈10× faster than qemu emulation" "info"
-			echo 0 > /proc/sys/fs/binfmt_misc/qemu-arm 2> /dev/null || true
-		else
-			display_alert "Host kernel can run armhf natively (CONFIG_COMPAT)" "no qemu-arm setup needed" "debug"
-		fi
-		return 0
-	fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/functions/rootfs/qemu-static.sh` around lines 209 - 218, The probe can
succeed via qemu-arm currently because you disable qemu-arm only after probing;
instead, save the current state of /proc/sys/fs/binfmt_misc/qemu-arm (e.g., read
its first line or existence), disable it before running the probe
"${armhf_probe}" --help, then if the probe succeeds treat COMPAT as available;
if the probe fails restore the saved qemu-arm state (and check the write
succeeded) and do not swallow any failure (propagate/log the error instead of
silently returning 0). Update the block that references prefer_native,
armhf_probe and /proc/sys/fs/binfmt_misc/qemu-arm so toggling qemu-arm is
reversible and failures to toggle are handled.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@lib/functions/rootfs/qemu-static.sh`:
- Around line 209-218: The probe can succeed via qemu-arm currently because you
disable qemu-arm only after probing; instead, save the current state of
/proc/sys/fs/binfmt_misc/qemu-arm (e.g., read its first line or existence),
disable it before running the probe "${armhf_probe}" --help, then if the probe
succeeds treat COMPAT as available; if the probe fails restore the saved
qemu-arm state (and check the write succeeded) and do not swallow any failure
(propagate/log the error instead of silently returning 0). Update the block that
references prefer_native, armhf_probe and /proc/sys/fs/binfmt_misc/qemu-arm so
toggling qemu-arm is reversible and failures to toggle are handled.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c892c189-2506-4ea0-afe1-c284fd476c58

📥 Commits

Reviewing files that changed from the base of the PR and between 42eb08d and c456ae4.

📒 Files selected for processing (1)
  • lib/functions/rootfs/qemu-static.sh

@iav iav force-pushed the fix/native-armhf-detection branch 2 times, most recently from 1c66025 to 12f0371 Compare May 25, 2026 02:12
@iav iav force-pushed the fix/native-armhf-detection branch from 752bb11 to dbd7005 Compare May 26, 2026 04:09
@iav iav force-pushed the fix/native-armhf-detection branch from dbd7005 to 67bfd9a Compare June 8, 2026 21:05

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
lib/functions/rootfs/qemu-static.sh (1)

182-188: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Call the arm64/arm special handler before the missing-file gate.

Line 182 still prevents prepare_host_binfmt_qemu_cross_arm64_host_armhf_target() from running whenever both /proc/sys/fs/binfmt_misc/qemu-arm and /usr/share/binfmts/qemu-arm already exist. That is exactly the normal packaged state on Debian/Ubuntu hosts, so the new native armhf probe and the disabled-entry/package recovery paths are skipped in the common case. Result: CONFIG_COMPAT hosts keep unnecessary qemu-arm, and a packaged-but-disabled registration is never repaired.

Suggested caller-side fix
-		if [[ ! -e "/proc/sys/fs/binfmt_misc/qemu-${wanted_arch}" || ! -e "/usr/share/binfmts/qemu-${wanted_arch}" ]]; then
-			display_alert "Updating binfmts" "update-binfmts --enable qemu-${wanted_arch}" "debug"
-
-			# special case: some arm64 machines cant' really run armhf binaries natively (Apple Silicon); check if that is the case and forcibly import and enable qemu-arm for them.
-			if [[ "${host_arch}" == "aarch64" && "${wanted_arch}" == "arm" ]]; then
-				prepare_host_binfmt_qemu_cross_arm64_host_armhf_target
-			else
+		if [[ "${host_arch}" == "aarch64" && "${wanted_arch}" == "arm" ]]; then
+			prepare_host_binfmt_qemu_cross_arm64_host_armhf_target
+		elif [[ ! -e "/proc/sys/fs/binfmt_misc/qemu-${wanted_arch}" || ! -e "/usr/share/binfmts/qemu-${wanted_arch}" ]]; then
+			display_alert "Updating binfmts" "update-binfmts --enable qemu-${wanted_arch}" "debug"
 				run_host_command_logged update-binfmts --enable "qemu-${wanted_arch}" "&>" "/dev/null" || display_alert "Failed to update binfmts" "update-binfmts --enable qemu-${wanted_arch}" "err" # log & continue on failure
-			fi
 		fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/functions/rootfs/qemu-static.sh` around lines 182 - 188, The check that
gates update-binfmts currently skips calling
prepare_host_binfmt_qemu_cross_arm64_host_armhf_target when both
/proc/sys/fs/binfmt_misc/qemu-${wanted_arch} and
/usr/share/binfmts/qemu-${wanted_arch} exist; move or add the special-case
invocation so that when host_arch == "aarch64" and wanted_arch == "arm" you call
prepare_host_binfmt_qemu_cross_arm64_host_armhf_target before (or irrespective
of) the existence checks, ensuring the ARM64→ARMHF probe and recovery run even
on systems with packaged qemu-arm entries.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@lib/functions/rootfs/qemu-static.sh`:
- Around line 182-188: The check that gates update-binfmts currently skips
calling prepare_host_binfmt_qemu_cross_arm64_host_armhf_target when both
/proc/sys/fs/binfmt_misc/qemu-${wanted_arch} and
/usr/share/binfmts/qemu-${wanted_arch} exist; move or add the special-case
invocation so that when host_arch == "aarch64" and wanted_arch == "arm" you call
prepare_host_binfmt_qemu_cross_arm64_host_armhf_target before (or irrespective
of) the existence checks, ensuring the ARM64→ARMHF probe and recovery run even
on systems with packaged qemu-arm entries.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3479c567-6f5d-4ddc-90ad-de23483117b3

📥 Commits

Reviewing files that changed from the base of the PR and between c456ae4 and 67bfd9a.

📒 Files selected for processing (2)
  • extensions/arm64-compat-vdso.sh
  • lib/functions/rootfs/qemu-static.sh
✅ Files skipped from review due to trivial changes (1)
  • extensions/arm64-compat-vdso.sh

iav added 3 commits June 9, 2026 00:38
…n arch-test gate

`arch-test arm` probes ARMv5 EABI, divergent from kernel COMPAT (armhf
EABI v5+), and returns failure on modern aarch64 hosts (Ubuntu Noble
6.8 / Ampere Altra) with a fully working CONFIG_COMPAT path — routing
mmdebstrap through qemu-user emulation at ~10× slowdown. Replace with
a 4-tier detection: prefer the packaged `/usr/share/binfmts/qemu-arm`
descriptor if present, trust an already-enabled kernel registration,
otherwise probe COMPAT directly by exec'ing `ld-linux-armhf.so.3`,
falling back to the hand-rolled descriptor only when none applies.

Assisted-by: Claude:claude-opus-4.7
…registration

When CONFIG_COMPAT runs armhf binaries natively on aarch64 (≈10× faster
than qemu-user emulation), but qemu-arm is registered in kernel binfmt_misc,
the previous "trust existing setup" check returned early and the build
went through qemu anyway. Empirically: helios4 build 41 min via qemu vs
19 min native COMPAT on droid (RK3399).

Reorder probes:

1. Snapshot initial qemu-arm state.
2. COMPAT probe must run with qemu-arm OFF — kernel binfmt_misc routes
   armhf exec through qemu first when enabled, making the probe a liar
   about native capability. Temp-disable before probe; on probe failure
   (no COMPAT, e.g. Apple Silicon under HVF where AArch32@EL0 SIGILL's)
   restore qemu-arm and fall through to qemu setup. Toggle errors are
   not swallowed.
3. Trust-existing path validates via arch-test arm — the `enabled` flag
   alone does not attest that the interpreter actually runs (stale path,
   removed package); without this, broken registrations passed setup
   and failed later in chroot.
4. _verify_qemu_arm_executes helper called after each successful
   import/enable/repair branch, so qemu issues fail fast at host-prepare
   time instead of mid-rootfs-build.

The descriptor itself is left intact — only the active flag is toggled.

Gated by ARMBIAN_PREFER_NATIVE_ARMHF=no for explicit opt-out. Behaviour
preserved on hosts without COMPAT.

Assisted-by: Claude:claude-opus-4.7
…uilds

Spell out the practical effect of CONFIG_COMPAT + COMPAT_VDSO + ARM64_32BIT_EL0:
an arm64 host running such a kernel can execute armhf userspace natively, so
rootfs/chroot steps run native instead of via qemu-user-static (~10× faster).
Cross-reference the rootfs probe and the ARMBIAN_PREFER_NATIVE_ARMHF switch,
and note the Apple-M-series exception (no AArch32@EL0 in silicon).

Assisted-by: Claude:claude-opus-4.7
@iav iav force-pushed the fix/native-armhf-detection branch from 67bfd9a to 31b375e Compare June 8, 2026 21:39
@iav

iav commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Re: caller gates prepare_host_binfmt_qemu_cross_arm64_host_armhf_target behind the ! -e … existence check

Good catch. The aarch64+arm special case now runs unconditionally — moved out of the ! -e … existence gate into an if … elif. On hosts where qemu-arm is already registered (the common packaged case) the handler still runs, so the native-COMPAT path is no longer skipped and a stale registration gets repaired. The handler is idempotent across every binfmt_misc state, so re-entry on an already-configured host is a no-op.

Fixed in 31b375e.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/functions/rootfs/qemu-static.sh`:
- Line 190: The command passes the literal strings "&>" and "/dev/null" into
update-binfmts via run_host_command_logged which prevents proper shell
redirection; remove those quoted redirection arguments from the
run_host_command_logged call and let run_host_command_logged handle
output/logging (or, if you must swallow output, perform the redirection outside
the function call), e.g. call run_host_command_logged with the command
"update-binfmts --enable qemu-${wanted_arch}" and keep the failure handler
display_alert "Failed to update binfmts" "update-binfmts --enable
qemu-${wanted_arch}" "err" after the call (or append || display_alert) so
update-binfmts does not receive malformed arguments.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 423aec3b-dbb2-476c-9fea-ef81ae7381df

📥 Commits

Reviewing files that changed from the base of the PR and between c456ae4 and 31b375e.

📒 Files selected for processing (2)
  • extensions/arm64-compat-vdso.sh
  • lib/functions/rootfs/qemu-static.sh
✅ Files skipped from review due to trivial changes (1)
  • extensions/arm64-compat-vdso.sh

Comment thread lib/functions/rootfs/qemu-static.sh
@iav iav marked this pull request as ready for review June 8, 2026 22:15
@iav iav requested a review from igorpecovnik as a code owner June 8, 2026 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

05 Milestone: Second quarter release 08 Milestone: Third quarter release Framework Framework components Needs review Seeking for review size/medium PR with more then 50 and less then 250 lines

Development

Successfully merging this pull request may close these issues.

2 participants