Skip to content

Secureboot: Image signing verification enhancements#3989

Merged
lguohan merged 2 commits intosonic-net:masterfrom
bhouse-nexthop:secureboot
Oct 18, 2025
Merged

Secureboot: Image signing verification enhancements#3989
lguohan merged 2 commits intosonic-net:masterfrom
bhouse-nexthop:secureboot

Conversation

@bhouse-nexthop
Copy link
Contributor

@bhouse-nexthop bhouse-nexthop commented Jul 21, 2025

What I did

The current signature verification of sonic images assumes the DB Keys are all Root CAs. The secureboot standard says nothing about this, the DBKeys are explicitly trusted by signing them with the KEK, and that signing method does not follow the standard X.509 PKI architecture. Therefore the DB Key is not guaranteed to be a CA Root (aka not self-signed). It is possible the DB Key was created as an intermediate, but since it is explicitly trusted that is ok.

Fixes sonic-net/sonic-buildimage#23406

How I did it

This adds this explicit trust of the DB Key as a secondary signing verification if the original verification fails. It disables looking inside the pkcs7 container for any keys at all and assumes the key specified is the exact key for the signature.

How to verify it

Build secureboot image signed with a DB Key that is not self-signed, then run through the sonic-installer install with that image and see the verification succeeds.

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@bhouse-nexthop
Copy link
Contributor Author

@ycoheNvidia since you contributed the signing validation for sonic images, can you review this PR modification?

@ycoheNvidia
Copy link
Contributor

Adding @davidpil2002 @DavidZagury @Yarden-Z as well

@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

bradh352 added 2 commits July 23, 2025 08:02
The DB Key is not guaranteed to be a CARoot.  It is possible to have
the DB Key be an intermediate as it is trusted via the EFI signing
methods which do not use the normal PKI methodology.

This adds a secondary signing verification, the original plus the
above described method.

Signed-off-by: Brad House <bhouse@nexthop.ai>
Signed-off-by: Travis Brown <travisb@nexthop.ai>
@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tjchadaga tjchadaga requested a review from qiluo-msft July 30, 2025 15:46
@tjchadaga
Copy link
Contributor

@qiluo-msft - could you please help with the review?

@ycoheNvidia ycoheNvidia self-requested a review August 3, 2025 11:16
Copy link
Contributor

@ycoheNvidia ycoheNvidia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me

@DavidZagury DavidZagury requested a review from Yarden-Z September 9, 2025 08:49
@bhouse-nexthop
Copy link
Contributor Author

@Yarden-Z any comments?
@qiluo-msft any reason not to merge this?

@bhouse-nexthop
Copy link
Contributor Author

@hdwhdw can you possibly take a look at this?

@hdwhdw hdwhdw self-requested a review October 17, 2025 22:43
@lguohan
Copy link
Contributor

lguohan commented Oct 18, 2025

Here’s a review of the changes made to verify_image_sign_common.sh in the referenced pull request:

Summary of Changes

  • The image signature verification logic has been refactored to try two different verification methods using openssl cms -verify:

    • Method 1: Standard certificate chain verification (-CAfile), expecting the DB Key as a self-signed root and the image signed with an intermediate certificate.
    • Method 2: Direct verification assuming the DB key signed the image (-nointern -noverify -certfile), skipping root CA checks.
  • The code now loops over both methods, returning success if either verification approach works.

Detailed Feedback

Positives:

  • Robustness Improved: By verifying the image signature using two different approaches, the script is now more resilient to different signing setups, covering cases where the DB key is either a root CA or just a trusted signer.
  • Clear Documentation: The added comments clearly explain the rationale for the two verification modes, making the logic easier to understand and maintain.
  • Backward Compatibility: The fallback ensures that existing images signed with either method will be accepted, reducing risk of verification failures due to changing certificate hierarchies.

Suggestions:

  • Variable Naming: The loop variable variant is slightly misleading since it sometimes contains multiple flags. Consider naming it verify_opts or similar for clarity.
  • Error Handling: If both verification methods fail, the script proceeds to clean up and returns 1 as before, which is appropriate. However, you might want to log which verification method(s) failed for troubleshooting.
  • Quoting Variables: For safety, it’s good practice to quote variables in shell scripts, e.g. "${CMS_SIG_FILE}", "${DATA_FILE}", and "${EFI_CERTS_DIR}/cert.pem" to avoid issues if paths contain spaces.
  • Hardcoded Paths: The use of /tmp/efi_certs is unchanged. Ensure this path is appropriate for all environments where the script is used.

Example Logging Improvement:

LOG=$(openssl cms -verify $no_check_time -noout ${variant} "$EFI_CERTS_DIR/cert.pem" -binary -in "${CMS_SIG_FILE}" -content "${DATA_FILE}" -inform pem 2>&1 > /dev/null )
if [ $VALIDATION_RES -eq 0 ]; then
    RESULT="CMS Verified OK (${variant})"
    return 0
fi

Final Assessment

The change is well-justified and improves signature verification compatibility.
Minor improvements for safety and diagnostics could be considered, but the new logic is solid and clearly documented.

If you need a more formal approval or additional suggestions, let me know!

@lguohan lguohan merged commit 55b665b into sonic-net:master Oct 18, 2025
6 checks passed
lguohan pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Oct 27, 2025
…s and bugfixes) (#23405)

#### Why I did it

The current sonic secureboot implementation assumes all assets are signed with the DB Key.  There are roughly 3600 kernel modules that get signed in the process.

Organizations may have varying security policy requirements where any signing requests may require signoff by one or more parties, this may simply be infeasible for that many requests.

Given that the DB key may be a long-lived key, and only RSA 2048 is often used or available, it is a security best-practice to sign as few assets as possible with such a key.

This PR is fully backwards compatible with any pre-existing usages.  It introduces a new `rules/config` setting of `SECURE_UPGRADE_KERNEL_CAFILE` which is a path to a file for all trusted keys to embed into the kernel.  If that setting is not specified, it defaults to `SECURE_UPGRADE_SIGNING_CERT`.  There are no other infrastructure changes to support ephemeral keys for kernel module signing.  It is up to the organization to generate the ephemeral keys and pass them into their own signing scripts as required.

Dependent PR:
 * sonic-net/sonic-linux-kernel#499

Related PRs:
 * sonic-net/sonic-utilities#3989
 * #23732
 * #23733
 * #23734
 * #23735
 * #23736

Fixes #23406

##### Work item tracking

#### How I did it

Worked through production signing steps and hit issues when requiring signing approval with our HSM.  This was the most elegant solution that required the minimal changes.

This has been fully tested.

#### How to verify it

Since there are no provided production signing script examples this would require a vendor with an existing implementation to validate.  Nexthop has validated this internally.

#### Which release branch to backport (provide reason below if selected)


#### Tested branch (Please provide the tested image version)

master as of 20250721

#### Description for the changelog
Secureboot: Production signing enhancement to allow ephemeral kernel module keys

#### Link to config_db schema for YANG module changes
N/A

#### A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Brad House <bhouse@nexthop.ai>
FengPan-Frank pushed a commit to FengPan-Frank/sonic-buildimage that referenced this pull request Dec 4, 2025
…s and bugfixes) (sonic-net#23405)

#### Why I did it

The current sonic secureboot implementation assumes all assets are signed with the DB Key.  There are roughly 3600 kernel modules that get signed in the process.

Organizations may have varying security policy requirements where any signing requests may require signoff by one or more parties, this may simply be infeasible for that many requests.

Given that the DB key may be a long-lived key, and only RSA 2048 is often used or available, it is a security best-practice to sign as few assets as possible with such a key.

This PR is fully backwards compatible with any pre-existing usages.  It introduces a new `rules/config` setting of `SECURE_UPGRADE_KERNEL_CAFILE` which is a path to a file for all trusted keys to embed into the kernel.  If that setting is not specified, it defaults to `SECURE_UPGRADE_SIGNING_CERT`.  There are no other infrastructure changes to support ephemeral keys for kernel module signing.  It is up to the organization to generate the ephemeral keys and pass them into their own signing scripts as required.

Dependent PR:
 * sonic-net/sonic-linux-kernel#499

Related PRs:
 * sonic-net/sonic-utilities#3989
 * sonic-net#23732
 * sonic-net#23733
 * sonic-net#23734
 * sonic-net#23735
 * sonic-net#23736

Fixes sonic-net#23406

##### Work item tracking

#### How I did it

Worked through production signing steps and hit issues when requiring signing approval with our HSM.  This was the most elegant solution that required the minimal changes.

This has been fully tested.

#### How to verify it

Since there are no provided production signing script examples this would require a vendor with an existing implementation to validate.  Nexthop has validated this internally.

#### Which release branch to backport (provide reason below if selected)

#### Tested branch (Please provide the tested image version)

master as of 20250721

#### Description for the changelog
Secureboot: Production signing enhancement to allow ephemeral kernel module keys

#### Link to config_db schema for YANG module changes
N/A

#### A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Brad House <bhouse@nexthop.ai>
Signed-off-by: Feng Pan <fenpan@microsoft.com>
xwjiang-ms pushed a commit to xwjiang-ms/sonic-buildimage that referenced this pull request Dec 22, 2025
…s and bugfixes) (sonic-net#23405)

#### Why I did it

The current sonic secureboot implementation assumes all assets are signed with the DB Key.  There are roughly 3600 kernel modules that get signed in the process.

Organizations may have varying security policy requirements where any signing requests may require signoff by one or more parties, this may simply be infeasible for that many requests.

Given that the DB key may be a long-lived key, and only RSA 2048 is often used or available, it is a security best-practice to sign as few assets as possible with such a key.

This PR is fully backwards compatible with any pre-existing usages.  It introduces a new `rules/config` setting of `SECURE_UPGRADE_KERNEL_CAFILE` which is a path to a file for all trusted keys to embed into the kernel.  If that setting is not specified, it defaults to `SECURE_UPGRADE_SIGNING_CERT`.  There are no other infrastructure changes to support ephemeral keys for kernel module signing.  It is up to the organization to generate the ephemeral keys and pass them into their own signing scripts as required.

Dependent PR:
 * sonic-net/sonic-linux-kernel#499

Related PRs:
 * sonic-net/sonic-utilities#3989
 * sonic-net#23732
 * sonic-net#23733
 * sonic-net#23734
 * sonic-net#23735
 * sonic-net#23736

Fixes sonic-net#23406

##### Work item tracking

#### How I did it

Worked through production signing steps and hit issues when requiring signing approval with our HSM.  This was the most elegant solution that required the minimal changes.

This has been fully tested.

#### How to verify it

Since there are no provided production signing script examples this would require a vendor with an existing implementation to validate.  Nexthop has validated this internally.

#### Which release branch to backport (provide reason below if selected)

#### Tested branch (Please provide the tested image version)

master as of 20250721

#### Description for the changelog
Secureboot: Production signing enhancement to allow ephemeral kernel module keys

#### Link to config_db schema for YANG module changes
N/A

#### A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Brad House <bhouse@nexthop.ai>
Signed-off-by: xiaweijiang <xiaweijiang@microsoft.com>
YairRaviv pushed a commit to YairRaviv/sonic-utilities that referenced this pull request Jan 12, 2026
#### What I did

The current signature verification of sonic images assumes the DB Keys are all Root CAs.  The secureboot standard says nothing about this, the DBKeys are explicitly trusted by signing them with the KEK, and that signing method does not follow the standard X.509 PKI architecture.  Therefore the DB Key is not guaranteed to be a CA Root (aka not self-signed).  It is possible the DB Key was created as an intermediate, but since it is explicitly trusted that is ok.

Fixes sonic-net/sonic-buildimage#23406

#### How I did it

This adds this explicit trust of the DB Key as a secondary signing verification if the original verification fails.  It disables looking inside the pkcs7 container for any keys at all and assumes the key specified is the exact key for the signature.

#### How to verify it

Build secureboot image signed with a DB Key that is not self-signed, then run through the sonic-installer install with that image and see the verification succeeds.


Signed-off-by: Brad House <bhouse@nexthop.ai>
Signed-off-by: Travis Brown <travisb@nexthop.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhancement: Secureboot production signing of kernel modules using ephemeral keys + bug fixes

6 participants