Skip to content

feat: Detect and fail on unrecognised envelope flags #2453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
aborgna-q merged 2 commits into from
Jul 18, 2025
Merged

feat: Detect and fail on unrecognised envelope flags #2453

aborgna-q merged 2 commits into from
Jul 18, 2025

Conversation

@aborgna-q
Copy link
Collaborator

@aborgna-q aborgna-q commented Jul 18, 2025

When reading an envelope header flags, we ignored any unrecognised bits.

This change adds a new error to both the python and rust decoders, that ensures we detect and report this unexpected flags (nudging the user to update their HUGR version).

This will let us add new config flags in the future that change the payload format, having older versions gracefully fail to read them.

@aborgna-q aborgna-q requested a review from a team as a code owner July 18, 2025 10:51
@aborgna-q aborgna-q requested a review from doug-q July 18, 2025 10:51
@codecov
Copy link

codecov bot commented Jul 18, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 85.71429% with 3 lines in your changes missing coverage. Please review.

Project coverage is 82.27%. Comparing base (3943499) to head (cb9dff6).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
hugr-py/src/hugr/envelope.py 70.00% 3 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2453      +/-   ##
==========================================
+ Coverage   82.03%   82.27%   +0.23%     
==========================================
  Files         247      247              
  Lines       46014    46044      +30     
  Branches    41608    41616       +8     
==========================================
+ Hits        37749    37884     +135     
+ Misses       6246     6141     -105     
  Partials     2019     2019
Flag Coverage Δ
python 87.69% <70.00%> (+2.35%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aborgna-q aborgna-q requested review from lmondada and removed request for doug-q July 18, 2025 16:04
lmondada
lmondada approved these changes Jul 18, 2025
View reviewed changes
hugr-core/src/envelope.rs
/// The specified payload format is not supported.
#[error(
"The envelope configuration has unknown {}. Please update your HUGR version.",
if flag_ids.len() == 1 {format!("flag #{}", flag_ids[0])} else {format!("flags {}", flag_ids.iter().join(", "))}
Copy link
Contributor

@lmondada lmondada Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: you're using # in the singular case. You can use it too in the plural by changing the join argument to ", #" as well as adding one to the string

hugr-core/src/envelope/header.rs
let mut flags = 0b01000000u8;
flags |= u8::from(self.zstd);
let mut flags = DEFAULT_FLAGS;
if self.zstd {
Copy link
Contributor

@lmondada lmondada Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks that's much clearer!

hugr-core/src/envelope/header.rs
let zstd = flags & ZSTD_FLAG != 0;

// Check if there's any unrecognized flags.
let other_flags = (flags ^ DEFAULT_FLAGS) & !ZSTD_FLAG;
Copy link
Contributor

@lmondada lmondada Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd find it easier to read as

let other_flags = flags & !(DEFAULT_FLAGS | ZSTD_FLAG);

Copy link
Collaborator Author

@aborgna-q aborgna-q Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not quite the same. DEFAULT_FLAGS has an always true value, so if there's a zero there we have to complain too.
(I should make that more explicit)

hugr-py/src/hugr/envelope.py
MAGIC_NUMBERS = b"HUGRiHJv"

# The all-unset header flags configuration.
# Bit 7 is always set to ensure we have a printable ASCII character.
Copy link
Contributor

@lmondada lmondada Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you can't reuse the same consts as un Rust, can you maybe add a comment about which file their Rust counterparts are defined in?

@aborgna-q aborgna-q added this pull request to the merge queue Jul 18, 2025
Merged via the queue into main with commit 5e36770 Jul 18, 2025
26 checks passed
@aborgna-q aborgna-q deleted the ab/envelope-unknown-flags branch July 18, 2025 16:34
This was referenced Jul 18, 2025
Merged
Closed
Closed
Merged
github-merge-queue bot pushed a commit that referenced this pull request Jul 24, 2025
@hugrbot @aborgna-q
chore: release (#2476)
9025bd0 
This release fixes multiple inconsistencies between the serialization
formats
and improves the error messages when loading unsupported envelopes.

We now also support nodes with up to `2^32` connections to the same port
(up from `2^16`).

---

## 🤖 New release

* `hugr-model`: 0.21.0 -> 0.22.0 (✓ API compatible changes)
* `hugr-core`: 0.21.0 -> 0.22.0 (✓ API compatible changes)
* `hugr-llvm`: 0.21.0 -> 0.22.0 (✓ API compatible changes)
* `hugr-passes`: 0.21.0 -> 0.22.0 (✓ API compatible changes)
* `hugr-persistent`: 0.1.0 -> 0.2.0 (✓ API compatible changes)
* `hugr`: 0.21.0 -> 0.22.0 (✓ API compatible changes)
* `hugr-cli`: 0.21.0 -> 0.22.0 (✓ API compatible changes)

<details><summary><i><b>Changelog</b></i></summary><p>

## `hugr-model`

<blockquote>

##
[0.22.0](hugr-model-v0.21.0...hugr-model-v0.22.0)
- 2025-07-24

### New Features

- Names of private functions become `core.title` metadata.
([#2448](#2448))
- include generator metatada in model import and cli validate errors
([#2452](#2452))
- Version number in hugr binary format.
([#2468](#2468))
- Use semver crate for -model version, and include in docs
([#2471](#2471))
</blockquote>

## `hugr-core`

<blockquote>

##
[0.22.0](hugr-core-v0.21.0...hugr-core-v0.22.0)
- 2025-07-24

### Bug Fixes

- Ensure SumTypes have the same json encoding in -rs and -py
([#2465](#2465))

### New Features

- Export entrypoint metadata in Python and fix bug in import
([#2434](#2434))
- Names of private functions become `core.title` metadata.
([#2448](#2448))
- [**breaking**] Use binary envelopes for operation lower_func encoding
([#2447](#2447))
- [**breaking**] Update portgraph dependency to 0.15
([#2455](#2455))
- Detect and fail on unrecognised envelope flags
([#2453](#2453))
- include generator metatada in model import and cli validate errors
([#2452](#2452))
- [**breaking**] Add `insert_region` to HugrMut
([#2463](#2463))
- Non-region entrypoints in `hugr-model`.
([#2467](#2467))
</blockquote>

## `hugr-llvm`

<blockquote>

##
[0.21.0](hugr-llvm-v0.20.2...hugr-llvm-v0.21.0)
- 2025-07-09

### New Features

- [**breaking**] No nested FuncDefns (or AliasDefns)
([#2256](#2256))
- [**breaking**] Split `TypeArg::Sequence` into tuples and lists.
([#2140](#2140))
- [**breaking**] More helpful error messages in model import
([#2272](#2272))
- [**breaking**] Merge `TypeParam` and `TypeArg` into one `Term` type in
Rust ([#2309](#2309))
- Add `MakeError` op ([#2377](#2377))
</blockquote>

## `hugr-passes`

<blockquote>

##
[0.22.0](hugr-passes-v0.21.0...hugr-passes-v0.22.0)
- 2025-07-24

### New Features

- ReplaceTypes allows linearizing inside Op replacements
([#2435](#2435))
- Add pass for DFG inlining
([#2460](#2460))
</blockquote>

## `hugr-persistent`

<blockquote>

##
[0.2.0](hugr-persistent-v0.1.0...hugr-persistent-v0.2.0)
- 2025-07-24

### New Features

- [**breaking**] Update portgraph dependency to 0.15
([#2455](#2455))
</blockquote>

## `hugr`

<blockquote>

##
[0.22.0](hugr-v0.21.0...hugr-v0.22.0)
- 2025-07-24

### Bug Fixes

- Ensure SumTypes have the same json encoding in -rs and -py
([#2465](#2465))

### New Features

- ReplaceTypes allows linearizing inside Op replacements
([#2435](#2435))
- Add pass for DFG inlining
([#2460](#2460))
- Export entrypoint metadata in Python and fix bug in import
([#2434](#2434))
- Names of private functions become `core.title` metadata.
([#2448](#2448))
- [**breaking**] Use binary envelopes for operation lower_func encoding
([#2447](#2447))
- [**breaking**] Update portgraph dependency to 0.15
([#2455](#2455))
- Detect and fail on unrecognised envelope flags
([#2453](#2453))
- include generator metatada in model import and cli validate errors
([#2452](#2452))
- [**breaking**] Add `insert_region` to HugrMut
([#2463](#2463))
- Non-region entrypoints in `hugr-model`.
([#2467](#2467))
</blockquote>

## `hugr-cli`

<blockquote>

##
[0.22.0](hugr-cli-v0.21.0...hugr-cli-v0.22.0)
- 2025-07-24

### New Features

- include generator metatada in model import and cli validate errors
([#2452](#2452))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

---------

Co-authored-by: Agustín Borgna <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Jul 25, 2025
@hugrbot @aborgna-q
chore(main): release hugr-py 0.13.0rc1 (#2302)
2ac643d 
🤖 I have created a release *beep* *boop*
---


##
[0.13.0rc1](hugr-py-v0.12.2...hugr-py-v0.13.0rc1)
(2025-07-24)


### ⚠ BREAKING CHANGES

* Lowering functions in extension operations are now encoded as binary
envelopes. Older hugr versions will error out when trying to load them.
* **py:** `EnvelopeConfig::BINARY` now uses the model binary encoding.
`EnvelopeFormat.MODULE` is now `EnvelopeFormat.MODEL`.
`EnvelopeFormat.MODULE_WITH_EXTS` is now
`EnvelopeFormat.MODEL_WITH_EXTS`
* hugr-model: Symbol has an extra field
* Renamed the `Any` type bound to `Linear`
* The model CFG signature types were changed.
* Added `TypeParam`s and `TypeArg`s corresponding to floats and bytes.
* `TypeArg::Sequence` needs to be replaced with `TypeArg::List` or
`TypeArg::Tuple`
* FuncDefns must be moved to beneath Module.
`Container::define_function` is gone, use
`HugrBuilder::module_root_builder`; similarly in hugr-py
`DefinitionBuilder` (`define_function` ->
`module_root_builder().define_function`). In hugr-llvm, some uses of

### Features

* Add `BorrowArray` extension
([#2395](#2395))
([782687e](782687e))
* Add `MakeError` op ([#2377](#2377))
([909a794](909a794)),
closes [#1863](#1863)
* add toposort to HUGR-py
([#2367](#2367))
([34eed34](34eed34))
* Add Visibility to FuncDefn/FuncDecl.
([#2143](#2143))
([5bbe0cd](5bbe0cd))
* Added float and bytes literal to core and python bindings.
([#2289](#2289))
([e9c5e91](e9c5e91))
* **core, llvm:** add array unpack operations
([#2339](#2339))
([a1a70f1](a1a70f1)),
closes [#1947](#1947)
* Detect and fail on unrecognised envelope flags
([#2453](#2453))
([5e36770](5e36770))
* Export entrypoint metadata in Python and fix bug in import
([#2434](#2434))
([d17b245](d17b245))
* Expose `BorrowArray` in `hugr-py`
([#2425](#2425))
([fdb675f](fdb675f)),
closes [#2406](#2406)
* include generator metatada in model import and cli validate errors
([#2452](#2452))
([f7cedb4](f7cedb4))
* Names of private functions become `core.title` metadata.
([#2448](#2448))
([4bc7f65](4bc7f65))
* No nested FuncDefns (or AliasDefns)
([#2256](#2256))
([214b8df](214b8df))
* Non-region entrypoints in `hugr-model`.
([#2467](#2467))
([7b42da6](7b42da6))
* Open lists and tuples in `Term`
([#2360](#2360))
([292af80](292af80))
* **py:** enable Model as default BINARY envelope format
([#2317](#2317))
([f089931](f089931))
* **py:** Helper methods to get the neighbours of a node
([#2370](#2370))
([bb6fa50](bb6fa50))
* **py:** Use SumValue serialization for tuples
([#2466](#2466))
([f615037](f615037))
* Rename 'Any' type bound to 'Linear'
([#2421](#2421))
([c2f8b30](c2f8b30))
* Split `TypeArg::Sequence` into tuples and lists.
([#2140](#2140))
([cc4997f](cc4997f))
* Standarize the string formating of sum types and values
([#2432](#2432))
([ec207e7](ec207e7))
* Use binary envelopes for operation lower_func encoding
([#2447](#2447))
([2c16a77](2c16a77))


### Bug Fixes

* Ensure SumTypes have the same json encoding in -rs and -py
([#2465](#2465))
([7f97e6f](7f97e6f))
* Escape html-like labels in DotRenderer
([#2383](#2383))
([eaa7dfe](eaa7dfe))
* Export metadata in Python
([#2342](#2342))
([7be52db](7be52db))
* Fix model export of `Opaque` types.
([#2446](#2446))
([3943499](3943499))
* Fixed bug in python model export name mangling.
([#2323](#2323))
([041342f](041342f))
* Fixed bugs in model CFG handling and improved CFG signatures
([#2334](#2334))
([ccd2eb2](ccd2eb2))
* Fixed export of `Call` and `LoadConst` nodes in `hugr-py`.
([#2429](#2429))
([6a0e270](6a0e270))
* Fixed invalid extension name in test.
([#2319](#2319))
([c58ddbf](c58ddbf))
* Fixed two bugs in import/export of function operations
([#2324](#2324))
([1ad450f](1ad450f))
* map IntValue to unsigned repr when serializing
([#2413](#2413))
([26d426e](26d426e)),
closes [#2409](#2409)
* Order hints on input and output nodes.
([#2422](#2422))
([a31ccbc](a31ccbc))
* **py:** correct ConstString JSON encoding
([#2325](#2325))
([9649a48](9649a48))
* StaticArrayVal payload encoding, improve roundtrip checker
([#2444](#2444))
([1a301eb](1a301eb))
* stringify metadata before escaping in renderer
([#2405](#2405))
([8d67420](8d67420))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Release-As: 0.13.0rc1

---------

Co-authored-by: Agustín Borgna <[email protected]>
@hugrbot hugrbot mentioned this pull request Jul 28, 2025
Merged
github-merge-queue bot pushed a commit that referenced this pull request Jul 28, 2025
@hugrbot
chore: release (#2483)
fb5bcfe 
## 🤖 New release

* `hugr-model`: 0.22.0 -> 0.22.1
* `hugr-core`: 0.22.0 -> 0.22.1
* `hugr-llvm`: 0.22.0 -> 0.22.1
* `hugr-passes`: 0.22.0 -> 0.22.1 (✓ API compatible changes)
* `hugr`: 0.22.0 -> 0.22.1 (✓ API compatible changes)
* `hugr-cli`: 0.22.0 -> 0.22.1
* `hugr-persistent`: 0.2.0 -> 0.2.1

<details><summary><i><b>Changelog</b></i></summary><p>

## `hugr-model`

<blockquote>

##
[0.22.0](hugr-model-v0.21.0...hugr-model-v0.22.0)
- 2025-07-24

### New Features

- Names of private functions become `core.title` metadata.
([#2448](#2448))
- include generator metatada in model import and cli validate errors
([#2452](#2452))
- Version number in hugr binary format.
([#2468](#2468))
- Use semver crate for -model version, and include in docs
([#2471](#2471))
</blockquote>

## `hugr-core`

<blockquote>

##
[0.22.0](hugr-core-v0.21.0...hugr-core-v0.22.0)
- 2025-07-24

### Bug Fixes

- Ensure SumTypes have the same json encoding in -rs and -py
([#2465](#2465))

### New Features

- Export entrypoint metadata in Python and fix bug in import
([#2434](#2434))
- Names of private functions become `core.title` metadata.
([#2448](#2448))
- [**breaking**] Use binary envelopes for operation lower_func encoding
([#2447](#2447))
- [**breaking**] Update portgraph dependency to 0.15
([#2455](#2455))
- Detect and fail on unrecognised envelope flags
([#2453](#2453))
- include generator metatada in model import and cli validate errors
([#2452](#2452))
- [**breaking**] Add `insert_region` to HugrMut
([#2463](#2463))
- Non-region entrypoints in `hugr-model`.
([#2467](#2467))
</blockquote>

## `hugr-llvm`

<blockquote>

##
[0.21.0](hugr-llvm-v0.20.2...hugr-llvm-v0.21.0)
- 2025-07-09

### New Features

- [**breaking**] No nested FuncDefns (or AliasDefns)
([#2256](#2256))
- [**breaking**] Split `TypeArg::Sequence` into tuples and lists.
([#2140](#2140))
- [**breaking**] More helpful error messages in model import
([#2272](#2272))
- [**breaking**] Merge `TypeParam` and `TypeArg` into one `Term` type in
Rust ([#2309](#2309))
- Add `MakeError` op ([#2377](#2377))
</blockquote>

## `hugr-passes`

<blockquote>

##
[0.22.1](hugr-passes-v0.22.0...hugr-passes-v0.22.1)
- 2025-07-28

### New Features

- Include copy_discard_array in DelegatingLinearizer::default
([#2479](#2479))
- Inline calls to functions not on cycles in the call graph
([#2450](#2450))
</blockquote>

## `hugr`

<blockquote>

##
[0.22.1](hugr-v0.22.0...hugr-v0.22.1)
- 2025-07-28

### New Features

- Include copy_discard_array in DelegatingLinearizer::default
([#2479](#2479))
- Inline calls to functions not on cycles in the call graph
([#2450](#2450))
</blockquote>

## `hugr-cli`

<blockquote>

##
[0.22.0](hugr-cli-v0.21.0...hugr-cli-v0.22.0)
- 2025-07-24

### New Features

- include generator metatada in model import and cli validate errors
([#2452](#2452))
</blockquote>

## `hugr-persistent`

<blockquote>

##
[0.2.0](hugr-persistent-v0.1.0...hugr-persistent-v0.2.0)
- 2025-07-24

### New Features

- [**breaking**] Update portgraph dependency to 0.15
([#2455](#2455))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).
This was referenced Jul 29, 2025
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@lmondada lmondada lmondada approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@aborgna-q @lmondada