Skip to content

Commit cb9db55

Browse files
authored
feat: let should_fail_with check that the failure reason contains the expected message (#5319)
# Description ## Problem Resolves #4786 ## Summary `#[test(should_fail_with = "message")]` will now check that "message" is a substring of the failure reason. I _think_ this is a backwards-compatible change. I thought about supporting regular expressions, as suggested in the related issue, but I didn't know how to signal that it's a regex or just "contains". I guess that could be done with another name, something like `should_with_with_regexp` 🤔 (in a separate PR, if really needed/wanted) ## Additional Context None ## Documentation Check one: - [ ] No documentation needed. - [x] Documentation included in this PR. - [ ] **[For Experimental Features]** Documentation to be submitted in a separate PR. # PR Checklist\* - [x] I have tested the changes locally. - [x] I have formatted the changes with [Prettier](https://prettier.io/) and/or `cargo fmt` on default settings.
1 parent 96ef87b commit cb9db55

4 files changed

Lines changed: 36 additions & 14 deletions

File tree

  • docs/docs/tooling
  • test_programs
    • noir_test_failure/should_fail_mismatch/src
    • noir_test_success/should_fail_with_matches/src
  • tooling/nargo/src/ops

docs/docs/tooling/testing.md

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ fn test_add() {
4242
}
4343
```
4444

45-
You can be more specific and make it fail with a specific reason by using `should_fail_with = "<the reason for failure>`:
45+
You can be more specific and make it fail with a specific reason by using `should_fail_with = "<the reason for failure>"`:
4646

4747
```rust
4848
fn main(african_swallow_avg_speed : Field) {
@@ -58,5 +58,22 @@ fn test_king_arthur() {
5858
fn test_bridgekeeper() {
5959
main(32);
6060
}
61-
6261
```
62+
63+
The string given to `should_fail_with` doesn't need to exactly match the failure reason, it just needs to be a substring of it:
64+
65+
```rust
66+
fn main(african_swallow_avg_speed : Field) {
67+
assert(african_swallow_avg_speed == 65, "What is the airspeed velocity of an unladen swallow");
68+
}
69+
70+
#[test]
71+
fn test_king_arthur() {
72+
main(65);
73+
}
74+
75+
#[test(should_fail_with = "airspeed velocity")]
76+
fn test_bridgekeeper() {
77+
main(32);
78+
}
79+
```

test_programs/noir_test_failure/should_fail_mismatch/src/main.nr

Lines changed: 6 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,9 @@
22
fn test_different_string() {
33
assert_eq(0, 1, "Different string");
44
}
5-
// The assert message has a space
6-
#[test(should_fail_with = "Not equal")]
7-
fn test_with_extra_space() {
8-
assert_eq(0, 1, "Not equal ");
9-
}
10-
// The assert message has a space
11-
#[test(should_fail_with = "Not equal")]
12-
fn test_runtime_mismatch() {
13-
// We use a pedersen commitment here so that the assertion failure is only known at runtime.
14-
assert_eq(std::hash::pedersen_commitment([27]).x, 0, "Not equal ");
15-
}
5+
6+
// The failure reason is a substring of the expected message, but it should be the other way around
7+
#[test(should_fail_with = "Definitely Not equal!")]
8+
fn test_wrong_expectation() {
9+
assert_eq(0, 1, "Not equal");
10+
}

test_programs/noir_test_success/should_fail_with_matches/src/main.nr

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,11 @@ fn test_should_fail_with_match() {
33
assert_eq(0, 1, "Not equal");
44
}
55

6+
#[test(should_fail_with = "Not equal")]
7+
fn test_should_fail_with_match_partial_match() {
8+
assert_eq(0, 1, "Definitely Not equal!");
9+
}
10+
611
#[test(should_fail)]
712
fn test_should_fail_without_match() {
813
assert_eq(0, 1);
@@ -48,6 +53,11 @@ unconstrained fn unconstrained_test_should_fail_with_match() {
4853
assert_eq(0, 1, "Not equal");
4954
}
5055

56+
#[test(should_fail_with = "Not equal")]
57+
unconstrained fn unconstrained_test_should_fail_with_match_partial_match() {
58+
assert_eq(0, 1, "Definitely Not equal!");
59+
}
60+
5161
#[test(should_fail)]
5262
unconstrained fn unconstrained_test_should_fail_without_match() {
5363
assert_eq(0, 1);

tooling/nargo/src/ops/test.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ fn check_expected_failure_message(
128128
};
129129

130130
let expected_failure_message_matches =
131-
matches!(&failed_assertion, Some(message) if message == expected_failure_message);
131+
matches!(&failed_assertion, Some(message) if message.contains(expected_failure_message));
132132
if expected_failure_message_matches {
133133
return TestStatus::Pass;
134134
}

0 commit comments

Comments
 (0)