Skip to content

fix: replace NaN and Infinity with null in JSON reports#421

Merged
rpreen merged 5 commits intomainfrom
fix/invalid-json-tokens
Mar 16, 2026
Merged

fix: replace NaN and Infinity with null in JSON reports#421
rpreen merged 5 commits intomainfrom
fix/invalid-json-tokens

Conversation

@ssrhaso
Copy link
Copy Markdown
Contributor

@ssrhaso ssrhaso commented Mar 6, 2026

Closes #366

JSON reports produced by attacks could contain bare NaN, Infinity,
and -Infinity tokens which are invalid per the JSON spec and cause
parse errors in browsers.

Changes:

  • Added float nan/inf guard in CustomJSONEncoder.default()
  • Sanitised output string in write_json() via .replace()
  • Added test to verify non-finite floats serialise as null

@ssrhaso ssrhaso requested review from jim-smith and rpreen March 6, 2026 08:57
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.51%. Comparing base (3ac6354) to head (9575ee2).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #421   +/-   ##
=======================================
  Coverage   99.51%   99.51%           
=======================================
  Files          23       23           
  Lines        2668     2678   +10     
=======================================
+ Hits         2655     2665   +10     
  Misses         13       13           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ssrhaso ssrhaso force-pushed the fix/invalid-json-tokens branch from 58e8789 to 256b959 Compare March 6, 2026 09:01
@ssrhaso ssrhaso added bug Something isn't working waiting This issue is waiting for something else to be completed (see issue for details) and removed waiting This issue is waiting for something else to be completed (see issue for details) labels Mar 6, 2026
@ssrhaso ssrhaso self-assigned this Mar 6, 2026
Comment on lines +129 to +131
attack_report = attack_report.replace("-Infinity", "null")
attack_report = attack_report.replace("Infinity", "null")
attack_report = attack_report.replace("NaN", "null")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that:

  1. this wont catch the case where it's "nan" instead of "NaN"
  2. if any part of the metrics include this text it will replace them too; try adding "BaNaNa": "BaNaNa" to the test data for example
  3. it seems safer to handle this problem in the JSON encoder?

@rpreen
Copy link
Copy Markdown
Contributor

rpreen commented Mar 11, 2026

  • Added float nan/inf guard in CustomJSONEncoder.default()

The code does not match your comments.

@ssrhaso ssrhaso force-pushed the fix/invalid-json-tokens branch from 2efd053 to 09b9a6e Compare March 11, 2026 19:29
@ssrhaso
Copy link
Copy Markdown
Contributor Author

ssrhaso commented Mar 11, 2026

  • Added float nan/inf guard in CustomJSONEncoder.default()

The code does not match your comments.

Apologies for any errors I have made on this issue, I have tried to implement your suggested changes, please let me know your thoughts @rpreen

@rpreen
Copy link
Copy Markdown
Contributor

rpreen commented Mar 11, 2026

I don't think this will work for numpy arrays.

Try adding the test case:

"array": np.array([1.0, np.nan, np.inf])

which should assert to the following:

assert inner["array"] == [1.0, None, None]

@ssrhaso
Copy link
Copy Markdown
Contributor Author

ssrhaso commented Mar 12, 2026

I don't think this will work for numpy arrays.

Try adding the test case:

"array": np.array([1.0, np.nan, np.inf])

which should assert to the following:

assert inner["array"] == [1.0, None, None]

Changes made and updated branch. I couldn't think of a different way other than the one you suggested for numpy array handling but lgtm. Let me know your thoughts @rpreen

Copy link
Copy Markdown
Contributor

@jim-smith jim-smith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me.
@rpreen ?
For reference, I encountered this problem when looking at results.json produced from running attacks on some very vulnerable xgboost models.
it was always the first entries in the roc_thresholds entries - which probably arise from a divide by zero issue

Copy link
Copy Markdown
Contributor

@rpreen rpreen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm; thanks for making the changes

@rpreen
Copy link
Copy Markdown
Contributor

rpreen commented Mar 16, 2026

You need to add your details to the CITATION.cff but you can do that in another PR - I'll just merge this one now.

@rpreen rpreen merged commit 1d0799a into main Mar 16, 2026
4 checks passed
@rpreen rpreen deleted the fix/invalid-json-tokens branch March 16, 2026 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]invalid characters in json reports produced by attacks

3 participants