Skip to content

Add record level reporting and PyTorch support to StructuralAttack#367

Merged
jim-smith merged 14 commits intomainfrom
364-new-feature-request-extend-structural-attacks-pytorch-models-and-store-record-level-outcomes
Sep 11, 2025
Merged

Add record level reporting and PyTorch support to StructuralAttack#367
jim-smith merged 14 commits intomainfrom
364-new-feature-request-extend-structural-attacks-pytorch-models-and-store-record-level-outcomes

Conversation

@kayatefi
Copy link
Copy Markdown
Contributor

  • Add store record level
  • Support PyTorch models

@codecov
Copy link
Copy Markdown

codecov bot commented Aug 27, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.42%. Comparing base (3acfd9e) to head (3ed3916).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #367      +/-   ##
==========================================
- Coverage   99.67%   99.42%   -0.26%     
==========================================
  Files          44       23      -21     
  Lines        4363     2615    -1748     
==========================================
- Hits         4349     2600    -1749     
- Misses         14       15       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jim-smith
Copy link
Copy Markdown
Contributor

jim-smith commented Sep 8, 2025

@kayatefi Before we can merge this pull request we should define a simple case where we know what the correct record-level output should be and then write an appropriate test to see it is generated properly.

I was thinking of doing something simple with a decision tree and a small dataset- maybe 20 cases (some being duplicates), of 2/3 variables with an xor decision function.

  • scenario 1: decision tree max depth 1: should get 50% accuracy, at least 10 records in each leaf so 5 of each class ==> should fail all the strucutural tests
  • scenario 2: let the tree grow deeper: should her able to manipulate it to show class disclosure and small group problems

@jim-smith jim-smith requested a review from rpreen September 10, 2025 23:31
@jim-smith
Copy link
Copy Markdown
Contributor

@rpreen I've been through Kay's code which was mostly fine then added the individual record-level reporting, which meant I had to rewrite/refactor it a bit.

Everything should be tested for correct functionality as well as coverage.

@rpreen rpreen changed the title Add store record level and support PyTorch models Add record level reporting and PyTorch support to StructuralAttack Sep 11, 2025
@jim-smith jim-smith merged commit bfc1f94 into main Sep 11, 2025
4 checks passed
@jim-smith jim-smith deleted the 364-new-feature-request-extend-structural-attacks-pytorch-models-and-store-record-level-outcomes branch September 11, 2025 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[New Feature Request] Extend structural attacks : PyTorch models and store record-level outcomes

3 participants