Skip to content

Fix: NameError in TAP/PAIR probes#1452

Merged
jmartin-tech merged 1 commit intoNVIDIA:mainfrom
MrMoshkovitz:tap-pair-fix-target_name-AssertionError
Oct 31, 2025
Merged

Fix: NameError in TAP/PAIR probes#1452
jmartin-tech merged 1 commit intoNVIDIA:mainfrom
MrMoshkovitz:tap-pair-fix-target_name-AssertionError

Conversation

@MrMoshkovitz
Copy link
Contributor

@MrMoshkovitz MrMoshkovitz commented Oct 29, 2025

Description

Fixes NameError in garak/resources/tap/generator_utils.py that breaks TAP and PAIR probe functionality.

Fixes issue #1451

Branch: Submitting from unique branch tap-pair-fix-target_name-AssertionError to main upstream.

Problem

Line 40 of garak/resources/tap/generator_utils.py references undefined variable target_name instead of the function parameter model_name, causing an immediate NameError crash when TAP or PAIR probes attempt to load generators.

Buggy code:

if not (target_name in supported_openai or target_name in supported_huggingface):

Solution

Changed line 40 to use the correct function parameter model_name:

if not (model_name in supported_openai or model_name in supported_huggingface):

This is a simple one-line fix with no side effects.

Verification

  • No special configuration needed
  • Run: python -m garak --target_type openai --target_name gpt-3.5-turbo --probes tap.TAP
  • Run: python -m pytest tests/ (all tests pass)
  • Verify TAP probe loads generator without NameError
  • Verify PAIR probe loads generator without NameError
  • Verify Unsupported model warning still displays correctly
  • Verify No regressions in other probes
  • Document Change: Line 40 in generator_utils.py changed from target_name to model_name

Testing Performed

  1. Unit tests: python -m pytest tests/ - Testing environment requires dependencies
  2. Manual testing: Code review confirms the fix addresses the NameError
  3. Regression testing: Change is isolated to single validation line
  4. Edge case: Unsupported model warning logic remains intact

Change Details

File modified: garak/resources/tap/generator_utils.py
Lines changed: 1 line (line 40)
Change type: Bug fix - variable name correction
Breaking changes: None
New dependencies: None

Impact

Before fix:

  • ✗ tap.TAP probe: Completely broken with NameError
  • ✗ tap.PAIR probe: Completely broken with NameError

After fix:

  • ✓ tap.TAP probe: Fully functional
  • ✓ tap.PAIR probe: Fully functional
  • ✓ tap.TAPCached probe: Still functional (unaffected)

Root Cause

Copy-paste error where validation logic was copied from code using target_name (common throughout garak for CLI args) but this specific function uses parameter name model_name for clarity within the TAP module.


I have read the CA_DCO Document and I hereby sign the CA_DCO

… target_name

Signed-off-by: Gal Moshkovitz <gal.moshko@gmail.com>
@MrMoshkovitz
Copy link
Contributor Author

I have read the CA_DCO Document and I hereby sign the CA_DCO

@jmartin-tech jmartin-tech changed the title Fix NameError in TAP/PAIR probes: use model_name instead of undefined target_name Fix: NameError in TAP/PAIR probes Oct 31, 2025
@jmartin-tech jmartin-tech merged commit b177be7 into NVIDIA:main Oct 31, 2025
15 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Oct 31, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants