Skip to content

Fix: atkgen probe verbose output#1447

Merged
jmartin-tech merged 2 commits intoNVIDIA:mainfrom
MrMoshkovitz:atkgen-fix-Turn-content
Oct 31, 2025
Merged

Fix: atkgen probe verbose output#1447
jmartin-tech merged 2 commits intoNVIDIA:mainfrom
MrMoshkovitz:atkgen-fix-Turn-content

Conversation

@MrMoshkovitz
Copy link
Contributor

Summary

Fixes AttributeError crash in atkgen probe by updating code to use correct Turn object structure (turns[-1].content.text instead of turns[-1].text).

Problem

The atkgen.Tox probe (and likely other atkgen probes) crashed immediately when attempting to log conversation output:

AttributeError: 'Turn' object has no attribute 'text'
  File "garak/probes/atkgen.py", line 208
    f"atkgen: 🦜 model: {this_attempt.prompt.turns[-1].text}"

This prevented the entire atkgen probe family from running.

Root Cause

The Turn object structure changed to use a nested architecture:

  • Turn has a .content attribute (which is a Message object)
  • Message has the .text attribute

The code was attempting to access .text directly on the Turn object, which no longer exists.

Solution

Updated line 208 in garak/probes/atkgen.py:

Before:

f"atkgen: 🦜 model: {Style.BRIGHT}{this_attempt.prompt.turns[-1].text}{Style.RESET_ALL}"

After:

f"atkgen: 🦜 model: {Style.BRIGHT}{this_attempt.prompt.turns[-1].content.text}{Style.RESET_ALL}"

Changes

  • Modified: garak/probes/atkgen.py (1 line changed)
    • Line 208: Changed .turns[-1].text to .turns[-1].content.text

Testing

Verified the fix works:

  1. Ran garak --target_type openai_rated --target_name gpt-3.5-turbo-instruct --probes atkgen.Tox --generations 10 -vv
  2. Probe executes without AttributeError
  3. Conversation output displays correctly: atkgen: 🦜 model: ...
  4. No other instances of this pattern found in the file

Impact

  • ✅ Fixes crash in atkgen probe family
  • ✅ Allows atkgen probes to run successfully
  • ✅ No breaking changes to other code
  • ✅ Minimal change (1 line)

Backward Compatibility

✅ Fully compatible - only fixes broken functionality

Related Issue

Closes #1444


Checklist:

  • Bug identified and root cause understood
  • Fix implemented with minimal change
  • Code tested and verified working
  • No other instances of the pattern found in file
  • No breaking changes introduced

Signed-off-by: Gal Moshkovitz <gal.moshko@gmail.com>
@MrMoshkovitz
Copy link
Contributor Author

I have read the DCO Document and I hereby sign the DCO

@github-actions
Copy link
Contributor

github-actions bot commented Oct 29, 2025

DCO Assistant Lite bot All contributors have signed the DCO ✍️ ✅

@MrMoshkovitz
Copy link
Contributor Author

I have read the CA_DCO Document and I hereby sign the CA_DCO

Signed-off-by: Gal Moshkovitz <gal.moshko@gmail.com>
github-actions bot added a commit that referenced this pull request Oct 29, 2025
@MrMoshkovitz
Copy link
Contributor Author

@jmartin-tech
I have read the CA_DCO Document and I hereby sign the CA_DCO
Done: #1447

@jmartin-tech jmartin-tech changed the title Fix AttributeError: use .turns[-1].content.text in atkgen probe Fix: atkgen probe verbose output Oct 29, 2025
@jmartin-tech jmartin-tech merged commit 2583b3c into NVIDIA:main Oct 31, 2025
15 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Oct 31, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AttributeError in atkgen probe - 'Turn' object has no attribute 'text'

2 participants