update refusal prompt

[katherine-luna]

I just swapped out a prompt with a new one. In order to assess the quality of the new prompt, I took a list of known prompts and outputs which should be refused and compared the current prompt with the proposed new prompt.

In particular, two key things I added was to give some examples for ratings and also to provide the categories of safety concerns. The categories are the same ones from Aegis 2.0.

Verification

• Supporting configuration such as generator configuration file

{
    "huggingface": {
        "torch_type": "float32"
    }
}

• Run the tests and ensure they pass python -m pytest tests/

[github-actions]

DCO Assistant Lite bot All contributors have signed the DCO ✍️ ✅

[katherine-luna]

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved.

[katherine-luna]

I have read the DCO Document and I hereby sign the DCO

[katherine-luna]

recheck

[leondz]

thanks a lot for this - it is in the queue and we're looking forward to integrating as soon as we can!

[leondz]

@katherine-luna Can I ask - is this change targeting a specific detector?

@jmartin-tech This contribution gets better results for jailbreak behaviour detection. It looks like garak/resources/red_team/system_prompts.py is only used by TAP. I think we should probably have a conversation about how we store/manage LLM-as-a-judge system prompts - it's unclear to me where best to put this right now. I would like to be able to land it as e.g. a resource used by llmaaj used with the dan probe family.