chore: Reformat docstrings to reduce docs issues by mikemckiernan · Pull Request #1268 · NVIDIA/garak

mikemckiernan · 2025-06-25T12:06:22Z

Change docstring-related docs build errors and warnings.

detectors.ansiescape file reffed detectors.always
revise docstring in attempt.py -- needs SME review
do we want basic.rst to include _plugins API?
format comments in function.py as RST
reformat docstring in litellm as RST instead of MD

Verification

Building the docs produces three remaining build warnings about documents that are not included in any toctree. That's the next PR.

jmartin-tech

Revisions to attempt.py look valid, there is ongoing work in #1254 that may expand or change typical use soon.

One minor request noted.

jmartin-tech · 2025-06-25T13:09:34Z

garak/probes/visual_jailbreak.py

    active = False

-    __doc__ = FigStepFull.__doc__ + " - Tiny version"
+    __doc__ = FigStepFull.__doc__ + "- Tiny version"


Minor quibble, the leading space here matches other suffix values used to indicate reduced, tiny or non_full versions.

Suggested change

__doc__ = FigStepFull.__doc__ + "- Tiny version"

__doc__ = FigStepFull.__doc__ + " - Tiny version"

garak/garak/probes/latentinjection.py

Line 86 in cfa7e33

non_full_suffix = " - lightweight version"

The space triggers this warning:

/home/local/repos/garak/docs/source/../../garak/probes/visual_jailbreak.py:docstring of garak.probes.visual_jailbreak.FigStep:6: ERROR: Unexpected indentation. [docutils]

Let me look for another alternative. (Thanks for the review, btw!)

Did some snooping. Seems NVIDIA tech docs should use the Chicago style for citing works. Revised as best as I could muster.

leondz · 2025-06-26T10:17:18Z

garak/attempt.py

+    * An attempt tracks a seed prompt and responses to the prompt.
+    * There is a 1:1 relationship between an attempt and a source prompt.
+    * Attempts track all generations.
+      This means messages tracks many histories, one per generation.


Suggested change

This means messages tracks many histories, one per generation.

This means ``messages`` tracks many histories, one per generation.

erickgalinkin

Left some minor tweaks. Some are my own opinions and thoughts or minor nits, but the ones for Single and Multiple are actually important. Provided suggestions there.

erickgalinkin · 2025-06-26T13:37:22Z

garak/generators/function.py


 class Multiple(Single):
-    """pass a module#function to be called as generator, with format function(prompt:str, generations:int, **kwargs)->List[Union(str, None)]"""
+    """pass a module function to call as a generator, with format ``function(prompt:str, generations:int, **kwargs)->List[Union(str, None)]``"""


Suggested change

"""pass a module function to call as a generator, with format ``function(prompt:str, generations:int, **kwargs)->List[Union(str, None)]``"""

"""Pass a function to call as a generator. Function must have the signature ``function(prompt:str, generations:int, **kwargs)->List[Union(str, None)]`` and should be supplied via cli or config in the format ``module#function``"""

Similar to the above. One must pass it in the config or on the cli as mymodule#myfunction. Perhaps this is an indication to improve this docstring more broadly -- suggestion above.

Appreciate the clarification, thanks very much!

erickgalinkin · 2025-06-26T13:41:54Z

garak/generators/function.py

+    """pass a module function to call as generator, with format ``function(prompt:str, **kwargs)->List[Union(str, None)]``.
+    The parameter `name` is reserved
+    """


Suggested change

"""pass a module function to call as generator, with format ``function(prompt:str, **kwargs)->List[Union(str, None)]``.

The parameter `name` is reserved

"""

"""Pass a function to call as a generator. Function should be supplied via cli or config in the format ``module#function`` and must have the signature ``function(prompt:str, **kwargs)->List[Union(str, None)]``.

The parameter `name` is reserved.

"""

The module#function syntax is deliberate. Adding a suggestion here.

erickgalinkin · 2025-06-26T13:44:17Z

garak/probes/phrasing.py

-of-the-art LLMs
+Refusal training is widely used to prevent LLMs from generating harmful, undesirable, or illegal outputs.
+We reveal a curious generalization gap in the current refusal training approaches:
+simply reformulating a harmful request in the past tense  is often sufficient to jailbreak many state- of-the-art LLMs


Suggested change

simply reformulating a harmful request in the past tense is often sufficient to jailbreak many state- of-the-art LLMs

simply reformulating a harmful request in the past tense is often sufficient to jailbreak many state-of-the-art LLMs

Could alternatively remove the dashes in state-of-the-art entirely.

erickgalinkin · 2025-06-26T13:45:06Z

garak/probes/visual_jailbreak.py

+    """Using another modal - image to assist jailbreak.
+    Refer to the following paper for more detail:
+
+    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023.
+    "FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts."
+    ArXiv. https://arxiv.org/pdf/2311.05608


Suggested change

"""Using another modal - image to assist jailbreak.

Refer to the following paper for more detail:

Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023.

"FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts."

ArXiv. https://arxiv.org/pdf/2311.05608

"""Using image modality to assist jailbreak.

Refer to the following paper for more detail:

Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023.

"FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts."

ArXiv. https://arxiv.org/pdf/2311.05608

- detectors.ansiescape file reffed detectors.always - revise docstring in attempt.py -- needs SME review - do we want basic.rst to include _plugins API? - format comments in function.py as RST - reformat docstring in litellm as RST instead of MD - Use Chicago style to ref arXiv papers - Incorp review feedback Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>

mikemckiernan force-pushed the mmck-docs-docstrings branch from f5e7932 to d20ae19 Compare June 25, 2025 12:08

jmartin-tech reviewed Jun 25, 2025

View reviewed changes

mikemckiernan force-pushed the mmck-docs-docstrings branch from d20ae19 to ebfb48c Compare June 25, 2025 14:47

leondz added the documentation Improvements or additions to documentation label Jun 26, 2025

leondz self-assigned this Jun 26, 2025

leondz approved these changes Jun 26, 2025

View reviewed changes

mikemckiernan force-pushed the mmck-docs-docstrings branch from ebfb48c to 78e54ec Compare June 26, 2025 12:58

erickgalinkin requested changes Jun 26, 2025

View reviewed changes

mikemckiernan force-pushed the mmck-docs-docstrings branch from 78e54ec to e4a7075 Compare June 26, 2025 14:27

leondz merged commit 6cf3a8e into NVIDIA:main Jun 27, 2025
13 of 15 checks passed

github-actions bot locked and limited conversation to collaborators Jun 27, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: Reformat docstrings to reduce docs issues#1268

chore: Reformat docstrings to reduce docs issues#1268
leondz merged 1 commit intoNVIDIA:mainfrom
mikemckiernan:mmck-docs-docstrings

mikemckiernan commented Jun 25, 2025

Uh oh!

jmartin-tech left a comment

Uh oh!

jmartin-tech Jun 25, 2025

Uh oh!

mikemckiernan Jun 25, 2025

Uh oh!

mikemckiernan Jun 25, 2025

Uh oh!

leondz Jun 26, 2025

Uh oh!

erickgalinkin left a comment

Uh oh!

erickgalinkin Jun 26, 2025

Uh oh!

mikemckiernan Jun 26, 2025

Uh oh!

erickgalinkin Jun 26, 2025

Uh oh!

erickgalinkin Jun 26, 2025

Uh oh!

erickgalinkin Jun 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	__doc__ = FigStepFull.__doc__ + "- Tiny version"
	__doc__ = FigStepFull.__doc__ + " - Tiny version"

	This means messages tracks many histories, one per generation.
	This means ``messages`` tracks many histories, one per generation.

	"""pass a module function to call as a generator, with format ``function(prompt:str, generations:int, **kwargs)->List[Union(str, None)]``"""
	"""Pass a function to call as a generator. Function must have the signature ``function(prompt:str, generations:int, **kwargs)->List[Union(str, None)]`` and should be supplied via cli or config in the format ``module#function``"""

	simply reformulating a harmful request in the past tense is often sufficient to jailbreak many state- of-the-art LLMs
	simply reformulating a harmful request in the past tense is often sufficient to jailbreak many state-of-the-art LLMs

Conversation

mikemckiernan commented Jun 25, 2025

Verification

Uh oh!

jmartin-tech left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

erickgalinkin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants