generators: check message count requested matches message count generated

[leondz]

Description

• raise exception if OpenAICompatible message count rcvd doesn't match generations requested
• raise exception in probe() if message count rcvd doesn't match generations requested

Open to moving checks (maybe _execute_all?) or adding a hook

Verification

Put this lying code in garak/generators/test.py and request > 1 generation:

class AlwaysOne(Generator):
    """ always return one message """

    name = "AlwaysOne"
    generator_family_name = "Test"
    supports_multiple_generations = True

    def _call_model(
        self, prompt: Conversation, generations_this_call: int = 1
    ) -> List[Message | None]:
        return [Message("")]

(garak) 13:15:19 x1:~/dev/garak [generators/check_generations_requested_matches] $ python -m garak -t test.AlwaysOne -g 4 -p phrasing
garak LLM vulnerability scanner v0.14.0.pre1 ( https://github.com/NVIDIA/garak ) at 2026-01-15T13:15:22.019254
📜 logging to /home/lderczynski/.local/share/garak/garak.log
🦜 loading generator: Test: AlwaysOne
📜 reporting to /home/lderczynski/.local/share/garak/garak_runs/garak.1158b59d-9ade-4421-bf24-7a31d5ce9b1b.report.jsonl
🕵️  queue of probes: phrasing.FutureTense, phrasing.PastTense
Generator did not return the requested number of responses (asked for 4 got 1). supports_multiple_generations may be set wrongly.                                      
(garak) 13:15:23 x1:~/dev/garak [generators/check_generations_requested_matches] $

[jmartin-tech]

These restrictions seem sensible to me. A minor nit pick added related to error phrasing.

I am not 100% sure however there might be an edge cases where a target might behave irregularly causing this to terminate a run that has already gathered responses for a probe. We can accept this as is and await user feedback on if this ends up being something we should handle more gracefully.

[leondz]

I am not 100% sure however there might be an edge cases where a target might behave irregularly causing this to terminate a run that has already gathered responses for a probe. We can accept this as is and await user feedback on if this ends up being something we should handle more gracefully.

Yeah, this is a risk. I suspect this should be caught at generator level, but let's wait for the signal to emerge first