GSK-2376 Add seed to LLM generators and evaluators in order to ensure reproducibility

[kevinmessiaen]

Description

Added rng_seed param to evaluators, generators and tests using LLM in. order to ensure reproducibility.

Related Issue

• GSK-2376 (Available on Linear)

Type of Change

• 📚 Examples / docs / tutorials / dependencies update
• 🔧 Bug fix (non-breaking change which fixes an issue)
• 🥂 Improvement (non-breaking change which improves an existing feature)
• 🚀 New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to change)
• 🔐 Security fix

[linear]

GSK-2376 Add seed to LLM generators and evaluators in order to ensure reproducibility

[sonarqubecloud]

Quality Gate failed

Failed conditions
68.4% Coverage on New Code (required ≥ 70%)

See analysis details on SonarCloud

[Hartorn]

No issue for me, would like @mattbit or @rabah-khalek opinion

[rabah-khalek]

looks okay to me, although one caveat: https://github.com/openai/openai-python/blob/a7115b5f33acd27326e5f78e19beb0d73bd3268e/src/openai/resources/chat/completions.py#L172-L176

          seed: This feature is in Beta. If specified, our system will make a best effort to
              sample deterministically, such that repeated requests with the same `seed` and
              parameters should return the same result. Determinism is not guaranteed, and you
              should refer to the `system_fingerprint` response parameter to monitor changes
              in the backend.

Could you add this to the docstrings @kevinmessiaen ?

Let me test it on an example and get back you with my final review

[rabah-khalek]

Apparently I ran out of quota to run gpt-models. I asked @kevinmessiaen and he verified it on the product description tutorial. Since we can't make functional tests with openai. There's nothing much more to do here (maybe just the docstring clarification, but that's minor).