Skip to content

Conversation

@plaguss
Copy link
Contributor

@plaguss plaguss commented Aug 27, 2024

Description

This PR adds a base method to allow easily printing the prompts of the tasks.

from distilabel.steps.tasks import URIAL

# only importable if placed at the root at the repo, any LLM will work
from tests.unit.conftest import DummyLLM

t = URIAL(llm=DummyLLM())
t.load()
t.print()
...
╭────────────────────────────────────────────────────────────────── Prompt: URIAL  ──────────────────────────────────────────────────────────────────╮
│ ╭───────────────────────────────────────────────────────────────── User Message ─────────────────────────────────────────────────────────────────╮ │
│ │ # Instruction                                                                                                                                  │ │
│ │                                                                                                                                                │ │
│ │ Below is a list of conversations between a human and an AI assistant (you).                                                                    │ │
│ │ Users place their queries under "# User:", and your responses are under  "# Assistant:".                                                       │ │
│ │ You are a helpful, respectful, and honest assistant.                                                                                           │ │
│ │ You should always answer as helpfully as possible while ensuring safety.                                                                       │ │
│ │ Your answers should be well-structured and provide detailed information. They should also have an engaging tone.                               │ │
│ │ Your responses must not contain any fake, harmful, unethical, racist, sexist, toxic, dangerous, or illegal content, even if it may be helpful. │ │
│ │ Your response must be socially responsible, and thus you can refuse to answer some controversial topics.                                       │ │
│ │                                                                                                                                                │ │
│ │                                                                                                                                                │ │
│ │ # User:                                                                                                                                        │ │
│ │                                                                                                                                                │ │
│ │ <PLACEHOLDER_INSTRUCTION>                                                                                                                      │ │
│ │                                                                                                                                                │ │
│ │ # Assistant:                                                                                                                                   │ │
│ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Example screenshot:

image

For new tasks, we should review the prompt, and if not rendering properly, override the _sample_inputs method to return the appropriate content.

Run the following to see all the prompts at once:

pytest tests/integration/test_prints.py -s -vv

@plaguss plaguss requested a review from gabrielmbmb August 27, 2024 19:18
@plaguss plaguss self-assigned this Aug 27, 2024
@plaguss plaguss added this to the 1.4.0 milestone Aug 27, 2024
@plaguss plaguss requested a review from dvsrepo August 27, 2024 19:19
@github-actions
Copy link

Documentation for this PR has been built. You can view it at: https://distilabel.argilla.io/pr-934/

@codspeed-hq
Copy link

codspeed-hq bot commented Aug 27, 2024

CodSpeed Performance Report

Merging #934 will not alter performance

Comparing pretty-print (a213728) with develop (2ce44f0)

Summary

✅ 1 untouched benchmarks

@plaguss plaguss marked this pull request as ready for review August 28, 2024 07:51
@plaguss plaguss added enhancement New feature or request question Further information is requested labels Sep 23, 2024
@plaguss plaguss merged commit 87683f0 into develop Oct 7, 2024
@plaguss plaguss deleted the pretty-print branch October 7, 2024 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request question Further information is requested

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants