[Eval] Add RAL to hex eval

[Emperorlou]

Thank you for contributing an eval! ♥️

🚨 Please make sure your PR follows these guidelines, failure to follow the guidelines below will result in the PR being closed automatically. Note that even if the criteria are met, that does not guarantee the PR will be merged nor GPT-4 access be granted. 🚨

PLEASE READ THIS:

In order for a PR to be merged, it must fail on GPT-4. We are aware that right now, users do not have access, so you will not be able to tell if the eval fails or not. Please run your eval with GPT-3.5-Turbo, but keep in mind as we run the eval, if GPT-4 gets higher than 90% on the eval, we will likely reject it since GPT-4 is already capable of completing the task.

We plan to roll out a way for users submitting evals to see the eval performance on GPT-4 soon. Stay tuned! Until then, you will not be able to see the eval performance on GPT-4. Starting April 10, the minimum eval count is 15 samples, we hope this makes it easier to create and contribute evals.

Also, please note that we're using Git LFS for storing the JSON files, so please make sure that you move the JSON file to Git LFS before submitting a PR. Details on how to use Git LFS are available here.

Eval details 📑

Eval name

RAL To Hex

Eval description

This converts RAL color codes to their hex color counterparts.

What makes this a useful eval?

Training an AI to understand and convert RAL color codes to hex color codes can enhance cross-disciplinary communication, bridging the gap between professionals like architects or manufacturers who use RAL codes and digital designers or developers who primarily use hex codes. This capability can also facilitate the automation of design tasks, significantly boosting efficiency and productivity in design-related industries.

Criteria for a good eval ✅

Below are some of the criteria we look for in a good eval. In general, we are seeking cases where the model does not do a good job despite being capable of generating a good response (note that there are some things large language models cannot do, so those would not make good evals).

Your eval should be:

• Thematically consistent: The eval should be thematically consistent. We'd like to see a number of prompts all demonstrating some particular failure mode. For example, we can create an eval on cases where the model fails to reason about the physical world.
• Contains failures where a human can do the task, but either GPT-4 or GPT-3.5-Turbo could not.
• Includes good signal around what is the right behavior. This means either a correct answer for Basic evals or the Fact Model-graded eval, or an exhaustive rubric for evaluating answers for the Criteria Model-graded eval.
• Include at least 15 high-quality examples.

If there is anything else that makes your eval worth including, please document it below.

Unique eval value

Insert what makes your eval high quality that was not mentioned above. (Not required)

Eval structure 🏗️

Your eval should

• Check that your data is in evals/registry/data/{name}
• Check that your YAML is registered at evals/registry/evals/{name}.yaml
• Ensure you have the right to use the data you submit via this eval

(For now, we will only be approving evals that use one of the existing eval classes. You may still write custom eval classes for your own cases, and we may consider merging them in the future.)

Final checklist 👀

Submission agreement

By contributing to Evals, you are agreeing to make your evaluation logic and data under the same MIT license as this repository. You must have adequate rights to upload any data used in an Eval. OpenAI reserves the right to use this data in future service improvements to our product. Contributions to OpenAI Evals will be subject to our usual Usage Policies (https://platform.openai.com/docs/usage-policies).

• I agree that my submission will be made available under an MIT license and complies with OpenAI's usage policies.

Email address validation

If your submission is accepted, we will be granting GPT-4 access to a limited number of contributors. Access will be given to the email address associated with the commits on the merged pull request.

• I acknowledge that GPT-4 access will only be granted, if applicable, to the email address used for my merged pull request.

Limited availability acknowledgment

We know that you might be excited to contribute to OpenAI's mission, help improve our models, and gain access to GPT-4. However, due to the requirements mentioned above and the high volume of submissions, we will not be able to accept all submissions and thus not grant everyone who opens a PR GPT-4 access. We know this is disappointing, but we hope to set the right expectation before you open this PR.

• I understand that opening a PR, even if it meets the requirements above, does not guarantee the PR will be merged nor GPT-4 access be granted.

Submit eval

• I have filled out all required fields of this form
• I have used Git LFS for the Eval JSON data
• (Ignore if not submitting code) I have run pip install pre-commit; pre-commit install and have verified that black, isort, and autoflake are running when I commit and push

Failure to fill out all required fields will result in the PR being closed.

Eval JSON data

Since we are using Git LFS, we are asking eval submitters to add in as many Eval Samples (at least 5) from their contribution here:

View evals in JSON

Eval

{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1000"}],"ideal":"#BEBD7F"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1001"}],"ideal":"#C2B078"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1002"}],"ideal":"#C6A664"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1003"}],"ideal":"#E5BE01"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1004"}],"ideal":"#CDA434"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1005"}],"ideal":"#A98307"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1006"}],"ideal":"#E4A010"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1007"}],"ideal":"#DC9D00"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1011"}],"ideal":"#8A6642"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1012"}],"ideal":"#C7B446"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1013"}],"ideal":"#EAE6CA"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1014"}],"ideal":"#E1CC4F"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1015"}],"ideal":"#E6D690"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1016"}],"ideal":"#EDFF21"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1017"}],"ideal":"#F5D033"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1018"}],"ideal":"#F8F32B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1019"}],"ideal":"#9E9764"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1020"}],"ideal":"#999950"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1021"}],"ideal":"#F3DA0B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1023"}],"ideal":"#FAD201"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1024"}],"ideal":"#AEA04B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1026"}],"ideal":"#FFFF00"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1027"}],"ideal":"#9D9101"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1028"}],"ideal":"#F4A900"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1032"}],"ideal":"#D6AE01"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1033"}],"ideal":"#F3A505"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1034"}],"ideal":"#EFA94A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1035"}],"ideal":"#6A5D4D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1036"}],"ideal":"#705335"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 1037"}],"ideal":"#F39F18"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2000"}],"ideal":"#ED760E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2001"}],"ideal":"#C93C20"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2002"}],"ideal":"#CB2821"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2003"}],"ideal":"#FF7514"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2004"}],"ideal":"#F44611"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2005"}],"ideal":"#FF2301"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2007"}],"ideal":"#FFA420"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2008"}],"ideal":"#F75E25"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2009"}],"ideal":"#F54021"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2010"}],"ideal":"#D84B20"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2011"}],"ideal":"#EC7C26"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2012"}],"ideal":"#E55137"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 2013"}],"ideal":"#C35831"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3000"}],"ideal":"#AF2B1E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3001"}],"ideal":"#A52019"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3002"}],"ideal":"#A2231D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3003"}],"ideal":"#9B111E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3004"}],"ideal":"#75151E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3005"}],"ideal":"#5E2129"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3007"}],"ideal":"#412227"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3009"}],"ideal":"#642424"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3011"}],"ideal":"#781F19"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3012"}],"ideal":"#C1876B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3013"}],"ideal":"#A12312"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3014"}],"ideal":"#D36E70"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3015"}],"ideal":"#EA899A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3016"}],"ideal":"#B32821"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3017"}],"ideal":"#E63244"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3018"}],"ideal":"#D53032"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3020"}],"ideal":"#CC0605"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3022"}],"ideal":"#D95030"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3024"}],"ideal":"#F80000"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3026"}],"ideal":"#FE0000"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3027"}],"ideal":"#C51D34"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3028"}],"ideal":"#CB3234"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3031"}],"ideal":"#B32428"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3032"}],"ideal":"#721422"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 3033"}],"ideal":"#B44C43"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4001"}],"ideal":"#6D3F5B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4002"}],"ideal":"#922B3E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4003"}],"ideal":"#DE4C8A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4004"}],"ideal":"#641C34"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4005"}],"ideal":"#6C4675"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4006"}],"ideal":"#A03472"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4007"}],"ideal":"#4A192C"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4008"}],"ideal":"#924E7D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4009"}],"ideal":"#A18594"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4010"}],"ideal":"#CF3476"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4011"}],"ideal":"#8673A1"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 4012"}],"ideal":"#6C6874"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5000"}],"ideal":"#354D73"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5001"}],"ideal":"#1F3438"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5002"}],"ideal":"#20214F"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5003"}],"ideal":"#1D1E33"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5004"}],"ideal":"#18171C"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5005"}],"ideal":"#1E2460"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5007"}],"ideal":"#3E5F8A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5008"}],"ideal":"#26252D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5009"}],"ideal":"#025669"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5010"}],"ideal":"#0E294B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5011"}],"ideal":"#231A24"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5012"}],"ideal":"#3B83BD"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5013"}],"ideal":"#1E213D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5014"}],"ideal":"#606E8C"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5015"}],"ideal":"#2271B3"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5017"}],"ideal":"#063971"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5018"}],"ideal":"#3F888F"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5019"}],"ideal":"#1B5583"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5020"}],"ideal":"#1D334A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5021"}],"ideal":"#256D7B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5022"}],"ideal":"#252850"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5023"}],"ideal":"#49678D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5024"}],"ideal":"#5D9B9B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5025"}],"ideal":"#2A6478"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 5026"}],"ideal":"#102C54"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6000"}],"ideal":"#316650"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6001"}],"ideal":"#287233"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6002"}],"ideal":"#2D572C"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6003"}],"ideal":"#424632"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6004"}],"ideal":"#1F3A3D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6005"}],"ideal":"#2F4538"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6006"}],"ideal":"#3E3B32"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6007"}],"ideal":"#343B29"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6008"}],"ideal":"#39352A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6009"}],"ideal":"#31372B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6010"}],"ideal":"#35682D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6011"}],"ideal":"#587246"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6012"}],"ideal":"#343E40"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6013"}],"ideal":"#6C7156"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6014"}],"ideal":"#47402E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6015"}],"ideal":"#3B3C36"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6016"}],"ideal":"#1E5945"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6017"}],"ideal":"#4C9141"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6018"}],"ideal":"#57A639"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6019"}],"ideal":"#BDECB6"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6020"}],"ideal":"#2E3A23"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6021"}],"ideal":"#89AC76"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6022"}],"ideal":"#25221B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6024"}],"ideal":"#308446"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6025"}],"ideal":"#3D642D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6026"}],"ideal":"#015D52"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6027"}],"ideal":"#84C3BE"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6028"}],"ideal":"#2C5545"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6029"}],"ideal":"#20603D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6032"}],"ideal":"#317F43"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6033"}],"ideal":"#497E76"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6034"}],"ideal":"#7FB5B5"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6035"}],"ideal":"#1C542D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6036"}],"ideal":"#193737"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6037"}],"ideal":"#008F39"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 6038"}],"ideal":"#00BB2D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7000"}],"ideal":"#78858B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7001"}],"ideal":"#8A9597"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7002"}],"ideal":"#7E7B52"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7003"}],"ideal":"#6C7059"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7004"}],"ideal":"#969992"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7005"}],"ideal":"#646B63"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7006"}],"ideal":"#6D6552"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7008"}],"ideal":"#6A5F31"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7009"}],"ideal":"#4D5645"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7010"}],"ideal":"#4C514A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7011"}],"ideal":"#434B4D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7012"}],"ideal":"#4E5754"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7013"}],"ideal":"#464531"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7015"}],"ideal":"#434750"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7016"}],"ideal":"#293133"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7021"}],"ideal":"#23282B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7022"}],"ideal":"#332F2C"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7023"}],"ideal":"#686C5E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7024"}],"ideal":"#474A51"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7026"}],"ideal":"#2F353B"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7030"}],"ideal":"#8B8C7A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7031"}],"ideal":"#474B4E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7032"}],"ideal":"#B8B799"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7033"}],"ideal":"#7D8471"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7034"}],"ideal":"#8F8B66"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7035"}],"ideal":"#D7D7D7"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7036"}],"ideal":"#7F7679"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7037"}],"ideal":"#7D7F7D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7038"}],"ideal":"#B5B8B1"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7039"}],"ideal":"#6C6960"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7040"}],"ideal":"#9DA1AA"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7042"}],"ideal":"#8D948D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7043"}],"ideal":"#4E5452"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7044"}],"ideal":"#CAC4B0"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7045"}],"ideal":"#909090"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7046"}],"ideal":"#82898F"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7047"}],"ideal":"#D0D0D0"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 7048"}],"ideal":"#898176"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8000"}],"ideal":"#826C34"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8001"}],"ideal":"#955F20"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8002"}],"ideal":"#6C3B2A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8003"}],"ideal":"#734222"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8004"}],"ideal":"#8E402A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8007"}],"ideal":"#59351F"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8008"}],"ideal":"#6F4F28"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8011"}],"ideal":"#5B3A29"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8012"}],"ideal":"#592321"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8014"}],"ideal":"#382C1E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8015"}],"ideal":"#633A34"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8016"}],"ideal":"#4C2F27"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8017"}],"ideal":"#45322E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8019"}],"ideal":"#403A3A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8022"}],"ideal":"#212121"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8023"}],"ideal":"#A65E2E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8024"}],"ideal":"#79553D"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8025"}],"ideal":"#755C48"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8028"}],"ideal":"#4E3B31"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 8029"}],"ideal":"#763C28"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9001"}],"ideal":"#FDF4E3"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9002"}],"ideal":"#E7EBDA"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9003"}],"ideal":"#F4F4F4"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9004"}],"ideal":"#282828"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9005"}],"ideal":"#0A0A0A"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9006"}],"ideal":"#A5A5A5"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9007"}],"ideal":"#8F8F8F"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9010"}],"ideal":"#FFFFFF"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9011"}],"ideal":"#1C1C1C"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9016"}],"ideal":"#F6F6F6"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9017"}],"ideal":"#1E1E1E"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9018"}],"ideal":"#D7D7D7"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9022"}],"ideal":"#9C9C9C"}
{"input":[{"role":"system","content":"Convert RAL color code to its hex representation."},{"role":"user","content":"RAL 9023"}],"ideal":"#828282"}

[usama-openai]

Thanks for submitting this eval! This PR looks good. I'm approving this PR.

[Emperorlou]

Great, thanks! Do you know if GPT4 API admissions are still available this way?

…

On Thu, 22 Jun 2023 at 13:48, Usama ***@***.***> wrote: ***@***.**** approved this pull request. Thanks for submitting this eval! This PR looks good. I'm approving this PR. — Reply to this email directly, view it on GitHub <#1218 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACM6HAI44SV3S4F3RUIELDXMSVR5ANCNFSM6AAAAAAZQSRYDU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[andrew-openai]

Yes, added!

[Emperorlou]

Awesome, thank you so much! Is there a way to gain access to the 32k similar to this? Or is it just a wait list?

…

On Thu, 22 Jun 2023 at 14:03, Andrew ***@***.***> wrote: Yes, added! — Reply to this email directly, view it on GitHub <#1218 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACM6HCBAAJJ5OM7AGIVBMDXMSXKFANCNFSM6AAAAAAZQSRYDU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[unicomp21]

@andrew-openai ditto, I'd like 32k access as well, if possible?

#771