This project benchmarks decision-making capabilities of large language models (LLMs) across different simulated environments using text-based interfaces. It introduces a translation layer from Gym to natural language, enabling LLMs to interact with and control classic Atari games.
This extension focuses on evaluating prompting strategies (e.g., naive, cot, reflexion_last, reflexion_max) across different LLMs (qwen7b, llama, gemma) and four distinct levels of information access.
We design environments with increasing accessibility to trajectory or expert information. The four levels of the benchmark are:
- Basic: No trajectory provided. This is the zero-shot baseline.
- Obscuring: A random policy trajectory is given, but the agent doesn't know it is random. Simulates limited noisy feedback.
- RL_traj: A trajectory generated by a trained RL agent is provided. On-policy imitation-like learning.
- Expert_goal: The goal or core instruction of an expert agent is provided. A high-level instruction-following scenario.
Each image is named using the structure:
<model>_<scenario>_<agent1>_<agent2>.png
<model>: one ofqwen7b,llama, orgemma<scenario>: one ofbasic,obscuring,RL_traj,expert_goal<agent1>vs.<agent2>: compared agent strategies
Create ./deciders/gpt.py to define your GPT-based decision-making agent:
import openai
class gpt:
def __init__(self, args):
if args.api_type == "azure":
openai.api_type = "azure"
openai.api_version = "2023-05-15"
openai.api_base = "https://<your-endpoint>.openai.azure.com/"
openai.api_key = "your-azure-key"
else:
openai.api_key = "your-openai-key"conda env create --file environment.yamlRun with shell script:
sh shell/test_cartpole.shOr manually:
python main_reflexion.py --env_name CartPole-v0 \
--init_summarizer cart_init_translator \
--curr_summarizer cart_basic_translator \
--decider exe_actor --prompt_level 1 --num_trails 1 \
--distiller guide_generator --api_type openaicot vs llama
cot vs qwen7b
cot vs qwen7b
naive vs llama
naive vs qwen7b
naive vs qwen7b
reflexion vs last
reflexion vs last
reflexion vs last
reflexion vs max
reflexion vs max
reflexion vs max
llama vs gemma
qwen7b vs gemma
qwen7b vs llama
llama vs gemma
qwen7b vs gemma
qwen7b vs llama
last vs llama
last vs qwen7b
last vs qwen7b
max vs llama
max vs qwen7b
max vs qwen7b
cot vs llama
cot vs qwen7b
cot vs qwen7b
naive vs llama
naive vs qwen7b
naive vs qwen7b
reflexion vs last
reflexion vs last
reflexion vs last
reflexion vs max
reflexion vs max
reflexion vs max
traj vs cot
traj vs cot
traj vs naive
traj vs naive
traj vs naive
traj vs reflexion
cot vs reflexion
cot vs reflexion
naive vs cot
naive vs reflexion
naive vs reflexion
reflexion vs last
basic vs RL
basic vs game
basic vs obscuring
game vs manual
obscuring vs RL
obscuring vs game
manual vs cot
manual vs cot
manual vs naive
manual vs naive
manual vs naive
manual vs reflexion
basic vs RL
basic vs game
basic vs obscuring
game vs manual
obscuring vs RL
obscuring vs game
cot vs reflexion
cot vs reflexion
naive vs cot
naive vs reflexion
naive vs reflexion
reflexion vs last
last vs basic
last vs basic
last vs basic
last vs game
last vs obscuring
last vs obscuring
max vs basic
max vs basic
max vs basic
max vs game
max vs obscuring
max vs obscuring
traj vs cot
traj vs cot
traj vs naive
traj vs naive
traj vs naive
traj vs reflexion
cot vs reflexion
cot vs reflexion
naive vs cot
naive vs reflexion
naive vs reflexion
reflexion vs last
basic vs RL
basic vs game
basic vs obscuring
game vs manual
obscuring vs RL
obscuring vs game
manual vs cot
manual vs cot
manual vs naive
manual vs naive
manual vs naive
manual vs reflexion
basic vs RL
basic vs game
basic vs obscuring
game vs manual
obscuring vs RL
obscuring vs game
cot vs reflexion
cot vs reflexion
naive vs cot
naive vs reflexion
naive vs reflexion
reflexion vs last
last vs basic
last vs basic
last vs basic
last vs game
last vs obscuring
last vs obscuring
max vs basic
max vs basic
max vs basic
max vs game
max vs obscuring
max vs obscuring
llama vs gemma
qwen7b vs gemma
qwen7b vs llama
llama vs gemma
qwen7b vs gemma
qwen7b vs llama
last vs llama
last vs qwen7b
last vs qwen7b
max vs llama
max vs qwen7b
max vs qwen7b
traj vs cot
traj vs cot
traj vs naive
traj vs naive
traj vs naive
traj vs reflexion
cot vs reflexion
cot vs reflexion
naive vs cot
naive vs reflexion
naive vs reflexion
reflexion vs last
basic vs RL
basic vs game
basic vs obscuring
game vs manual
obscuring vs RL
obscuring vs game
manual vs cot
manual vs cot
manual vs naive
manual vs naive
manual vs naive
manual vs reflexion
basic vs RL
basic vs game
basic vs obscuring
game vs manual
obscuring vs RL
obscuring vs game
cot vs reflexion
cot vs reflexion
naive vs cot
naive vs reflexion
naive vs reflexion
reflexion vs last
last vs basic
last vs basic
last vs basic
last vs game
last vs obscuring
last vs obscuring
max vs basic
max vs basic
max vs basic
max vs game
max vs obscuring
max vs obscuring
- Translate your Gym env to TextGym format in
envs/ - Add PPO/expert results to
record_reflexion.csv - Add prompt templates and few-shot examples in
prompts/ - Test using
shell/scripts or CLI
├── fig1/ # All visualizations
├── envs/ # Text-based Gym translators
├── prompts/ # Prompt templates & few-shots
├── deciders/ # LLM-based agents
├── shell/ # Experiment shell scripts
├── README_visualizations.md # Image gallery (192 images)
└── record_reflexion.csv # Reward logs
If you use this project or our visualization protocol, please cite this work (citation info to be added post-publication).