README.md

Prompts

We provide a set of task-specific prompt templates that are known to work well with Cosmos-Reason1:

• Caption
• Question
  • Question
  • Multiple Choice Question
• Temporal
  • Temporal Caption (json)
  • Temporal Caption (text)
  • Temporal Localization
• Critic
  • Video Analyzer
  • Video Critic
• Domain Specific
  • Action Planning
  • AV
  • Driving
  • Robot
• Utility
  • Prompt Upsampler

Addons

These are added to the system prompt to provide additional instructions:

• Reasoning
• English

Questions

Example questions:

• What are the potential safety hazards?
• Describe what is happening in this video/image.
• What objects do you see?
• What actions are being performed?
• Are there any people in this media?
• What is the robot doing?
• Is this demonstration successful?
• What could be improved in this process?