feat: add images support to the Codex Typescript SDK

[needs]

Extend run and runStreamed input to be either a string or structured input. A structured input is an array of text parts and/or image paths, which will then be fed to the CLI through the --image argument. Text parts are combined with double newlines. For instance:

const turn = await thread.run([
  { type: "text", text: "Describe these screenshots" },
  { type: "local_image", path: "./ui.png" },
  { type: "local_image", path: "./diagram.jpg" },
  { type: "text", text: "Thanks!" },
]);

Ends up launching the CLI with:

codex exec --image foo.png --image bar.png "Describe these screenshots\n\nThanks!" 

The complete Input type for both function now is:

export type UserInput =
  | {
      type: "text";
      text: string;
    }
  | {
      type: "local_image";
      path: string;
    };

export type Input = string | UserInput[];

This brings the Codex SDK closer to feature parity with the CLI. Adresses #5280 .

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[needs]

I have read the CLA Document and I hereby sign the CLA

[pakrym-oai]

Hi! Thank you for the contribution.

If you don't mind let's implement this by changing input: string on run and runStreamed to be of string | UserInput[] type.

where UserInput is

type UserInput = {
   type: "text",
   text: string,
} | {
   type: "local_image",
   path: string
}

We combine all text parts into the prompt and turn all images into --image parameter.

[needs]

Hi! Thanks for your feedback.

The Input type has been extended the suggested way. Texts parts are combined with two newlines. This is done through the new normalizeInput function which returns a combined prompt and a list of image paths.

Added a second test that make sure text parts are combined the expected way.

Available for any follow ups!

[pakrym-oai]

Thank you!