Interpolation

This plugin allows you to "interpolate" between two text prompts and see the results in the UI.

Given a left_prompt, a right_prompt, and an interpolation strength alpha, the plugin will embed the left and right prompts into $v_l$ and $v_r$ and then sort by similarity on similarity index with selected brain key, with the query

$$v = \alpha v_l + (1 - \alpha) v_r$$

Note: This plugin requires a similarity index that supports prompts (i.e. embeds text and images) to be present on the dataset. You can create one with:

import fiftyone as fo
import fiftyone.brain as fob

dataset = fo.load_dataset("my_dataset")
fob.compute_similarity(
    dataset,
    brain_key="my_brain_key",
    model_name="clip-vit-base32-torch",
    metric="cosine",
    )

Watch On Youtube

Installation

fiftyone plugins download https://github.com/jacobmarks/concept-interpolation

Refer to the main README for more information about managing downloaded plugins and developing plugins locally.

Operators

open_interpolation_panel

• Opens the interpolation panel on click
• Only activated when the dataset has a similarity index

interpolator

• Runs the interpolation