Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 91 additions & 7 deletions docs/contributing/mcp-apps-architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ The injected script provides the MCP Apps API to widget code:
window.mcpApp = {
toolInput: { ... }, // Tool input arguments
toolResult: { ... }, // Tool execution result
hostContext: { ... }, // Host context (theme, locale, etc.)
hostContext: { ... }, // Host context (theme, locale, timezone, etc.)

// Call another MCP tool
async callTool(name, args = {}) { ... },
Expand All @@ -205,13 +205,54 @@ window.mcpApp = {
};
```

### Host Context

The `hostContext` object provides information about the host environment (as of PR #1038):

```typescript
{
theme: "light" | "dark",
displayMode: "inline" | "pip" | "fullscreen",
availableDisplayModes: ["inline", "pip", "fullscreen"],
viewport: {
width: number, // Current viewport width in pixels
height: number, // Current viewport height in pixels
maxHeight: number // Maximum available height
},
locale: string, // BCP 47 locale (e.g., "en-US", "ja-JP")
timeZone: string, // IANA timezone (e.g., "America/New_York")
platform: "web" | "desktop" | "mobile",
userAgent: string, // Browser user agent string
deviceCapabilities: {
hover: boolean, // Whether hover interactions are supported
touch: boolean // Whether touch input is supported
},
safeAreaInsets: {
top: number, // Safe area inset in pixels
right: number,
bottom: number,
left: number
},
toolInfo: {
id: string, // Tool call ID
tool: {
name: string,
inputSchema: object,
description?: string
}
}
}
```

Widgets receive the full host context during initialization via the `ui/initialize` response. When context values change (e.g., theme toggle, display mode change), the host sends a `ui/notifications/host-context-changed` notification with only the changed fields.

### Events

Widgets can listen for these events:

- `mcp:tool-input` - Tool input received
- `mcp:tool-result` - Tool result received
- `mcp:context-change` - Host context changed (theme, etc.)
- `mcp:context-change` - Host context changed (theme, display mode, locale, timezone, etc.)
- `mcp:tool-cancelled` - Tool was cancelled
- `mcp:teardown` - Widget is about to be torn down

Expand Down Expand Up @@ -271,21 +312,64 @@ Serves the sandbox proxy HTML that creates the double-iframe architecture.

### Content Security Policy

The sandbox proxy uses a permissive CSP to allow widget content:
As of PR #1038, MCP Apps supports two CSP enforcement modes:

**Permissive Mode** (default) - Allows all HTTPS resources for development and testing:

```html
<meta
http-equiv="Content-Security-Policy"
content="
default-src * 'unsafe-inline' 'unsafe-eval' data: blob: filesystem: about:;
script-src * 'unsafe-inline' 'unsafe-eval' data: blob:;
style-src * 'unsafe-inline' data: blob:;
img-src * data: blob: https: http:;
media-src * data: blob: https: http:;
connect-src * data: blob: https: http: ws: wss: about:;
...
"
/>
```

**Widget-Declared Mode** - Enforces CSP based on widget metadata (SEP-1865):

```html
<meta
http-equiv="Content-Security-Policy"
content="
default-src 'self';
script-src 'self' 'unsafe-inline' 'unsafe-eval' blob: data: https://cdn.tailwindcss.com ...;
style-src * blob: data: 'unsafe-inline';
connect-src *;
script-src 'self' 'unsafe-inline';
style-src 'unsafe-inline' https://example.com;
img-src data: https://example.com;
media-src data: https://example.com;
connect-src https://api.example.com;
...
"
/>
```

The CSP mode is controlled via the playground toolbar (shield icon) and passed to the backend when fetching widget content. CSP violations are captured and reported to the host for debugging.

**CSP Violation Reporting:**

When a widget violates CSP rules, the sandbox proxy captures the `securitypolicyviolation` event and forwards it to the host:

```javascript
document.addEventListener('securitypolicyviolation', function(e) {
window.parent.postMessage({
type: 'mcp-apps:csp-violation',
directive: e.violatedDirective,
blockedUri: e.blockedURI,
sourceFile: e.sourceFile,
lineNumber: e.lineNumber,
effectiveDirective: e.effectiveDirective,
timestamp: Date.now()
}, '*');
});
```

Violations are displayed in the CSP debug panel with suggested fixes for the widget's metadata.

### Iframe sandbox attributes

```html
Expand All @@ -297,7 +381,7 @@ The sandbox proxy uses a permissive CSP to allow widget content:
- Double-iframe provides origin isolation
- `allow-same-origin` required for localStorage access
- Widgets should be treated as semi-trusted code
- CSP headers restrict network access
- CSP enforcement helps restrict network access in production

## Related files

Expand Down
45 changes: 39 additions & 6 deletions docs/contributing/playground-architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -688,14 +688,18 @@ const metadata = mcpClientManager.getAllToolsMetadata("server1");
// Returns: { tool1: { _meta: {...} }, tool2: { _meta: {...} } }
```

### Device Globals for ChatGPT Apps
### Device Globals for ChatGPT Apps and MCP Apps

As of PR #1026, the playground provides device context to ChatGPT Apps and MCP Apps through configurable device globals. These settings are only applied when the UI Playground is active; outside the playground, defaults are used.
As of PR #1026 and #1038, the playground provides device context to ChatGPT Apps and MCP Apps through configurable device globals. These settings are only applied when the UI Playground is active; outside the playground, defaults are used.

**Device Settings:**

- **Device Type** - `'mobile'`, `'tablet'`, or `'desktop'`. Controlled by the device type selector in the playground toolbar. Outside the playground, automatically detected from window size via `getDeviceType()`.

- **Locale** - BCP 47 locale code (e.g., `'en-US'`, `'ja-JP'`). Controlled by the locale selector in the playground toolbar. Outside the playground, defaults to `navigator.language`.

- **Timezone** (MCP Apps only) - IANA timezone identifier (e.g., `'America/New_York'`, `'Asia/Tokyo'`). Controlled by the timezone selector in the playground toolbar. Outside the playground, defaults to `Intl.DateTimeFormat().resolvedOptions().timeZone`.

- **Device Capabilities** - Input method support:
- `hover` (boolean) - Whether hover interactions are supported
- `touch` (boolean) - Whether touch input is supported
Expand All @@ -707,24 +711,53 @@ As of PR #1026, the playground provides device context to ChatGPT Apps and MCP A

Configured through the safe area editor with presets for common devices (iPhone notch, Dynamic Island, Android gesture navigation). Outside the playground, all insets default to 0.

**Protocol Detection:**

The playground automatically detects which app protocol is in use based on tool metadata:

- **ChatGPT Apps** - Tools with `openai/outputTemplate` metadata
- **MCP Apps** - Tools with `ui/resourceUri` metadata
- **Mixed/None** - Shows ChatGPT Apps controls by default

The toolbar adapts to show protocol-specific controls (e.g., timezone selector only appears for MCP Apps).

**Implementation:**

The playground store (`ui-playground-store.ts`) maintains these settings and provides them to the ChatGPT app renderer. When widgets are initialized, these values are passed through the widget data store and injected into the `window.openai` API:
The playground store (`ui-playground-store.ts`) maintains these settings and provides them to the appropriate renderer:

**ChatGPT Apps** - Values passed via `window.openai` API:

```typescript
// Passed to widget via window.openai
{
userAgent: {
device: { type: deviceType },
capabilities: { hover, touch }
},
safeArea: {
insets: { top, bottom, left, right }
}
},
locale: "en-US"
}
```

**MCP Apps** - Values passed via `ui/initialize` host context:

```typescript
{
theme: "dark",
displayMode: "inline",
availableDisplayModes: ["inline", "pip", "fullscreen"],
viewport: { width: 430, height: 932, maxHeight: 932 },
locale: "en-US",
timeZone: "America/New_York",
platform: "mobile", // Derived from device type
userAgent: navigator.userAgent,
deviceCapabilities: { hover: false, touch: true },
safeAreaInsets: { top: 44, right: 0, bottom: 34, left: 0 }
}
```

Widgets can use these values to adapt their layout and interactions for different device contexts. For example, disabling hover-based tooltips on touch devices or adjusting padding for safe area insets.
Widgets can use these values to adapt their layout and interactions for different device contexts. For example, disabling hover-based tooltips on touch devices, adjusting padding for safe area insets, or formatting dates/times according to the user's timezone.

### Server Instructions Integration

Expand Down
60 changes: 51 additions & 9 deletions docs/inspector/llm-playground.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -212,21 +212,28 @@ These settings are automatically passed to widgets via the `window.openai` API,
When viewing tool results with custom UI components, you can access debugging information using the icon buttons in the tool header:

- **Data** (database icon) - View tool input, output, and error details
- **Widget State** (box icon) - Inspect the current widget state and see when it was last updated
- **Globals** (globe icon) - View global values passed to the widget (theme, display mode, locale, etc.)
- **Widget State** (box icon) - Inspect the current widget state and see when it was last updated (ChatGPT Apps only)
- **CSP** (shield icon) - View CSP violations and suggested fixes for both ChatGPT Apps and MCP Apps

Click any icon to toggle the corresponding debug panel. Click again to close it.

<Note>
The Widget State tab only appears for ChatGPT Apps (OpenAI SDK). MCP Apps
(SEP-1865) do not support persistent widget state.
</Note>

### Content Security Policy (CSP)

The UI Playground includes CSP enforcement controls to help you test widget security configurations. You can switch between two CSP modes using the shield icon in the toolbar:

- **Permissive** (default) - Allows all HTTPS resources, suitable for development and testing
- **Strict** - Only allows domains declared in the widget's `openai/widgetCSP` metadata field
- **Widget-Declared** - Only allows domains declared in the widget's CSP metadata field

When a widget violates CSP rules in strict mode, you'll see a badge on the CSP debug tab showing the number of blocked requests. Click the CSP tab to view:
The CSP mode applies to both ChatGPT Apps (`openai/widgetCSP`) and MCP Apps (`ui/csp` per SEP-1865).

- **Suggested fix** - Copyable JSON snippet to add to your widget's `openai/widgetCSP` field
When a widget violates CSP rules in widget-declared mode, you'll see a badge on the CSP debug tab showing the number of blocked requests. Click the CSP tab to view:

- **Suggested fix** - Copyable JSON snippet to add to your widget's CSP metadata field
- **Blocked requests** - List of all CSP violations with directive and URI details
- **Declared domains** - The `connect_domains` and `resource_domains` your widget currently declares

Expand All @@ -239,11 +246,40 @@ This helps you identify which external resources your widget needs and configure

## UI Playground device settings

The UI Playground provides controls to simulate different device environments for testing ChatGPT Apps and MCP Apps. These settings affect how widgets receive device context through the `window.openai` API.
The UI Playground provides controls to simulate different device environments for testing ChatGPT Apps and MCP Apps. These settings affect how widgets receive device context.

### Protocol-aware controls

The playground automatically detects which app protocol is in use and shows appropriate controls:

- **ChatGPT Apps** - Tools with `openai/outputTemplate` metadata
- **MCP Apps** - Tools with `ui/resourceUri` metadata

Some controls (like timezone) are specific to MCP Apps and only appear when an MCP Apps widget is active.

### Device type

Select between mobile, tablet, or desktop device types. This affects the `window.openai.userAgent.device.type` value that widgets receive. The device type selector is located in the playground toolbar.
Select between mobile, tablet, or desktop device types. This affects the device context that widgets receive:

- **ChatGPT Apps**: `window.openai.userAgent.device.type`
- **MCP Apps**: `hostContext.platform` (derived as "mobile", "web", or "desktop")

The device type selector is located in the playground toolbar.

### Locale

Choose from common BCP 47 locales (e.g., en-US, ja-JP, es-ES) to test internationalization. Available for both ChatGPT Apps and MCP Apps:

- **ChatGPT Apps**: `window.openai.locale`
- **MCP Apps**: `hostContext.locale`

### Timezone (MCP Apps only)

Select an IANA timezone identifier (e.g., America/New_York, Asia/Tokyo) to test timezone-aware widgets. This control only appears when testing MCP Apps:

- **MCP Apps**: `hostContext.timeZone`

The timezone selector includes common zones with UTC offset information for easy reference.

### Device capabilities

Expand All @@ -252,7 +288,10 @@ Toggle hover and touch capabilities to simulate different input methods:
- **Hover** - Indicates whether the device supports hover interactions (typically enabled for desktop, disabled for mobile)
- **Touch** - Indicates whether the device supports touch input (typically enabled for mobile/tablet, disabled for desktop)

These settings are reflected in `window.openai.userAgent.capabilities.hover` and `window.openai.userAgent.capabilities.touch`.
These settings are reflected in:

- **ChatGPT Apps**: `window.openai.userAgent.capabilities.hover` and `window.openai.userAgent.capabilities.touch`
- **MCP Apps**: `hostContext.deviceCapabilities.hover` and `hostContext.deviceCapabilities.touch`

### Safe area insets

Expand All @@ -266,7 +305,10 @@ Configure safe area insets to simulate device notches, rounded corners, and gest
- **Android** - Android gesture navigation (24px top, 16px bottom)
- **Custom values** - Manually adjust top, bottom, left, and right insets in pixels

Widgets receive these values through `window.openai.safeArea.insets` and can use them to adjust their layout accordingly.
Widgets receive these values through:

- **ChatGPT Apps**: `window.openai.safeArea.insets`
- **MCP Apps**: `hostContext.safeAreaInsets`

#### Fullscreen navigation

Expand Down
Loading