User option to enable/disable out-of-focus live previews

[Artheriax]

Description

I've been triggered a lot lately by the late and buggy way live previews show up, sometimes not even at all, also it made my gens laggy, sometimes gens would get stuck, so I made it step based, changeable in settings too in case you want the live preview to generate every 2, 3 or 4 and etc... I did the same to the progress tracking and added a fallback in case the per step based progress tracking fails.

Technical details

The live preview system previously relied on frontend HTTP polling every 500ms, the backend would decode latents and JPEG-encode on every poll, regardless of whether the sampling step had changed. This caused unnecessary GPU work (same latent decoded 10-40x per step), increased latency and made previews feel sluggish or sometimes fail to appear (what I said above).

This PR replaces the polling architecture with a step-driven push model:

Preview images are now generated at sampling step boundaries (configurable via Show live preview every N steps) and pushed immediately via WebSocket (/ws/preview), no more waiting for the next poll cycle or redundant re-decoding.

Progress tracking is also step-based, pushed on every callback step through the same WebSocket channel. HTTP polling is retained as a low-frequency fallback (2s default) for timeout/completion detection if the WebSocket connection drops.

A step-level latch in do_set_current_image() prevents re-decoding the same latent on subsequent polls, eliminating the primary source of wasted GPU time.

The base64 image data URI is now cached in assign_current_image() rather than re-encoded on every HTTP request.

Testing

As for testing, I've tested it a lot with different settings and with things like detailer for example, also the performance is great, something I instantly noticed during gens.

Personal note

As always feel free to comment, I learn from people like you :)

Image

[Artheriax]

I'll fix problems tomorrow

[QualiaRain]

Edit (clarification): this comment was written against an earlier revision of this PR (commit d6b47591, since force-pushed away), where the preview code lived in modules/progress.py / modules/shared_state.py and push_live_preview() re-encoded an image that assign_current_image() had already cached. The current rewrite restructured all of this into modules/api/preview.py with a single-encode push_step(), so the specific code referenced below no longer exists in the PR. Leaving the original text for thread history.

---

Nice! I had Claude take a look, I hope this is of use to you.

1. Double JPEG encode per step (the important one). assign_current_image() already encodes the preview to a JPEG data-URI and caches it in shared.state.live_preview_b64. Then push_live_preview() re-encodes the same image a second time. With the default show_progress_every_n_steps = 1 that's two full JPEG encodes every sampling step, synchronously inside the diffusers callback - so it adds latency to the gen loop instead of removing it. Reusing the cached data-URI fixes it:

def push_live_preview(image, step: int, steps: int, progress: float):
    try:
        b64 = shared.state.live_preview_b64  # already encoded in assign_current_image; avoids a 2nd JPEG encode per step
        if b64 is None:
            buffered = io.BytesIO()
            image.save(buffered, format='jpeg', quality=60)
            b64 = f'data:image/jpeg;base64,{base64.b64encode(buffered.getvalue()).decode("ascii")}'
        data = {
            'type': 'preview',
            'live_preview': b64,
            'id_live_preview': shared.state.id_live_preview,
            'step': step,
            'steps': steps,
            'progress': progress,
            'job': shared.state.job,
        }
        preview_manager.push(data)
    except Exception as e:
        debug_log(f'Preview push error: {e}')

(live_preview_b64 already carries the data:image/jpeg;base64, prefix, so no double-prefixing.)

2. WebSocket reconnect edge (minor). In progressBar.ts, ws.onclose only reconnects if (pollingTimer !== undefined). If polling is ever disabled (live_preview_refresh_period = 0), pollingTimer is never set, so a dropped socket never reconnects and previews stop for the rest of the session. Gating on an explicit ended flag keeps it reconnecting while the task is live:

// with the other `let` declarations:
let ended = false;

// first line inside removeLivePreview:
const removeLivePreview = (ok = false) => {
  ended = true;
  // ...

// in ws.onclose:
ws.onclose = () => {
  debug('ws', 'disconnected');
  ws = null;
  if (!ended) {
    wsReconnect = window.setTimeout(connectWebSocket, 3000);
  }
};

Both are static-analysis findings (ruff-clean, compiles) - not run against a live server.

[vladmandic]

using websockets for preview is not inherently a bad idea, but premise how current polling works is inherently wrong.
polling period triggers checks by client, but it does not mean that server will encode and resend same image every single time, there is quite a lot of caching happening and if there are no changes, server will just say "hey, i don't have anything new for you". polling also allows for client to selectively stop asking server, e.g. if your sdnext tab in browser is inactive or you're doing something else its intelligent enough to stop polling until it comes into focus.

i'm open to adding websocket push method, but this pr changes too much of existing infrastructure for it to be a viable thing.

now, having said that, if there are bugs in existing polling based live preview, lets find and fix those.

[Artheriax]

using websockets for preview is not inherently a bad idea, but premise how current polling works is inherently wrong. polling period triggers checks by client, but it does not mean that server will encode and resend same image every single time, there is quite a lot of caching happening and if there are no changes, server will just say "hey, i don't have anything new for you". polling also allows for client to selectively stop asking server, e.g. if your sdnext tab in browser is inactive or you're doing something else its intelligent enough to stop polling until it comes into focus.

i'm open to adding websocket push method, but this pr changes too much of existing infrastructure for it to be a viable thing.

now, having said that, if there are bugs in existing polling based live preview, lets find and fix those.

I'll change the PR later today

[Artheriax]

WebSocket/Polling hybrid implementation

Now you can choose between WebSocket and Polling for live previews.
I did not manage to find fixes for the polling method, but as far as I've tested, WebSocket seems to work better for me.

Visibility detection is built into the WebSocket path too. When you switch tabs, the client tells the server to skip JPEG encoding, so bandwidth drops to near-zero when you're not watching. For people actively watching their progress, WebSocket is superior.

New settings

Settings → Live Previews

Setting	Default	Component	Description
live_preview_transport	"Polling"	gr.Dropdown	Switch between "Polling" and "WebSocket"
live_preview_ws_interval	1	gr.Slider (1–20)	Send a preview frame every N denoising steps (only works when WebSocket is enabled

Changes

New: modules/api/preview.py

• PreviewManager class - manages WebSocket connections, receives visibility-change messages from the client, and provides push_step() for the generation thread and push_complete() for job-end notification
• JPEG encode + JSON progress dispatch cross-thread via asyncio.run_coroutine_threadsafe() - no base64, no HTTP overhead, binary frames only when id_live_preview changes
• ws_preview() endpoint handler - accept, listen for {"type":"visibility", "visible": true|false}, handle disconnect

Modified: modules/api/api.py

• Registers ws://host/sdapi/v1/preview WebSocket endpoint (with subpath support)

Modified: modules/processing_callbacks.py

• Hook in diffusers_callback() (line 234): when transport is WebSocket, calls preview_manager.push_step() every N steps - zero polling, step-driven push from the generation thread

Modified: modules/processing.py

• push_complete() at the end of process_images_inner() (line 579) so clients receive completed: true and cleanly disconnect

Modified: ui/progressBar.ts

• startHttpPolling() - extracted HTTP polling path, unchanged behavior, still the default
• startLivePreviewWebSocket() - WebSocket client with:
  • Binary JPEG blob handling via URL.createObjectURL
  • JSON progress frames feed the same onProgressHandler as polling
  • visibilitychange listener sends visibility to server > skips JPEG encode when tab hidden
  • 1-second fallback timeout reverts to HTTP polling if WebSocket fails to open
  • URL revocation on each new frame to prevent memory leaks

Wire protocol

Client → Server:  {"type":"visibility","visible":true|false}
Server → Client:  {"active":true,"step":5,"steps":20,"progress":0.25,"job":"txt2img","paused":false,"has_image":true}
Server → Client:  <binary JPEG bytes>  (only when has_image:true)
Server → Client:  {"active":false,"step":0,"steps":0,"progress":1.0,"job":"","paused":false,"completed":true,"has_image":false}

Thread safety

Operation	Thread	Notes
push_step() / push_complete()	Generation thread (holds queue_lock)	Encodes JPEG, dispatches via run_coroutine_threadsafe()
_broadcast()	Event loop thread	Sends JSON + binary WebSocket frames
handle_message()	Event loop thread	Sets atomic _client_hidden flag

Backward compatibility

• /internal/progress HTTP endpoint preserved unchanged
• Polling remains the default transport (live_preview_transport = "Polling")
• Extensions and CLI scripts continue to work via the existing HTTP endpoints
• All existing settings (live_preview_refresh_period, show_progress_type, etc.) unaffected

[vladmandic]

this is not "live preview fixes", this is a completly new method for live preview and it's has its own pros AND cons.

before even considering, I have to ask why - what exactly is the problem with current live preview and how does this solve it. cant just say "I had problems and now this works".

[Artheriax]

this is not "live preview fixes", this is a completly new method for live preview and it's has its own pros AND cons.

before even considering, I have to ask why - what exactly is the problem with current live preview and how does this solve it. cant just say "I had problems and now this works".

Whenever I'm generating, there is a delay with polling rate implemention, sometimes the steps get stuck or it doesn't update the live preview until it approaches 90% of the first pass which most of the time is too late for it to cancel it when I'm not happy with the preview result and then it already starts with hires fix or detailer and takes longer for me to cancel. You also said polling implementation pauses when not focused on the tab, well I have multiple monitors and most of the time I'm doing something else while I wait for the gen and then the preview doesn't show where the image or video is at, which would be pretty handy for me to have. Whenever I try a other UI like reforged, forge neo, comfy and etc... they always show the progress since the first step of the gen and does it each step or each 2 steps, while the SD.NEXT implementation is inconsistent on my end. When I tested the WebSocket implementation it worked like the other UIs, instant and not affecting generation time like I used to get, I sent 2 video's to you on discord, so that you can see what I mean.

[vladmandic]

that's why i asked for description of the issue. now lets separate the stories:
http polling vs websocket push has no impact on your issue, its a side-effect of your implementation, not the fact that its websocket or push.

• original polling does preview vae decode in a separate python loop which means if primary loop is saturated, vae preview will always run with much lower priority
• your push approach runs decode in the main loop and only uses secondary runner for actual push so its not blocking. but thats after vae decode already happened.

as a result, your approach gives more deterministic preview timings, but has far bigger impact on actual generation time. thus i said "it has pros and cons"
and yes, it also experiences delays sometimes because of required torch sync when handing off latents to a different model. one solution for that would be to explicitly call torch sync in callback, but that would slow down main loop even further.

now, regarding showing preview while not-in-focus, that can be solved with a user option regardless of if its push or poll.

ideally, we should be using torch streams instead of relying on python runners, but i've been reluctant to do so due to streams compatibility issues with different gpu architectures.

going back on polling - its cheap and commonly used. even if you set polling to 10ms, it should not have noticeable impact. its far from original statement that it runs decode on every poll.

[Artheriax]

that's why i asked for description of the issue. now lets separate the stories: http polling vs websocket push has no impact on your issue, its a side-effect of your implementation, not the fact that its websocket or push.

• original polling does preview vae decode in a separate python loop which means if primary loop is saturated, vae preview will always run with much lower priority
• your push approach runs decode in the main loop and only uses secondary runner for actual push so its not blocking. but thats after vae decode already happened.

as a result, your approach gives more deterministic preview timings, but has far bigger impact on actual generation time. thus i said "it has pros and cons" and yes, it also experiences delays sometimes because of required torch sync when handing off latents to a different model. one solution for that would be to explicitly call torch sync in callback, but that would slow down main loop even further.

now, regarding showing preview while not-in-focus, that can be solved with a user option regardless of if its push or poll.

ideally, we should be using torch streams instead of relying on python runners, but i've been reluctant to do so due to streams compatibility issues with different gpu architectures.

going back on polling - its cheap and commonly used. even if you set polling to 10ms, it should not have noticeable impact. its far from original statement that it runs decode on every poll.

Alright, Now I get it, I'll go look for adding that user option to enable/disable not-in-focus previews

[vladmandic]

i'm ok with adding this option to setting, but dist/* files should not be part of the pr - i will run a rebuild on my end, otherwise we have too much churn.

[Artheriax]

i'm ok with adding this option to setting, but dist/* files should not be part of the pr - i will run a rebuild on my end, otherwise we have too much churn.

Undid changes to dist files

[vladmandic]

lgtm, i'll merge soon, cant today/tomorrow as i'm out