-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Add fast render path for SOGS spherical harmonics #7796
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Co-authored-by: Will Eastcott <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds a fast, approximate SH render path for SOGS by precomputing a full SH evaluation once per frame along the camera’s Z axis and streaming results through a texture, reducing per-gaussian work.
- Registered and integrated new WGSL/GLSL chunks (
gsplatEvalSH,gsplatQuatToMat3) in shader collections. - Implemented
GSplatResolveSHto bake SH in a 2D texture and toggle via a newfastSHflag. - Exposed
fastSHonGSplatComponent/GSplatInstanceand wired through resource, instance, and component layers.
Reviewed Changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/scene/shader-lib/wgsl/collections/... | Added imports/includes for the new WGSL SH and quaternion chunks |
| src/scene/shader-lib/wgsl/chunks/gsplat/... | Added gsplatQuatToMat3, gsplatEvalSH, and replaced inline logic |
| src/scene/shader-lib/glsl/... | Mirrored WGSL changes in GLSL shader collections and chunks |
| src/scene/gsplat/gsplat-resource-base.js | Stubbed evalTextureSize API (returns zero-sized vector) |
| src/scene/gsplat/gsplat-resolve-sh.js | New GSplatResolveSH class for palette texture generation |
| src/scene/gsplat/gsplat-instance.js | Extended GSplatInstance to accept fastSH option and manage resolver |
| src/framework/components/gsplat/system.js | Inserted new fastSH property into the component schema |
| src/framework/components/gsplat/component.js | Added fastSH getter/setter on the component and passed it to instance |
Comments suppressed due to low confidence (1)
src/framework/components/gsplat/system.js:20
- Inserting
fastSHinto the existing schema order shifts property indices and can break serialized scenes. To maintain backwards compatibility, append new properties at the end of_properties.
'fastSH',
Fixes: #7789
This PR adds an option to render SOGS spherical harmonics using a fast and memory efficient, but approximate calculation. Instead of using the gaussian's view vector to evaluate spherical harmonics, we use the camera's Z axis.
Background
SOGS compresses spherical harmonics by constructing a palette of 64K coefficients using k-means and then references these once per gaussian. A scene of 1 million gaussians goes from requiring 45MB of SH data to 5MB.
However at render time, the GPU must access the coefficient data in render order. This results in the GPU accessing the palette data almost randomly, thus killing the memory cache and slowing rendering.
With this PR, instead of evaluating the spherical harmonic once for every gaussian, we evaluate all 64K spherical harmonics once per frame. We evaluate all spherical harmonics using the camera's Z direction, instead of the correct approach of using every gaussian's view vector.
The approximation results in accurate spherical harmonic contribution for gaussians at the center of the screen. Gaussians further from center get less accurate contribution.
API changes
highQualitySHboolean on thegsplatcomponent. This value is false by default.Results
Here is the before and after profile on a pixel 7 pro. 45ms -> 34ms GPU time. The saving is very dependent on the scene being rendered and the device.
And here is a wide angle comparison between high quality and low quality SH: