Is there a 3rd party widget allowing interactive texture inspection?

[andyborrell]

I'm looking for a widget that allows interactive inspecting of a texture. In a similar way to how ImPlot allows the user to zoom and pan a graph, this would allow the user to zoom and pan a texture in an ImGui window.

Ideally it would have features like:

• Draw grid lines between pixels when the user zooms in more than a certain amount
• Show a tool tip with UV & RGB values of the pixel beneath the cursor
• Auto mapping of float textures to 0-1 range, with the option for the user to tweak the mapping
• A checkbox to show alpha channel as greyscale
• Individual RGB checkboxes to allow individual channels to be inspected

I would love it if something like this exists. If it doesn't exist then I'm thinking of starting it, so let me know if this sounds useful and if you have any other ideas for features.

[IceyChiu]

I'm looking for a widget that allows interactive inspecting of a texture. In a similar way to how ImPlot allows the user to zoom and pan a graph, this would allow the user to zoom and pan a texture in an ImGui window.

Ideally it would have features like:

• Draw grid lines between pixels when the user zooms in more than a certain amount
• Show a tool tip with UV & RGB values of the pixel beneath the cursor
• Auto mapping of float textures to 0-1 range, with the option for the user to tweak the mapping
• A checkbox to show alpha channel as greyscale
• Individual RGB checkboxes to allow individual channels to be inspected

I would love it if something like this exists. If it doesn't exist then I'm thinking of starting it, so let me know if this sounds useful and if you have any other ideas for features.

I'm looking for this widget as you now and found the image inspect tool: https://github.com/CedricGuillemet/imgInspect, but there are something wrong with the magnified pixel's location, i'd be appreciate if you slove this problem and contact me.

[ocornut]

imgInspect was curiously missing from our wiki list, added it now:
https://github.com/ocornut/imgui/wiki/Useful-Extensions#image-manipulation

[andyborrell]

Thanks @IceyChiu & @ocornut. This looks useful!

[ocornut]

To answer your original post: this would certainly be useful, but some of the features may be hard to implement in a way which makes them easy to share between code bases (but figuring out how to do so would largely increase the value of a widget like that).

[andyborrell]

That's a good point. I was only really thinking about how to make it work for OpenGL. imgInspect seems to work by taking a pointer to the texture data, thereby avoiding any API-specific code to access the texture data. I could also implement some of the features I described like that. But that won't work for the channel selection, float value mapping, etc.

I was originally planning on just writing a glsl shader to handle all that. But perhaps to make it more backend-agnostic I could have a per-backend function which accepts an ImGui texture handle and a structure describing the required color transformation and returns a new texture handle. It would then be up to this per-backend code to create the required shader and blit the texture to the new texture. Obviously this per-backend code would be smart enough to only build the shader once, and to reuse blit destination textures from frame to frame.

This approach does introduce an extra blit, but I feel like it would be easier to make work for a range of backends since it separates the code to transform the pixels from the code to render the texture in the UI.

Also this would be similar to the way different backends work in ImGui so I guess people wouldn't find it too strange to work with.

Any thoughts on this approach?

[PathogenDavid]

Any thoughts on this approach?

Personally I think it'd be easier to just implement a shader per-backend.

I'd actually go as far to say that I probably wouldn't use a widget that asked me to implement a CPU-side texture transformation to accomplish this. (If anything but that this sort of thing becomes pretty messy in the modern low-level APIs. You'd also run into perf issues with bigger textures.)

The CPU-centric approach also wouldn't work well for things like render targets since you'd have to download them from the GPU, perform the transformation, and then reupload. It also wouldn't be great if you're using compressed textures. (For both of these reasons, you should probably consider retrieving the RGB values for a specific coordinate would need to be backend-specific as well.)

---

If you really want to avoid requiring a custom shader specific to this widget, maybe let the backend expose which features are supported? For example, filtering out color channels is something that can be accomplished with the graphics APIs directly. (glColorMask, OMSetBlendState.SampleMask, D3D12_RENDER_TARGET_BLEND_DESC.RenderTargetWriteMask, and VkPipelineColorBlendStateCreateInfo.blendConstants to name a few.)

---

Somewhat besides the point, but with a custom shader you could let it deal with the pixel grid drawing as well.

[andyborrell]

@PathogenDavid I didn't mean to do it on the CPU. I just meant there would be a CPU-side function that should be implemented per-backend. Then it's up to that function to build its own shader and use it to transform from one texture to another on the GPU. On further thought it might be better to break this interface into a few functions:

Initialize() // implementation can build shader if needed
AllocateTexture (width, height, channels, type)  // The widget code can manage reuse of textures so this won't be called very often)
Render (src, dst, transformConfig)

Note that these implementations would be included with the widget code so you wouldn't actually need to write anything to use it. (Not saying I would write them all, but hopefully if people found it useful they would add implementations for popular backends)

Your approach of just using a per-backend shader to render the texture directly sounds easier at first, but if I'm not overlooking something it would be hard to integrate into ImGui because there's no way to render an image in ImGui with a custom shader without changing the backend code.

[PathogenDavid]

if I'm not overlooking something it would be hard to integrate into ImGui because there's no way to render an image in ImGui with a custom shader without changing the backend code.

You'll never get a solution that doesn't require at least some backend support, but ImDrawList has the ability to add arbitrary draw-time callbacks specifically for this sort of thing.

Here's a quick and dirty modification of the imgui_impl_opengl3 backend and example_glfw_opengl3 showing it being used to render ImGui widgets with only the red channel active. (I only modified imgui_impl_opengl3 for the sake of simplicity, The modifications could've been outside of it with a little bit of extra effort.)

ImGui::Begin("GH-4135 Shader Swap Demo");
ImVec2 textureSize = ImVec2((float)io.Fonts->TexWidth, (float)io.Fonts->TexHeight);

ImGui::Text("Before changing the shader...");

ImGui::GetWindowDrawList()->AddCallback(ImGui_ImplOpenGL3_SetRedOnlyShader, nullptr);
ImGui::Text("Now the shader is red-only!");
ImGui::Image(io.Fonts->TexID, textureSize);
ImGui::GetWindowDrawList()->AddCallback(ImDrawCallback_ResetRenderState, nullptr);

ImGui::Text("All back to normal!");
ImGui::Image(io.Fonts->TexID, textureSize);
ImGui::End();

Patch for posterity

From 0f236815307b23c217967e1b274b7c20034f72b6 Mon Sep 17 00:00:00 2001
From: David Maas
Date: Tue, 25 May 2021 11:07:30 -0500
Subject: [PATCH] Created shader swap demo for
 https://github.com/ocornut/imgui/issues/4135

---
 backends/imgui_impl_opengl3.cpp        | 50 ++++++++++++++++++++++++++
 backends/imgui_impl_opengl3.h          |  2 ++
 examples/example_glfw_opengl3/main.cpp | 14 ++++++++
 3 files changed, 66 insertions(+)

diff --git a/backends/imgui_impl_opengl3.cpp b/backends/imgui_impl_opengl3.cpp
index 60177042..ab3430b5 100644
--- a/backends/imgui_impl_opengl3.cpp
+++ b/backends/imgui_impl_opengl3.cpp
@@ -160,6 +160,10 @@ static GLuint       g_AttribLocationVtxPos = 0, g_AttribLocationVtxUV = 0, g_Att
 static unsigned int g_VboHandle = 0, g_ElementsHandle = 0;
 static bool         g_HasClipOrigin = false;
 
+static GLuint g_RedOnlyShaderHandle = 0;
+static GLuint g_RedOnlyFragHandle = 0;
+static float g_CurrentProjectionMatrix[4][4];
+
 // Functions
 bool    ImGui_ImplOpenGL3_Init(const char* glsl_version)
 {
@@ -311,6 +315,11 @@ static void ImGui_ImplOpenGL3_SetupRenderState(ImDrawData* draw_data, int fb_wid
         { 0.0f,         0.0f,        -1.0f,   0.0f },
         { (R+L)/(L-R),  (T+B)/(B-T),  0.0f,   1.0f },
     };
+
+    // We need to rebind uniforms when the shader changes, so we save a copy of the projection transform
+    static_assert(sizeof(ortho_projection) == sizeof(g_CurrentProjectionMatrix), "Projection transform sizes must match");
+    memcpy(g_CurrentProjectionMatrix, ortho_projection, sizeof(ortho_projection));
+
     glUseProgram(g_ShaderHandle);
     glUniform1i(g_AttribLocationTex, 0);
     glUniformMatrix4fv(g_AttribLocationProjMtx, 1, GL_FALSE, &ortho_projection[0][0]);
@@ -336,6 +345,13 @@ static void ImGui_ImplOpenGL3_SetupRenderState(ImDrawData* draw_data, int fb_wid
     glVertexAttribPointer(g_AttribLocationVtxColor, 4, GL_UNSIGNED_BYTE, GL_TRUE,  sizeof(ImDrawVert), (GLvoid*)IM_OFFSETOF(ImDrawVert, col));
 }
 
+void ImGui_ImplOpenGL3_SetRedOnlyShader(const ImDrawList*, const ImDrawCmd*)
+{
+    glUseProgram(g_RedOnlyShaderHandle);
+    glUniform1i(g_AttribLocationTex, 0);
+    glUniformMatrix4fv(g_AttribLocationProjMtx, 1, GL_FALSE, &g_CurrentProjectionMatrix[0][0]);
+}
+
 // OpenGL3 Render function.
 // Note that this implementation is little overcomplicated because we are saving/setting up/restoring every OpenGL state explicitly.
 // This is in order to be able to run within an OpenGL engine that doesn't do so.
@@ -645,6 +661,17 @@ bool    ImGui_ImplOpenGL3_CreateDeviceObjects()
         "    Out_Color = Frag_Color * texture(Texture, Frag_UV.st);\n"
         "}\n";
 
+    const GLchar* fragment_shader_glsl_130_red_only =
+        "uniform sampler2D Texture;\n"
+        "in vec2 Frag_UV;\n"
+        "in vec4 Frag_Color;\n"
+        "out vec4 Out_Color;\n"
+        "void main()\n"
+        "{\n"
+        "    Out_Color = Frag_Color * texture(Texture, Frag_UV.st);\n"
+        "    Out_Color.gb = vec2(0, 0);\n"
+        "}\n";
+
     const GLchar* fragment_shader_glsl_300_es =
         "precision mediump float;\n"
         "uniform sampler2D Texture;\n"
@@ -669,6 +696,7 @@ bool    ImGui_ImplOpenGL3_CreateDeviceObjects()
     // Select shaders matching our GLSL versions
     const GLchar* vertex_shader = NULL;
     const GLchar* fragment_shader = NULL;
+    assert(glsl_version == 130); // Backend only updated for the example, which defaults to 130 on desktop OpenGL.
     if (glsl_version < 130)
     {
         vertex_shader = vertex_shader_glsl_120;
@@ -703,6 +731,18 @@ bool    ImGui_ImplOpenGL3_CreateDeviceObjects()
     glCompileShader(g_FragHandle);
     CheckShader(g_FragHandle, "fragment shader");
 
+    const GLchar* red_only_fragment_shader_with_version[2] = { g_GlslVersionString, fragment_shader_glsl_130_red_only };
+    g_RedOnlyFragHandle = glCreateShader(GL_FRAGMENT_SHADER);
+    glShaderSource(g_RedOnlyFragHandle, 2, red_only_fragment_shader_with_version, NULL);
+    glCompileShader(g_RedOnlyFragHandle);
+    CheckShader(g_RedOnlyFragHandle, "red-only fragment shader");
+
+    g_RedOnlyShaderHandle = glCreateProgram();
+    glAttachShader(g_RedOnlyShaderHandle, g_VertHandle);
+    glAttachShader(g_RedOnlyShaderHandle, g_RedOnlyFragHandle);
+    glLinkProgram(g_RedOnlyShaderHandle);
+    CheckProgram(g_RedOnlyShaderHandle, "red-only shader program");
+
     g_ShaderHandle = glCreateProgram();
     glAttachShader(g_ShaderHandle, g_VertHandle);
     glAttachShader(g_ShaderHandle, g_FragHandle);
@@ -715,6 +755,13 @@ bool    ImGui_ImplOpenGL3_CreateDeviceObjects()
     g_AttribLocationVtxUV = (GLuint)glGetAttribLocation(g_ShaderHandle, "UV");
     g_AttribLocationVtxColor = (GLuint)glGetAttribLocation(g_ShaderHandle, "Color");
 
+    // Assert the shaders have compatible signatures
+    assert(g_AttribLocationTex == glGetUniformLocation(g_RedOnlyShaderHandle, "Texture"));
+    assert(g_AttribLocationProjMtx == glGetUniformLocation(g_RedOnlyShaderHandle, "ProjMtx"));
+    assert(g_AttribLocationVtxPos == (GLuint)glGetAttribLocation(g_RedOnlyShaderHandle, "Position"));
+    assert(g_AttribLocationVtxUV == (GLuint)glGetAttribLocation(g_RedOnlyShaderHandle, "UV"));
+    assert(g_AttribLocationVtxColor == (GLuint)glGetAttribLocation(g_RedOnlyShaderHandle, "Color"));
+
     // Create buffers
     glGenBuffers(1, &g_VboHandle);
     glGenBuffers(1, &g_ElementsHandle);
@@ -737,9 +784,12 @@ void    ImGui_ImplOpenGL3_DestroyDeviceObjects()
     if (g_ElementsHandle)   { glDeleteBuffers(1, &g_ElementsHandle); g_ElementsHandle = 0; }
     if (g_ShaderHandle && g_VertHandle) { glDetachShader(g_ShaderHandle, g_VertHandle); }
     if (g_ShaderHandle && g_FragHandle) { glDetachShader(g_ShaderHandle, g_FragHandle); }
+    if (g_RedOnlyShaderHandle && g_RedOnlyFragHandle) { glDetachShader(g_ShaderHandle, g_FragHandle); }
     if (g_VertHandle)       { glDeleteShader(g_VertHandle); g_VertHandle = 0; }
     if (g_FragHandle)       { glDeleteShader(g_FragHandle); g_FragHandle = 0; }
+    if (g_FragHandle)       { glDeleteShader(g_RedOnlyFragHandle); g_RedOnlyFragHandle = 0; }
     if (g_ShaderHandle)     { glDeleteProgram(g_ShaderHandle); g_ShaderHandle = 0; }
+    if (g_RedOnlyShaderHandle) { glDeleteProgram(g_RedOnlyShaderHandle); g_RedOnlyShaderHandle = 0; }
 
     ImGui_ImplOpenGL3_DestroyFontsTexture();
 }
diff --git a/backends/imgui_impl_opengl3.h b/backends/imgui_impl_opengl3.h
index 8c0126d8..abb5bb0a 100644
--- a/backends/imgui_impl_opengl3.h
+++ b/backends/imgui_impl_opengl3.h
@@ -36,6 +36,8 @@ IMGUI_IMPL_API void     ImGui_ImplOpenGL3_DestroyFontsTexture();
 IMGUI_IMPL_API bool     ImGui_ImplOpenGL3_CreateDeviceObjects();
 IMGUI_IMPL_API void     ImGui_ImplOpenGL3_DestroyDeviceObjects();
 
+IMGUI_IMPL_API void ImGui_ImplOpenGL3_SetRedOnlyShader(const ImDrawList*, const ImDrawCmd*);
+
 // Specific OpenGL ES versions
 //#define IMGUI_IMPL_OPENGL_ES2     // Auto-detected on Emscripten
 //#define IMGUI_IMPL_OPENGL_ES3     // Auto-detected on iOS/Android
diff --git a/examples/example_glfw_opengl3/main.cpp b/examples/example_glfw_opengl3/main.cpp
index 78b96423..b8db2a9e 100644
--- a/examples/example_glfw_opengl3/main.cpp
+++ b/examples/example_glfw_opengl3/main.cpp
@@ -199,6 +199,20 @@ int main(int, char**)
             ImGui::End();
         }
 
+        ImGui::Begin("GH-4135 Shader Swap Demo");
+        ImVec2 textureSize = ImVec2((float)io.Fonts->TexWidth, (float)io.Fonts->TexHeight);
+
+        ImGui::Text("Before changing the shader...");
+
+        ImGui::GetWindowDrawList()->AddCallback(ImGui_ImplOpenGL3_SetRedOnlyShader, nullptr);
+        ImGui::Text("Now the shader is red-only!");
+        ImGui::Image(io.Fonts->TexID, textureSize);
+        ImGui::GetWindowDrawList()->AddCallback(ImDrawCallback_ResetRenderState, nullptr);
+
+        ImGui::Text("All back to normal!");
+        ImGui::Image(io.Fonts->TexID, textureSize);
+        ImGui::End();
+
         // Rendering
         ImGui::Render();
         int display_w, display_h;
-- 
2.31.1.windows.1

[andyborrell]

You'll never get a solution that doesn't require at least some backend support

I'm not sure this is true.

The great thing about widgets like imgInspect and ImPlot is that they don't require backend changes (or any changes to ImGui code at all).

Admittedly my proposal is kind of a backend "extension" but it's not a change to ImGui code or its included backends. I don't think your changes would be accepted since in my opinion this isn't something that should be a built-in part of ImGui and would be more appropriate as a separate project.

[PathogenDavid]

I don't think your changes would be accepted since in my opinion this isn't something that should be a built-in part of ImGui and would be more appropriate as a separate project.

That is not what I am proposing, that would be ludicrous.

Please re-read my comment, specifically this part:

(I only modified imgui_impl_opengl3 for the sake of simplicity, The modifications could've been outside of it with a little bit of extra effort.)

[ocornut]

It’s not an easy thing to solve but you could at least probably provide compile-time opt-in implementations for a few api (eg OpenGL and DX11) that may be used as-is, and maybe your widget can work without them and just have a few features disabled. At the end of the day better start making something you can use yourself (while keeping sharability in mind) and see where it goes.

[andyborrell]

@PathogenDavid after rereading I can see that your patch is indeed quite helpful. It seems like the only annoying thing, in the OpenGL implementation at least, is that if that code was outside of imgui_impl_opengl3 it would need to duplicate the code to calculate the projection matrix. I guess that's probably not a big deal though.

[PathogenDavid]

it would need to duplicate the code to calculate the projection matrix. I guess that's probably not a big deal though.

If you really wanted to avoid it you could use glGetUniform before changing the shader. But yeah, it's pretty simple to just re-create it.