[Feature Request] Support for Axis-Based Image Stretching

[Devu-trenser]

This feature request is on behalf of FlyWheel.io

What feature or change would you like to see made?

• Add support for a stretch option that allows images to be stretched along specific axes.

Why should we prioritize this feature?

• Provide a more consistent viewing experience across different imaging modalities.

cc: @kgoettler

[sedghi]

we discussed this in office hours i guess?

[wayfarer3130]

I created a separate PR that is related to this one, but not the same. However, both should be done as a single change:
[#2350] describes a bug in the camera positions drifting.
The fix is to have original camera information as one component, and then compute fixed delta in a consistent fashion, so that every update uses the new information as an absolute update rather than being as a specific "set it to this value" which can cause drifts.
When this is done, adding a non uniform anisotropic scaling could be included as additional properties.
Without doing it that way, the existing calculations of focal point/position become problematic as they directly interact with the non uniform scaling.

That is, the scaling should be something like:

scalingDirection = [1,0] (for example to scale x non-uniformly, and leave y identical)
aspectScale = 3

This would give a 3:1 scaling on the x:y direction. The x direction would change scale values separately from the y, ONLY for canvas to index/world coordinates, without any changes in index to world coordinates.

Other linear and non-linear transforms are possible, but it would be very helpful to have an explanation of how they get used as skew isn't a common operation when measurements are included. Not saying it is impossible to use for registration, just tricky.

[Devu-trenser]

@wayfarer3130 I haven’t seen that PR yet. Could you please share the link so I can take a look?

[wayfarer3130]

@wayfarer3130 I haven’t seen that PR yet. Could you please share the link so I can take a look?

Sorry, should have said, proposed change rather than a PR. The idea is to do some of that type of scaling capability, to fix a number of outstanding bugs.

[nithin-trenser]

@wayfarer3130 I’m currently analyzing the feasibility of implementing this feature and will share updates once I have more details.

[nithin-trenser]

@wayfarer3130

To implement stretching, we first tried applying setScale on the actor. With this approach, annotations could also be stretched by making the necessary adjustments to the canvasToWorld and worldToCanvas methods. However, even when scaling was applied to both the image and segmentation actors, the drawn segments did not stretch consistently with the reference image.

Cs3d changed :- main...nithin-trenser:cornerstone3D:feat/stretch
Viewer changes :- OHIF/Viewers@master...nithin-trenser:Viewers:stretch-viewport

As an alternative, we explored overriding vtkCamera’s getProjectionMatrix method and applying the aspectRatio directly in the projection matrix. This approach ensured that both segmentations and annotations stretched together with the reference image, without requiring additional modifications to canvasToWorld and worldToCanvas.
Existing drift comes from repeated changes to position / focalPoint.
This method only modifies the projection transformation once per render, leaving camera values untouched. Therefore, it avoids contributing to the drift issue and keeps zoom/pan/rotate behavior consistent.

For both approaches, the following issues were identified and are expected to be resolved as follows in the second approach:

• Circle annotations that should appear as ellipses in a stretched viewport are still rendered as circles. This is expected to be fixed by treating circle annotations as a special case of ellipses.

• When drawing segmentations, the segmented pixels inside the circular brush cursor are rendered as ellipses instead of maintaining a consistent circular fill. This is expected to be resolved by updating the fill strategies to adapt to the aspect ratio changes.

Could you please confirm this would be a safer approach to implement the stretching with minimal impact and issues?

[Devu-trenser]

We have refined the original feature request, which had a broader scope, to focus specifically on stretch functionality. This adjustment reduces the scope to a more manageable changes

cc: @wayfarer3130 @nithin-trenser @sen-trenser

[wayfarer3130]

We have refined the original feature request, which had a broader scope, to focus specifically on stretch functionality. This adjustment reduces the scope to a more manageable changes

cc: @wayfarer3130 @nithin-trenser @sen-trenser

Do you want just stretch, or do you want skew? If stretch, only in x/y canvas directions, or do you want it in an arbitrary vector?

[Devu-trenser]

We have refined the original feature request, which had a broader scope, to focus specifically on stretch functionality. This adjustment reduces the scope to a more manageable changes
cc: @wayfarer3130 @nithin-trenser @sen-trenser

Do you want just stretch, or do you want skew? If stretch, only in x/y canvas directions, or do you want it in an arbitrary vector?

Just stretch, applied along the X/ Y canvas directions.

[nithin-trenser]

The implementation for this feature is in progress. I’ll open a PR once it’s ready.