OrbitControls: Better TouchPad support.

[philippb]

Related problem

We're coming from OrbitControls which work with the mouse and a touch screen (like iPad/iPhone).
OrbitControls (and other controls) don't work with touchpad (mac book). This is quite unintuitive for people who use touchpad a lot. Multiple big applications like Figma support two finger zoom with the touchpad.

Currently two finger pan zooms in the entire browser window as three-js does not recognize the events and the browser takes over.

Solution

Describe the solution you'd like

We would like to add native touchpad support to OrbitControls. In specific

• Two finger pan orbit controls
• Two finger pinch to zoom
• Two finger rotate
• Command ⌘ + two finger pan == zoom
• Option/alt + two finger pan == pan (vs currently ⌘+Prim mouse down, 1 finger pan)

Key bindings

Mac	Windows
Command ⌘	Control
Option⌥	Alt

This is behavior similar to Blender, except for added pan functionality with option + two finger

Describe alternatives you've considered

The alternative is to have touchpad users use the application like if they have a mouse and not offer native touchpad support.

Tasks

• Agree with project maintainers on a API
• Evaluate touchpad events for cross browser functionality
• Implement

Implementation plan

We aim to implement this feature ourselves. We're filing a GitHub issue to be able to communicate about the implementation details with the projects core maintainers.

[CodyJasonBennett]

You might want to see #21989.

Live link:

• https://raw.githack.com/danielefornari/three.js/dev/examples/misc_controls_arcball.html

[philippb]

Thank you @CodyJasonBennett , I've played around with the live link you provided. All the multi touch gestures that we're looking for and know from Figma and other 3D software does not seem to work.
I get the idea of the Archball, I just think it's something different than what we're looking for.

This may or may not be applicable for your use case, but you might consider giving Akihiro Oyamada's camera controls a try. I'm using it and have found that it has fantastic touch capabilities with advanced camera controls that are highly customizable. It uses the pointer events API, and works well on touch devices (tested on an iPad Pro 2018 and Windows 11 touchscreen).

[Saquib764]

@philippb It is possible to implement Pan and zoom features using only Trackpad gesture. Currently trackpad emits wheel event on two finger pan and pinch zoom (for pinch zoom, ctrlKey = true). This event appears to be supported in all major browser, except Safari. However, Safari have an additional gesture events which can be used to implement pan and zoom feature with consistent user experience.

Useful post regarding this - https://kenneth.io/post/detecting-multi-touch-trackpad-gestures-in-javascript

Apple's gesture event - https://developer.mozilla.org/en-US/docs/Web/API/GestureEvent

---

The approach is not a standard one. However, I checked the implementation on design tools like Figma, draw.io, and Miro... they appear to be using the approach mentioned above. Therefore, perhaps same can be implemented in here.
@Mugen87 @natarius

[ansonkao]

Just wanted to add that the proposed touchpad behaviour is the default for Blender, and IMO very intuitive and discoverable for new users, and reducing the need to learn hotkeys or toggle between tools.

If what I'm working on now using THREE takes off I would be interested in contributing! TBD

Also, linking relevant hacks from SO: https://stackoverflow.com/questions/60678494/orbit-controls-follow-the-mouse-without-clicking-three-js

[LeviPesin]

May be related: #24215 (merged but then reverted)

[deaconudanandrei]

For anyone looking for a solution, here's a custom CameraController React component with the necessary calculations for panning and zooming using wheel evens from a trackpad.

import { useEffect } from "react";
import { useThree } from "@react-three/fiber";


const CameraController = () => {
	const { camera, gl, size } = useThree();
	const zoomSpeed = 1.05; // Sensitivity of zoom, adjust as needed
	const minimumScale = 2;
	const maximumScale = 6400;
	const panSpeed = 0.5; // Adjust pan sensitivity as needed

	useEffect(() => {
		const handleWheel = event => {
			event.preventDefault();

			const rect = gl.domElement.getBoundingClientRect();
			const pointerX = event.clientX - rect.left; // Mouse x position within the canvas
			const pointerY = event.clientY - rect.top; // Mouse y position within the canvas

			const oldZoom = camera.zoom;
			let newZoom = oldZoom;

			if (event.ctrlKey) {
				// Calculate new zoom
				if (event.deltaY > 0) {
					newZoom = oldZoom / zoomSpeed;
				} else if (event.deltaY < 0) {
					newZoom = oldZoom * zoomSpeed;
				}
				newZoom = Math.max(Math.min(Math.round(newZoom * 100) / 100, maximumScale), minimumScale);

				// Calculate the factors needed to adjust the camera position
				const zoomFactor = newZoom / oldZoom;
				const midpointX = (pointerX / size.width) * 2 - 1;
				const midpointY = -(pointerY / size.height) * 2 + 1;

				// Calculate world space coordinates for the pointer
				const dx = (midpointX * (camera.right - camera.left)) / 2 / oldZoom;
				const dy = (midpointY * (camera.top - camera.bottom)) / 2 / oldZoom;

				// Adjust camera position based on the zoom factor and pointer position in world space
				camera.position.x -= dx * (1 - zoomFactor);
				camera.position.y -= dy * (1 - zoomFactor);

				// Apply the new zoom
				camera.zoom = newZoom;
			} else {
				// Pan handling
				camera.position.x += event.deltaX * panSpeed * (1 / camera.zoom);
				camera.position.y -= event.deltaY * panSpeed * (1 / camera.zoom);
			}

			camera.updateProjectionMatrix();
		};

		gl.domElement.addEventListener("wheel", handleWheel, { passive: false });

		return () => {
			gl.domElement.removeEventListener("wheel", handleWheel);
		};
	}, [camera, gl.domElement, size.width, size.height]);

	return null;
};

export default CameraController;