Marker tracking in WebXR

[m-alkalbani]

There have been a few requests for WebXR to support marker and image tracking in the past few years, especially with a proposal already in place for marker tracking.

Looking at previous issues, the main request from developers is to be able to track a physical marker in a WebXR experience (be that an image, a fiducial marker or QR codes) in a way that is more robust than current open source solutions such ARToolKit, MindAR etc, and less costly than premium solutions. It might be valuable to revisit this to discuss the following:

• distinguish between fiducial, image and QR code markers in the context of the proposal in place
• revisit standardisation challenges discussed in the past (are they still there?)
• discuss support for QR code scanning in WebXR (can that be a potential starting point that works across devices/browsers? would that need an entirely different proposal?)

/facetoface

[Maksims]

A significant "hold back" on current implementation, is that it requires providing images in advance (on session start).
An ability to manage tracked images at runtime, instead of on session start - would open up much more possibilities.

Also, QR Markers - there are generic standards, that allows to have a CV system that can dynamically identify them, and track them. Providing it's decoded value (as an event when first found?) and transform data on frame updates. This would open up so many more possibilities for developers.

[ryanbarkernz]

Looks like 8th Wall will be open sourcing soon based on their release today that they are shutting down. Hopefully that will give another decent option for marker tracking outside of WebXR.

My understanding is that part of the problem is WebXR not having access to autofocus (apparently for tracking stability)?

In terms of marker tracking and localisation with basic fiducial/arUco markers, you can get a solution working decently with a combination of OpenCV. See examples: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webxr_test_pages/webxr-samples/proposals/camera-access-marker.html or https://github.com/mozilla/webxr-polyfill/blob/master/examples/opencv-aruco/index.html

OpenCV can also scan QR codes but the problem I hit is that without autofocus, getting close to a QR code of normal size generally means it's too blurry to read info from.

[AdaRoseCannon]

/agenda to maybe settle on QR codes??

[mkeblx]

Most platforms support QR code tracking, so that could be a good candidate for initial support.

Android XR:
https://developer.android.com/develop/xr/openxr/extensions/XR_ANDROID_trackables_qr_code

Meta Horizon:
https://developers.meta.com/horizon/documentation/unity/unity-mr-utility-kit-qrcode-detection

visionOS:
https://developer.apple.com/documentation/visionOS/locating-and-decoding-barcodes-in-3d-space