Support Oculus

[bghgary]

See https://developer.oculus.com/downloads/package/oculus-openxr-mobile-sdk/.

We have support for OpenXR but we have only tested this on Windows so far. Now that Occulus has an SDK for this, it would be good to test it out on Occulus also.

[TiagoSilvaPereira]

@bghgary I'm interested in testing this out and contributing as much as possible. Could you please advise me on what I should do to test the OpenXR API with Quest 2?

[bghgary]

I'm not familiar with the Oculus SDK, but I would imagine we need to:

1. Install the SDK (as noted above).
2. Build and run Babylon Native Playground with vr = true in experience.js with the Quest 2 plugged in.
3. See what kind of problems there are with the code.

Most likely there will be issues in this part of the code: https://github.com/BabylonJS/BabylonNative/tree/master/Dependencies/xr/Source/OpenXR

We might also need to update the OpenXR dependency from this folder.

[TiagoSilvaPereira]

Thank you, @bghgary I'll try it this weekend

[ryantrem]

Currently, the Android compilation of Babylon Native specifically chooses to use ARCore as the XR solution (as you can see here: https://github.com/BabylonJS/BabylonNative/blob/master/Dependencies/xr/CMakeLists.txt). I'd think the goal would be to use the Android compilation of Babylon Native on Oculus Quest, and as such we'd need a variant of the Android compilation that uses OpenXR instead (or in addition via some runtime configuration?), or maybe a different compilation specifically targeting Oculus Quest (I don't know enough about Oculus Quest development to know which option might make the most sense).

[TiagoSilvaPereira]

@ryantrem, thank you for this information... maybe an ANDROID_OPENXR option? I think it would make PicoVR 4 also compatible. I'll try first to adapt the android build to use OpenXR (if I manage to see a cube rotating on my Quest device I'm happy), then I'll check how to add it to the configuration. I pretend to share my progress here so we can discuss it

[TiagoSilvaPereira]

@bghgary @ryantrem I managed to compile the Android Playground and run it on the phone. Now I'm trying to run it on a Quest 2 device (arm64-v8a), but when trying to build, it is failing to link the libastc-encoder.a library (it seems BGFX is using it). I tried to research and understand what is happening, but I need help to get through. Do you know how I can fix that?

Build command failed.
Error while executing process ninja.exe with arguments {-C C:\Users\tiago\code\3d\BabylonNative\Apps\Playground\Android\app\.cxx\cmake\debug\arm64-v8a BabylonNativeJNI}
ninja: Entering directory `C:\Users\tiago\code\3d\BabylonNative\Apps\Playground\Android\app\.cxx\cmake\debug\arm64-v8a'
[1/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_platform_isa_detection.cpp.o
[2/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_weight_quant_xfer_tables.cpp.o
[3/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_find_best_partitioning.cpp.o
[4/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_pick_best_endpoint_format.cpp.o
[5/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_averages_and_directions.cpp.o
[6/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_ideal_endpoints_and_weights.cpp.o

// Failing here

[7/137] Linking CXX static library C:\Users\tiago\code\3d\BabylonNative\build\Android_arm64-v8a\Dependencies\bgfx.cmake\libastc-encoder.a
FAILED: C:/Users/tiago/code/3d/BabylonNative/build/Android_arm64-v8a/Dependencies/bgfx.cmake/libastc-encoder.a 
cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E rm -f 


C:\Users\tiago\code\3d\BabylonNative\build\Android_arm64-v8a\Dependencies\bgfx.cmake\libastc-encoder.a && C:\Users\tiago\AppData\Local\Android\Sdk\ndk\21.4.7075529\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-ar.exe qc C:\Users\tiago\code\3d\BabylonNative\build\Android_arm64-v8a\Dependencies\bgfx.cmake\libastc-encoder.a  ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_averages_and_directions.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_block_sizes.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_color_quantize.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_color_unquantize.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_compress_symbolic.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_compute_variance.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_decompress_symbolic.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_diagnostic_trace.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_entry.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_find_best_partitioning.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_ideal_endpoints_and_weights.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_image.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_integer_sequence.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_mathlib.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_mathlib_softfloat.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_partition_tables.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_percentile_tables.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_pick_best_endpoint_format.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_platform_isa_detection.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_quantization.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_symbolic_physical.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_weight_align.cpp.o ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_weight_quant_xfer_tables.cpp.o && C:\Users\tiago\AppData\Local\Android\Sdk\ndk\21.4.7075529\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-ranlib.exe C:\Users\tiago\code\3d\BabylonNative\build\Android_arm64-v8a\Dependencies\bgfx.cmake\libastc-encoder.a && cd ."
C:\Users\tiago\AppData\Local\Android\Sdk\ndk\21.4.7075529\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-ar.exe: ../../../../../../../../build/Android_arm64-v8a/Dependencies/bgfx.cmake/CMakeFiles/astc-encoder.dir/bimg/3rdparty/astc-encoder/source/astcenc_averages_and_directions.cpp.o: No such file or directory

[8/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/parseConst.cpp.o
[9/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/limits.cpp.o
[10/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/intermOut.cpp.o
[11/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/preprocessor/PpAtom.cpp.o
[12/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/Versions.cpp.o
[13/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/reflection.cpp.o
[14/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/glslang.dir/CInterface/glslang_c_interface.cpp.o
[15/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/preprocessor/Pp.cpp.o
[16/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/preprocessor/PpScanner.cpp.o
[17/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/preprocessor/PpContext.cpp.o
[18/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/preprocessor/PpTokens.cpp.o
[19/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/propagateNoContraction.cpp.o
[20/137] Building CXX object ../../../../../../../../build/Android_arm64-v8a/Dependencies/glslang/glslang/CMakeFiles/MachineIndependent.dir/MachineIndependent/linkValidate.cpp.o
ninja: build stopped: subcommand failed.