Support for multiple views

[ryantrem]

Currently in Babylon Native, the lifetime of the engine is tied to a single view. We should support multi-view. I would imagine this working by:

1. Having a small Babylon object that controls the lifetime of the engine.
2. Allow for multiple native views, where each one effectively registers a view exposed to Babylon.js.
3. On the js side, basically make it work the same as multi-view with an HTML DOM (where there are multiple canvases (one per view) which are associated with cameras, and the render loop just renders each view). I imagine to do this we might need to introduce the concept of a view identity or name that can be matched up between the native side (where an EngineView is declared) and the js side (where we try to render a camera to a view).

[ryantrem]

After thinking about this more, here is what I propose:

Babylon.js Changes

• Don't expose Canvas anywhere in the public API. Instead, expose an opaque (or base class) RenderView (or RenderTarget, or RenderSurface, or some other similar name) that just exposes a name/id.
• Add a global function called FindRenderView (or whatever) that takes the name/id of a RenderView and returns the opaque object.
• FindRenderView should be based on document.getElementById and return a RenderView that wraps a Canvas.
• Engine.RegisterView should take a RenderView (instead of a Canvas) and a Camera
• DeviceInputSystem's constructor should take a RenderView (or name/id) instead of an Engine (input should be scoped to a view, not an engine).

Babylon Native Changes

• Expose a native (platform specific implementations) function AddRenderView (part of some plugin) to add a RenderView (e.g. a Surface on Android). This is the native equivalent of putting a Canvas in the DOM with a specific name/id.
• Babylon Native's implementation of FindRenderView should return a RenderView that wraps a platform specific render surface (e.g. a Surface for Android) that has been added by the above API.
• Actually support rendering to multiple surfaces.
• NativeInput and DeviceInputSystem should be updated to take RenderView (or name/id). Somewhere in the code path of AddRenderView we should also create a NativeInput instance for that view.

Once the above is done, then the JavaScript code can be the same between the browser and native, where it just queries for a RenderView (via FindRenderView) and calls RegisterView (e.g. associates a Camera with a RenderView).

Babylon React Native Changes

• EngineView already gets a Camera assigned to it. When the above changes are in place, then when Camera is set on EngineView it should get the associated Engine from the Camera and set that Engine on the nested NativeEngineView.
• When Engine is set on NativeEngineView, it should generate a unique view id, and call AddRenderView with the underlying render surface (e.g. a Surface on Android).
• On the JavaScript side, EngineView can then call FindRenderView to get the RenderView that was just made available by the native side, and call RegisterView to associate the specified Camera with the RenderView.

[bghgary]

cc @deltakosh

[obasille]

Following our discussion on the forum, here is a use case for multiple views:
A ReactNative app showing a list of items (that may be scrolled vertically), witch each item showing:

1. A 3D mesh
2. Some native UI controls (on top)

As an example:
(the small chevron on the top right of the die mesh is a UI button that opens up a pop up menu)

[ryantrem]

@bghgary and I recently discussed this more, so I'm going to capture some updated ideas from that discussion as well as some thinking after that discussion (needs additional input from @bghgary).

Based on the current state of the code (3+ years after this issue was created), I propose:

1. If a Window is passed to the Babylon::Graphics::Device constructor, then that is the single window that we will render to, just like today.
2. if a Window is not passed to Babylon::Graphics::Device (and therefore a Device is), then you can call a new RegisterWindow function, where you pass in a WindowT and an identifier (probably just a string).
3. Expose to JavaScript something like NativeGraphics.getWindowById(name: string) that would return a minimal reference to a "native window." This could be used directly, or could be used in a findRenderView (or findWindow, or whatever) abstraction like described above.
4. Don't have DeviceSourceManager be constructed from a "view." Instead, a representation of the "window" (or its identity) should flow through the input system and be surfaced in the IUIEvent (e.g. as the target property), and then the consumer can filter input events for their view(s).

Separate Windows should be represented as frame buffers in BGFX (see the createFrameBuffer overload that takes a void* _nwh), and Babylon Native would have to be updated to be able to manage and activate these frame buffers during rendering.

It may also be simpler to support multiple views via multiple engine instances before supporting multiple views via BJS registerView.

With the above, support for single and multi view would look something like:

	Babylon.js	Babylon Native	Babylon React Native
Single View	Single Canvas in HTML, get Canvas with document.getElementById and pass to Engine constructor	Graphics::Device initialized with the single Window, instantiate NativeEngine with no arguments (maintains backward compatibility)	EngineView with a Camera generates a unique Window id/name which is used in RegisterWindow, useEngine with no args
Multi Engine / Multi View	Multiple Canvass in HTML, get Canvas with document.getElementById and pass each one to a separate Engine instance	Graphics::Device initialized without a Window, multiple Windows registered via Graphics::Device::RegisterWindow, get Window with NativeGraphics.getWindowById and pass each one to a separate NativeEngine instance, and that Window information would be used to determine with BGFX FrameBuffer to render to when each NativeEngine instance is rendering	EngineView with a Camera generates a unique Window id/name which is used in RegisterWindow and also "returned" to the EngineView caller, the id/name is passed into useEngine which uses NativeGraphics.getWindowById to get the Window which is in turn passed to the NativeEngine constructor
Single Engine / Multi View	Multiple Canvass in HTML, get Canvas with document.getElementById and pass each one into a call to registerView on a single Engine instance, filter input events by comparing Canvas instances to the target property of the IUIEvent	Graphics::Device initialized without a Window, multiple Windows registered via Graphics::Device::RegisterWindow, get Window with NativeGraphics.getWindowById and pass each one into a call to registerView on a single NativeEngine instance, filter input events by comparing Window (info) instances to the target property of the IUIEvent	EngineView with a Camera generates a unique Window id/name which is used in RegisterWindow and also "returned" to the EngineView caller, useEngine with no args, filter input events by comparing Window (info) instances to the target property of the IUIEvent

Note that this means BRN would switch to always using RegisterWindow and initializing BN with just a device. This means the only API change is to allow a Window id/name to optionally be passed to useEngine, and for EngineView to "return" a Window id/name.

[bghgary]

Closing for now. Reopen if we want to support this feature.