Android openGL ES 3 support

[joliver82]

Added gles30 and newer functionality for android. All the information at the forum thread: https://hub.jmonkeyengine.org/t/opengl-es-3-0-and-newer-on-android/41913

Chage list:

• Initialize the proper opengl context (2.0, 3.0, 3.1…)
• Added Caps for all of the new supported features
• Added Instancing as core feature of GLES30 and newer
• Properly adding versioning to shaders (#version xxx es) to shaders
• Added GLSL320,GLSL310 and/or GLSL300 to most materials depending on the required level for them to work properly
• Added depth24 texture usage if hardware supports it
• Enabled Geometry shaders and tessellation (3.2 core feature and 3.1 as extension) and fixed compilation errors on the examples
• Enabled glBlitFramebuffer
• Added GLSL300 to filters not able to work with GLSL100.
• Created Glsl300ShaderGenerator to support GLSL300 in shadernodes
• Fixed most shader compilation issues (strict type checking, adding OpenGL ES specific extensions, function usage depending on version and some other)
• Set default precision to highp solving most filter issues although maybe a little slower. Added specific sampler precision definition for gles300 or newer
• Make use of GL_TEXTURE_MAX_LEVEL in GLES300 to avoid black rendering of non complete mipmapped textures
• Changed some assets from compressed formats not supported in android to RGB/RGBA depending on the needs so all effects and tests can run also on android
• updated precompiled android.jar to have access to newest android API
• enabled some other core features of GLES300, GLES310 and GLES320
• Fixed build issues in jme3-android-examples
• Added 3D texture and texture array support
• Added float texture formats, fixed rgb8 format if having cap rgb8 and fixed glTexImage2D call on AndroidGL
• PBRLighting works and any other effect requiring HDR texture formats should work also
• Effects and postprocessing
• Multisampling

Known issues/pending work:

• Fixed most issues with shadow rendering in android (GLES 2.0 and GLES 3.0 or better) although some glitches still apear randomly depending on ShadowMode of the nodes and the device
• Fixed water filter. Still some rendering glitches in my old redmi3 (gles3.0). The water renders properly but the rest of the scene is overwritten in red. Not reproduced in any other device I've tried

[riccardobl]

Also, would it be possible to make the build script download lib/android.jar from an official source?
It would be preferable than uploading a mistery jar to the repo.

I agree but a previous file of this one was included this way before and honestly I don't know where I could get this jar directly from the net. Android.jar is extracted to platform path of android sdk when installing it. Maybe adding a copy action to the gradle scripts as the sdk dir is set in the local.properties file?

If this is a dependency, gradle can just load it from the sdk folder. I will open an issue for that, but for now i just want to verify the integrity of the android.jar provided in this pr.

This is the sha256 hash of android.jar from the last android studio release
31cca6a9203b04349e88ec65cc1a2b915058b699b28ab9259b98d30d5c3b26ac android.jar

and this is the hash of the jar provided in this pr
d20853d289a5161d9cf1c1268d7923076487d4fa68c970b0733e2d5a0ccfc3c5 android.jar

Can you point me to the release from which this jar comes from?

[joliver82]

Sure, it's the android.jar from api level 28. It should be located at your android sdk installation path. In my case /opt/android-sdk/ :

sha256sum /opt/android-sdk/platforms/android-28/android.jar
d20853d289a5161d9cf1c1268d7923076487d4fa68c970b0733e2d5a0ccfc3c5 /opt/android-sdk/platforms/android-28/android.jar

[riccardobl]

Ok, i was testing the wrong file.

[riccardobl]

Is jme3-effects/src/main/resources/Common/MatDefs/Water/Textures/water_normalmap_dxt1.dds used anywhere?

[riccardobl]

Is it a good idea to force this on the developer?
I know the code already forced precision mediump float; , but this looks like bad practice to me.

[joliver82]

So you mean removing all these inserts and making the game/shader developer? This will lead to plenty of current jme3 shaders not to compile on gles at all.

[riccardobl]

Could it be implemented in GLSLCompat? So that devs can leave it out for custom shaders.

[joliver82]

Not really, precision directive must be after last #extension line. All shaders include GLSLCompat, then other requirements (that could have extension directives) and then the shader code, so having precision defined in GLSLCompat will make compilation fail

[riccardobl]

Isn't the shader loader supposed to move the #extension directives to the top?

[joliver82]

Yes, you're right and should work. I'll change and test it and get back to you

[joliver82]

I've tried changed it and some shaders included by default in jme3 like Common/MatDefs/Terrain/TerrainLighting.frag don't include GLSLCompat.glsllib. We could either leave it as it's now or review all shaders to include GLSLCompat as you think is a better option 🤔

[riccardobl]

Yeah, not a problem, we can add the GLSLCompat to the remaining shaders with a future PR,

[joliver82]

Is jme3-effects/src/main/resources/Common/MatDefs/Water/Textures/water_normalmap_dxt1.dds used anywhere?

Not at all, I didn't realize that file when removing the others 🤦‍♂️ just removed and comited

[riccardobl]

If possible i'd like to see the precision instructions moved to the GLSLCompat as we discussed.
That aside, i think this is ready to be merged/

[stephengold]

I'd like to include this in the 3.3.0-alpha3 release. How soon can it be finished?

[joliver82]

From my side it's finished. @riccardobl changed it to Testing but I don't know who and when should test it before merging

[stephengold]

Have the precision instructions been moved to GLSLCompat?

[joliver82]

No because plenty of shaders would stop working as they don't include GLSLCompat. So if I understood Riccardo correctly, we'll leave this for the future

[riccardobl]

I meant to add the precision to GLSLCompat now and include GLSLCompat on all the shaders in a future pr, but if you think we should delay that and leave the pr as it is, then it's fine either way.

Since i'm not an android developer i'm waiting for the approvation of someone else who can validate and maybe test this pr on an existing application to see if we missed something.

[kaloyanDEV]

Testing on existing application using https://github.com/joliver82/jmonkeyengine and android-gl3 branch

def jmerepo = 'org.jmonkeyengine'
def jmeversion = '3.3.0-alpha2'
android.defaultConfig.javaCompileOptions.annotationProcessorOptions.includeCompileClasspath = true
dependencies {
    compile(project(":game")) {
        exclude module: "jme3-lwjgl"
        exclude module: "jme3-desktop"
        exclude group: 'org.codehaus.groovy', module: 'groovy-all'
        exclude module: 'log4j-slf4j-impl'
        exclude module: "jme3-core"
        exclude module: "jme3-effects"
    }
    compile 'org.apache.logging.log4j:log4j-core:2.5'

//    compile jmerepo + ':jme3-android:' + jmeversion

    compile files('/home/kaloyan/Projects/jmonkeyengine/jme3-android/build/libs/jme3-android-3.3.0-android-gl3-SNAPSHOT.jar')
    compile files('/home/kaloyan/Projects/jmonkeyengine/jme3-core/build/libs/jme3-core-3.3.0-android-gl3-SNAPSHOT.jar')
    compile files('/home/kaloyan/Projects/jmonkeyengine/jme3-effects/build/libs/jme3-effects-3.3.0-android-gl3-SNAPSHOT.jar')




    compile(jmerepo + ':jme3-android-native:' + jmeversion) {
        exclude module: "jme3-android"
        exclude module: "jme3-core"
    }

    implementation 'com.google.guava:guava:27.1-android'
}

public class MainActivity extends Activity {

private AndroidHarnessFragment jmeFragment;
//....
public static class JmeFragment extends AndroidHarnessFragment {
    public JmeFragment() {
        // Set main project class (fully qualified path)
        appClass = Main.class.getName();

        // Set the desired EGL configuration
        eglBitsPerPixel = 24;
        eglAlphaBits = 0;
        eglDepthBits = 16;
        eglSamples = 0;
        eglStencilBits = 0;

        // Set the maximum framerate
        // (default = -1 for unlimited)
        frameRate = 60;

        // Set the maximum resolution dimension
        // (the smaller side, height or width, is set automatically
        // to maintain the original device screen aspect ratio)
        // (default = -1 to match device screen resolution)
        maxResolutionDimension = -1;

        // Set input configuration settings
        joystickEventsEnabled = false;
        keyEventsEnabled = true;
        mouseEventsEnabled = true;

        // Set application exit settings
        finishOnAppStop = true;
        handleExitHook = true;
        exitDialogTitle = "Do you want to exit?";
        exitDialogMessage = "Use your home key to bring this app into the background or exit to terminate it.";

        // Set splash screen resource id, if used
        // (default = 0, no splash screen)
        // For example, if the image file name is "splash"...
        //     splashPicID = R.drawable.splash;
        //splashPicID = 0;
        splashPicID = R.drawable.splash;
    }
}

protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    //--------- JME Stuff ---------

    // Set window fullscreen and remove title bar
    requestWindowFeature(Window.FEATURE_NO_TITLE);
    getWindow().setFlags(
            WindowManager.LayoutParams.FLAG_FULLSCREEN,
            WindowManager.LayoutParams.FLAG_FULLSCREEN);
    setContentView(R.layout.main);

    // find the fragment
    FragmentManager fm = getFragmentManager();
    jmeFragment = (AndroidHarnessFragment) fm.findFragmentById(R.id.jmeFragment);
}
//...

}

public class Main extends SimpleApplication {
//...
public void start() {
super.start(); //To change body of generated methods, choose Tools | Templates.
this.getContext().getSettings().setRenderer(AppSettings.LWJGL_OPENGL3);
}
}

Using test data from same branch (sun is just  DirectionalLight)

public class TestPBR extends BaseAppState {

private Node modelNode;
private Geometry model;
private Material pbrMat;

private float roughness = 0.0f;
private float frame;
private AssetManager assets;

private Node tex;

private Container buttons;

@Override
protected void initialize(Application app) {

    assets = app.getAssetManager();

    modelNode = (Node) new Node("modelNode");
    modelNode.getLocalTranslation().y -= 2f;
    model = (Geometry) assets.loadModel("Models/Tank/tank.j3o");
    model.setLocalScale(0.5f);
    MikktspaceTangentGenerator.generate(model);
    modelNode.attachChild(model);
    pbrMat = assets.loadMaterial("Models/Tank/tank.j3m");
    pbrMat.setFloat("Roughness", roughness);
    model.setMaterial(pbrMat);

    final EnvironmentCamera envCam = new EnvironmentCamera(256, new Vector3f(0, 3f, 0));
    getStateManager().attach(envCam);

    getApplication().getInputManager().addListener(new ActionListener() {
        @Override
        public void onAction(String name, boolean isPressed, float tpf) {

            if (name.equals("rup") && isPressed) {
                roughness = FastMath.clamp(roughness + 0.1f, 0.0f, 1.0f);
                pbrMat.setFloat("Roughness", roughness);
            }
            if (name.equals("rdown") && isPressed) {
                roughness = FastMath.clamp(roughness - 0.1f, 0.0f, 1.0f);
                pbrMat.setFloat("Roughness", roughness);
            }

            if (name.equals("light") && isPressed) {
                getState(LightState.class).getSun().setDirection(getApplication().getCamera().getDirection().normalize());
            }
        }
    }, "light", "rup", "rdown");

    getApplication().getInputManager().addMapping("light", new KeyTrigger(KeyInput.KEY_J));
    getApplication().getInputManager().addMapping("rup", new KeyTrigger(KeyInput.KEY_F));
    getApplication().getInputManager().addMapping("rdown", new KeyTrigger(KeyInput.KEY_G));

    getState(LightState.class).getSun().setDirection(new Vector3f(0.8999188f, -0.34202015f, -0.2704965f));

    //buttons
    float width = app.getContext().getSettings().getWidth();
    float height = app.getContext().getSettings().getHeight();

    buttons = new Container();
    buttons.setLocalScale(1);
    buttons.setPreferredSize(new Vector3f(width * 0.24f, height * .3f, 0));

    ActionButton button = buttons.addChild(new ActionButton(new CallMethodAction("Roughness UP", this, "rup")));
    button.setTextVAlignment(VAlignment.Center);
    button.setFontSize(height * 0.03f);
    button = buttons.addChild(new ActionButton(new CallMethodAction("Roughness DOWN", this, "rdown")));
    button.setTextVAlignment(VAlignment.Center);
    button.setFontSize(height * 0.03f);
    button = buttons.addChild(new ActionButton(new CallMethodAction("LIGHT", this, "light")));
    button.setTextVAlignment(VAlignment.Center);
    button.setFontSize(height * 0.03f);

    buttons.setLocalTranslation(0, height * 0.625f, 0);
}

public void rup() {
    roughness = FastMath.clamp(roughness + 0.1f, 0.0f, 1.0f);
    pbrMat.setFloat("Roughness", roughness);
}

public void rdown() {
    roughness = FastMath.clamp(roughness - 0.1f, 0.0f, 1.0f);
    pbrMat.setFloat("Roughness", roughness);
}

public void light() {
    getState(LightState.class).getSun().setDirection(getApplication().getCamera().getDirection().normalize());
}


protected void cleanup(Application aplctn) {
}

protected void onEnable() {
    Node gui = ((Main) getApplication()).getGuiNode();
    gui.attachChild(buttons);
    GuiGlobals.getInstance().requestFocus(buttons);
}

protected void onDisable() {
}

public void update(float tpf) {
    frame++;

    if (frame == 2) {
        modelNode.removeFromParent();
        final LightProbe probe = LightProbeFactory.makeProbe(
                getApplication().getStateManager().getState(EnvironmentCamera.class),
                ((Main) getApplication()).getRootNode(),
                new JobProgressAdapter<LightProbe>() {

            @Override
            public void done(LightProbe result) {
                System.err.println("Done rendering env maps");
                tex = EnvMapUtils.getCubeMapCrossDebugViewWithMipMaps(result.getPrefilteredEnvMap(), assets);
            }
        });
        ((SphereProbeArea) probe.getArea()).setRadius(100);
        ((Main) getApplication()).getRootNode().addLight(probe);
        //getStateManager().getState(EnvironmentManager.class).addEnvProbe(probe);

    }
    if (frame > 10 && modelNode.getParent() == null) {
        ((Main) getApplication()).getRootNode().attachChild(modelNode);
    }
}

}

On Xiaomi Redmi Note 3 (android 6) reflection from LightProbe is rendered (it is taking long)

[kaloyanDEV]

Problem appears while trying to use running heightmap on 3.3.0-alpha2

java.lang.NoSuchMethodError: No virtual method put(II)V in class Lcom/jme3/scene/mesh/IndexBuffer; or its super classes (declaration of 'com.jme3.scene.mesh.IndexBuffer' appears in /data/app/com.tech.gladiators-1/base.apk:classes5.dex)
        at com.jme3.terrain.geomipmap.LODGeomap$VerboseBuffer.put(LODGeomap.java:931)
        at com.jme3.terrain.geomipmap.LODGeomap.writeIndexArrayLodDiff(LODGeomap.java:175)
        at com.jme3.terrain.geomipmap.LODGeomap.createMesh(LODGeomap.java:85)
        at com.jme3.terrain.geomipmap.LODGeomap.createMesh(LODGeomap.java:77)
        at com.jme3.terrain.geomipmap.TerrainPatch.<init>(TerrainPatch.java:190)
        at com.jme3.terrain.geomipmap.TerrainQuad.createQuadPatch(TerrainQuad.java:722)
        at com.jme3.terrain.geomipmap.TerrainQuad.split(TerrainQuad.java:567)
        at com.jme3.terrain.geomipmap.TerrainQuad.<init>(TerrainQuad.java:223)
        at com.jme3.terrain.geomipmap.TerrainQuad.createQuad(TerrainQuad.java:611)
        at com.jme3.terrain.geomipmap.TerrainQuad.split(TerrainQuad.java:569)
        at com.jme3.terrain.geomipmap.TerrainQuad.<init>(TerrainQuad.java:223)
        at com.jme3.terrain.geomipmap.TerrainQuad.createQuad(TerrainQuad.java:611)
        at com.jme3.terrain.geomipmap.TerrainQuad.split(TerrainQuad.java:569)
        at com.jme3.terrain.geomipmap.TerrainQuad.<init>(TerrainQuad.java:223)
        at com.jme3.terrain.geomipmap.TerrainQuad.<init>(TerrainQuad.java:176)
        at com.jme3.terrain.geomipmap.TerrainQuad.<init>(TerrainQuad.java:157)

[joliver82]

@kaloyanDEV About the heightmap issue, did it work previously in main jme3? Did you run a sample from jme3-examples or any personal code? Could you send me a simple testcase if the later case?

[kaloyanDEV]

@joliver82

    AssetManager assetManager = getApplication().getAssetManager();

        /**
         * 1. Create terrain material and load four textures into it.
         */
        Material mat_terrain = new Material(assetManager,
                "Common/MatDefs/Terrain/Terrain.j3md");

        /**
         * 1.1) Add ALPHA map (for red-blue-green coded splat textures)
         */
        mat_terrain.setTexture("Alpha", assetManager.loadTexture(
                "Textures/Terrain/alphamap.png"));

        /**
         * 1.2) Add ROCKS texture into the red layer (Tex1).
         */
        Texture grass = assetManager.loadTexture(
                "Textures/Terrain/rock.png");
        grass.setWrap(Texture.WrapMode.Repeat);
        mat_terrain.setTexture("Tex1", grass);
        mat_terrain.setFloat("Tex1Scale", 24f);

        /**
         * 1.3) Add GRASS texture into the green layer (Tex2)
         */
        Texture dirt = assetManager.loadTexture(
                "Textures/Terrain/grass.png");
        dirt.setWrap(Texture.WrapMode.Repeat);
        mat_terrain.setTexture("Tex2", dirt);
        mat_terrain.setFloat("Tex2Scale", 32f);

        /**
         * 1.4) Add SOMETHING texture into the blue layer (Tex3)
         */
        Texture rock = assetManager.loadTexture("Textures/Terrain/black_patch.png");
        rock.setWrap(Texture.WrapMode.Repeat);
        mat_terrain.setTexture("Tex3", rock);
        mat_terrain.setFloat("Tex3Scale", 28f);
        /**
         * 2. Create the height map
         */
        AbstractHeightMap heightmap = null;
        Texture heightMapImage = assetManager.loadTexture(
                "Textures/Terrain/untitled.png");
        heightmap = new ImageBasedHeightMap(heightMapImage.getImage());
        heightmap.load();

        /**
         * 3. We have prepared material and heightmap. Now we create the actual
         * terrain: 3.1) Create a TerrainQuad and name it "my terrain". 3.2) A
         * good value for terrain tiles is 64x64 -- so we supply 64+1=65. 3.3)
         * We prepared a heightmap of size 512x512 -- so we supply 512+1=513.
         * 3.4) As LOD step scale we supply Vector3f(1,1,1). 3.5) We supply the
         * prepared heightmap itself.
         */
        TerrainQuad terrain = new TerrainQuad("my terrain", 65, 513, heightmap.getHeightMap());

        /**
         * 4. We give the terrain its material, position & scale it, and attach
         * it.
         */
        terrain.setMaterial(mat_terrain);
        terrain.setLocalScale(.15f, .1f, .2f);

if you can not reproduce it I will send you the assets also

[riccardobl]

What is the status for this?

[Ali-RS]

The issue that @kaloyanDEV mentioned seems related to terrain stuff, not sure what it has to do with this PR 🙄

[kaloyanDEV]

yes Terrain.j3md is not part of changes but might be new post in hub if someone try using it. I run all tests from JME3 Tests Android apk successfully from this joliver repo.

[joliver82]

Hi, yes, there's nothing explicitly related to terraing in this PR. From my point of view, if @kaloyanDEV tested every other tests and worked, we could merge and leave this new issue for a future PR. I'll get into it also once I have some more spare time

[Ali-RS]

Okay, I am letting this to be for 2 more days, going to merge it after 2 days if no objection is raised until then.