Use the libretro VFS interface in libretro builds

[white-axe]

• Closes libretro: make rom access use the libretro VFS api for using SAF files served by a document provider in android #21087

Allows, for example, loading games from the Storage Access Framework when using the libretro core in the RetroArch Android app.

I removed the duplicate header files in the libretro directory in this repository that are already present in libretro-common, by the way, because this pull request relies on libretro-common which in turn relies on libretro-common's versions of the duplicated header files. I see no reason to have two copies of the libretro-common header files anyways.

[i30817]

That looks more complicated than I was expecting.

Don't mind me just posting to get subscribed to notifications in the mobile GitHub interface

[white-axe]

It's because the original code depends on library functions that themselves use stdio functions in a few places. I rewrote them to be more agnostic of what file system API is being used.

• ZipFileReader and the zip-related functions in Core/Loaders.cpp are changed to use libzip's functions that take IO callback functions instead of the equivalent ones that operate directly on stdio.
• Ditto for the PNG image writer in pngSave().
• The gzip file writer in SymbolMap.cpp now manually calls the low-level deflate() function and manually writes the deflated data bit-by-bit to the output file instead of using gzopen()/gzdopen().

The file_stream_transforms.h header file that redirects the remaining stdio calls made from directly inside of the PPSSPP codebase also doesn't like it when you write std::FILE, std::fclose, etc. so I removed the std:: from the beginning.

Also, I've consolidated the various platform-dependent methods of seeking files into File::Fseek(), File::Fseektell() and File::Ftell() helper functions so that I could add a method for seeking files on libretro only without having to insert ifdefs everywhere.

[hrydgard]

Pretty impressive! Yeah, this is probably really the only way to go about it.

I kicked off the CI. Will give it some light testing later before merge.

[hrydgard]

As you can see in the CI reports, there are some build issues for non-libretro builds.

[hrydgard]

Triggered a new build. Since this is your first PR, CI needs manual triggering. I'll try to keep up reasonably.

[hrydgard]

Thanks for the fixes! Will do a final review tomorrow, I think we'll be able to get this in shortly.

[hrydgard]

Running on Android, there seems to be some pointer truncation problem in the new libpng code:

01-05 15:14:55.188 13705 13705 F DEBUG   : signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
01-05 15:14:55.188 13705 13705 F DEBUG   : Abort message: 'Pointer tag for 0x72a8d2d2c8 was truncated, see 'https://source.android.com/devices/tech/debug/tagged-pointers'.'
01-05 15:14:55.188 13705 13705 F DEBUG   :     x0  0000000000000000  x1  0000000000003573  x2  0000000000000006  x3  0000007288543170
01-05 15:14:55.188 13705 13705 F DEBUG   :     x4  7260761f37623163  x5  7260761f37623163  x6  7260761f37623163  x7  7f7f7f7f7f7f7f7f
01-05 15:14:55.188 13705 13705 F DEBUG   :     x8  00000000000000f0  x9  0000007625ced498  x10 ffffff80fffffbdf  x11 0000000000000000
01-05 15:14:55.188 13705 13705 F DEBUG   :     x12 00000000012e4022  x13 0000000276aefd5c  x14 00331a828343861a  x15 0000000026762762
01-05 15:14:55.188 13705 13705 F DEBUG   :     x16 0000007625d99fe0  x17 0000007625d85fc0  x18 0000007284ae0000  x19 0000000000003535
01-05 15:14:55.188 13705 13705 F DEBUG   :     x20 0000000000003573  x21 00000000ffffffff  x22 0000000000003535  x23 0000000000003535
01-05 15:14:55.188 13705 13705 F DEBUG   :     x24 00000072885445f0  x25 00000072885445f0  x26 0000007288544930  x27 0000007650fa8940
01-05 15:14:55.188 13705 13705 F DEBUG   :     x28 000000728844c000  x29 00000072885431f0
01-05 15:14:55.188 13705 13705 F DEBUG   :     lr  0000007625d24e34  sp  0000007288543150  pc  0000007625d24e64  pst 0000000000001000
01-05 15:14:55.188 13705 13705 F DEBUG   : 25 total frames
01-05 15:14:55.188 13705 13705 F DEBUG   : backtrace:
01-05 15:14:55.188 13705 13705 F DEBUG   :       #00 pc 000000000005ae64  /apex/com.android.runtime/lib64/bionic/libc.so (abort+168)
01-05 15:14:55.188 13705 13705 F DEBUG   :       #01 pc 0000000000045430  /apex/com.android.runtime/lib64/bionic/libc.so (free+104)
01-05 15:14:55.188 13705 13705 F DEBUG   :       #02 pc 0000000001901cb4  /arm64/libppsspp_jni.so (pngLoadPtr(unsigned char const*, unsigned long, int*, int*, unsigned char**)+304)
01-05 15:14:55.188 13705 13705 F DEBUG   :       #03 pc 0000000001d622b0  /arm64/libppsspp_jni.so (Image::LoadPNG(char const*)+124)
01-05 15:14:55.188 13705 13705 F DEBUG   :       #04 pc 0000000001d4eaf4  /arm64/libppsspp_jni.so
01-05 15:14:55.188 13705 13705 F DEBUG   :       #05 pc 0000000001d4d8d4  /arm64/libppsspp_jni.so (GenerateUIAtlas(Draw::DrawContext*, Atlas*, float, bool)+300)
01-05 15:14:55.188 13705 13705 F DEBUG   :       #06 pc 0000000001d4f3e4  /arm64/libppsspp_jni.so (AtlasProvider(Draw::DrawContext*, AtlasChoice, float, bool)+144)
01-05 15:14:55.188 13705 13705 F DEBUG   :       #07 pc 0000000001a6803c  /arm64/libppsspp_jni.so
01-05 15:14:55.189 13705 13705 F DEBUG   :       #08 pc 0000000001a67fd0  /arm64/libppsspp_jni.so (AtlasData std::__ndk1::__invoke_void_return_wrapper<AtlasData, false>::__call[abi:ne210000]<AtlasData (*&)(Draw::DrawContext*, AtlasChoice, float, bool), Draw::DrawContext*, AtlasChoice, float, bool>(AtlasData (*&)(Draw::DrawContext*, AtlasChoice, float, bool), Draw::DrawContext*&&, AtlasChoice&&, float&&, bool&&)+52)
01-0

[hrydgard]

Actually, that's not the root cause. It just fails to load all pngs from assets on Android. Assets are read from the APK as a zip file, and something is wrong with that.

pngLoadPtr does need *image_data_ptr = nullptr; at the start to avoid this misleading error though.

[hrydgard]

Yeah, the zipfilereader is not functioning correctly, it reads garbled data. Not sure why.

[hrydgard]

Same happens on standalone in Windows - the zipfilereader is busted.

On Windows, we don't use the zipfilereader for assets, so the app starts. But if you try to install something from the homebrew store, it blows up.

It's clear you didn't test this.

[white-axe]

Should be fixed for installing games from the homebrew store, but the Android app is still broken. I'm working on fixing that now.

[white-axe]

Okay, the non-libretro Android builds should work again.

[hrydgard]

Hm. PPSSPP never really has any need to access 4GB+ files (as PSP ISOs are a lot smaller than that, even the largest "combined" ISOs like the Final Fantasy Type-0 translation).

Combining fseek and lseek like that looks super scary, and I'm not sure if we shouldn't just do a plain old fseek there instead.

[white-axe]

Alright, I've changed that code path to use normal fseek and ftell.