Imported NES games (via scan) won't appear in "Explore" categories—Year, Developer, Genre, etc

[rezabrando]

Description

Imported NES games (via scan function) won't appear in "Explore" categories.

Expected behavior

Explore should include NES games like any other playlists in its categories.

Actual behavior

NES games won't appear in the categories.

Steps to reproduce the bug

1. Scan some NES games and make sure they are added to your library.
2. Go to Explore and see games By Year or By Genre or By Developer. None of them will be included.
3. Scan another system—I've tested Mega Drive, Game Boy, and Game Boy Color myself—and they will be included in Explore categories.

Bisect Results

Happened after a clean install.

Version/Commit

• Git Version: 5e551dd (August 8 2020)
• RetroArch: 1.9.0

Environment information

• OS: Android 6.0.1 and Windows 7 SP1
  I'm using official builds.

[i30817]

There is also visible lag collecting this information the first time after retroarch is run. Not sure if that 'playlist' top level info can be cached to file in a better way, but i'll just mention it.

I'm sure it's painful for devices that are lower powered, especially if more large rdb databases like the nes become admissible.

Since I already mentioned something i dislike i'll mention another: genres only having 'game genre' metadata information is disappointing. 'Thematic' subgenres genres like horror, comedy, first person, adventure, metroidvania, even other narrative characteristics like multiple protagonists (think FF type 0), female protagonist, adult protagonist, etc are not represented there.

Especially the first. I'm not often interested in knowing if a game is a platformer, but if it's horror, or if it has rpg elements or if has mazes (metroidvania) or other setting or cast centric characteristics. Hardcoregaming101 has a good tag system for this: http://www.hardcoregaming101.net/reviews/

especially look at the 'genres, subgenres and themes' parts.

[boomercore]

Confirming this in a fresh nightly build as of this posting. What's odd about it is that I can examine the ROM's database entry in the UI, and it will show the Genre, Developer, and Releasedate Year but it simply will not appear in those categories in Explore. Imagine Explore simply has some maturing to do as a feature.

[schellingb]

First of all sorry for noticing this issue so late. I should probably check this issue tracker for "Explore" related issues from time to time.

As far as I can tell it's all a data problem.

Here's an example of what it looks like inside "Nintendo - Nintendo Entertainment System.rdb":

So what we can see, only the ROM variant of "Contra (USA)" with CRC F6035030 has a release date 1988/02/01 associated.
Even worse than that, a row with no name has the release date 1988/10/01 and CRC 7D3F6F3D. AND THEN there is another row with the same CRC but the name "Super Mario Bros. 2 (USA)" and no release date.

Because the explore view looks up games by CRC first and then by name, it ends up actually finding the release date of the one "Super Mario Bros. 2 (USA)" ROM variant with CRC 7D3F6F3D but not of the two "Contra (USA)" rom variants that don't have CRC F6035030.

On the other hand examining the ROM's database entry looks up information by name, so it ends up finding the release date of either of the 3 "Contra (USA)" ROM variants but never of "Super Mario Bros. 2 (USA)".

Now if the database were to look like this all these issues would go away:

When I developed this feature, I was always hoping that it would somehow bring more attention and care to the databases.
If fixing the database is more complicated than modifying the explore view to prefer name matches over CRC matches then that could be done I guess.

[schellingb]

Oh, I forgot to mention, you can fix the data yourself rather easy.

The tool I used to look into the RDB files is my own RDBEd. If you open the "Nintendo - Nintendo Entertainment System.rdb" file and select Tools -> Unify Meta Data with Equal Field... and select "Name" as the Merge Key and press OK, it will fix up the database and it should work.

It's also advised to do Validate -> Validate Unique Field..., press OK, and then click the number of warnings in the status bar to find all rows with duplicated CRCs. After sorting by CRC it's easy to either remove duplicated/bad rows or fix them to make sure the meta data fields are all filled in.

[kp8888]

Is there any chance that the fixed database files can be added to the online updater?

@schellingb, I followed your directions but ended up with over 6000 warnings. Is going through all those conflicts manually truly the easiest way to fix the database, or am I using your tool incorrectly? I really want the explore menu to work correctly for NES games. I hope you can help me. Cheers.