[Feature Request] add sd-extension-aesthetic-scorer scoring for SDXL compatibility

[rebootjac]

I appreciate how this project supports and uses aesthetic_score from aesthetic-image-scorer. However, the aesthetic-image-scorer project has not been updated in a long time and is likely abandoned. Worse, there is a current known bug that you can read about here: tsngo/stable-diffusion-webui-aesthetic-image-scorer/issues/25 that causes a very large performance hit when generating anything with SDXL models. I myself can also vouch for this same error and performance hit.

As a replacement, I found https://github.com/vladmandic/sd-extension-aesthetic-scorer which solves the SDXL slowdown problem and gets rid of all of the errors mentioned in the support post. However, Images Browser does not recognize these type of tags when sorting by aesthetic_score built-in filter. As a workaround, I have had limited success attempting to replace the strings in the new extension to write the same meta information that Images Browser uses (aesthetic_scores) instead of (Score) which does indeed work but there are two problems:

1. In my testing, the meta information gets written as aesthetic_score:: [rating] note the extra colon. This is likely a simple string/logic error I made when replacing the strings. Side note: I am not a coder but I do have some scripting experience and haven't been able to figure this out yet.

2. Most importantly though, the sd-extension-aesthetic-scorer project uses a 1-10 scoring system instead of 1-5 (I think) like the outdated one does.

Ideally, it would be best if the Images Browser Extension could incorporate and utilize the scoring from both methods and store them as separate variables OR for the new method, use a scaling modifier to write the new scoring method as 1-5 so it is compatible with sd-extension-aesthetic-scorer.

[AlUlkesh]

Ok, this is odd. I just installed that extension - and it just works.

After creating an image with it enabled, I get this in the pnginfo:
Steps: 6, Sampler: DPM++ SDE Karras, CFG scale: 2, Seed: 3314045208, Size: 1024x1024, Model hash: 676f0d60c8, Model: dreamshaperXL_turboDpmppSDE, VAE hash: 235745af8d, VAE: sdxl_vae_fp16fix.safetensors, Version: v1.7.0-449-g26003706, Score: 6.55

The last entry here is "Score: 6.55" and "Score" is a keyword the image browser knows.

Also I don't think there is any limitation on the value of the score. Are you perhaps confusing that with "Ranking", which is another thing entirely?

Perhaps check if you have the current versions of everything and no conflicting extensions?

[rebootjac]

Thanks for testing this with me and apologies on the confusion from my first post, I know you are working hard to implement the new Gradio UI version (I can't wait!)

I think I found a bug. Yes, aesthetic-scorer does indeed add the meta information and is readable in pnginfo for me as well. I can also use the minimum/maximum score for the new tags which I actually didn't know I could do but now I know I can.

What is still not working for me (after updating extensions and SD) is that if you try to sort the images by aesthetic_score in Images Browser, it doesn't understand to include the new images tagged as, "Score" using that filter. Since those two scoring methods are essentially doing the same thing, IMO this would be confusing for the end-user. Hence why I was trying to shoehorn and change the aesthetic-scorer to output the tag, "aesthetic_score" instead of "Score" so Images Browser can use it :)

I'm not sure what the best solution for this would be. On one hand, you don't want to confuse the two scoring methods since they are slightly different from each other (more on that below.) So is the answer to create a different sorting filter for it or incorporate both? I'm not sure.

According to aesthetic-scorer project's homepage:

This extension uses different method than Aesthetic Image Scorer extension which:

Uses modified https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/1831 implementation

[AlUlkesh]

I see. Yeah, we'd need some info in the meta data, on which scoring method was used. And then convert them to the same scale, if that is possible. Which it would be now, but if the next method introduces negative scores, I'm not so sure...

However, how likely is it that someone would be using two methods at the same time? I suspect you'd switch only once to the new method.

So perhaps a migrate function, that runs everything once through the new method?

[AlUlkesh]

BTW, the image browser currently handles both keywords interchangeably:

stable-diffusion-webui-images-browser/scripts/image_browser.py

Line 1205 in 3d2d0f4

aes_cache = wib_db.load_exif_data_by_key(aes_cache, "aesthetic_score", "Score")

[AlUlkesh]

I guess for a quick and dirty workaround you could modify the script for the new method to divide the score by 2...

[rebootjac]

I know we are talking about minute changes between the two scoring methods.

BTW, the image browser currently handles both keywords interchangeably:

stable-diffusion-webui-images-browser/scripts/image_browser.py

Line 1205 in 3d2d0f4

aes_cache = wib_db.load_exif_data_by_key(aes_cache, "aesthetic_score", "Score")

Ah, so if the new method uses 1-10, we can read that value in divided by 2. That makes sense.

I know for me, I would prefer the ability to lump the two scoring methods together under that filter somehow. Maybe to future-proof with the new UI coming have a dialog box so that the end-user could define and add their own tags to it?

Great ideas, thank you.

[AlUlkesh]

Maybe to future-proof with the new UI coming have a dialog box so that the end-user could define and add their own tags to it?

Yes, definitely some kind of tagging system.

Thanks.