This script will scan image files in your Hydrus instance for AI metadata (prompts and workflows). It will then tag those images with appropriate tags, that way they can be searched in Hydrus and easily identified in Duplicate Processing.
ComfyUI workflows will be tagged like 'meta:ai embedded comfyui workflow' for example.
Only JPEG and PNG files are scanned.
This script will NOT add tags used in a prompt.
Python 3
pillow (pip install --upgrade pillow)
python hydrus_ai_metadata_finder.py --hydrus-url "http://192.168.1.35:45960" --hydrus-api-key "a2ad91f39ed311fe5d5a7fc7d102e6187d22jj9e131f8c9b01ee1e526ddc9d23" --tag-domain "my tags" --tags "meta:ai generated"
Provided API key must have rights to read images and write tags.
--tag-domain option has to be given tag domain name you wish to write resulting tags to. Typically it's either 'my tags' or 'downloader tags'.
--tags option can be given multiple tags separated by comma.
Like "meta:ai generated,system:archived time: since 2024-01-01"
Another option available is --run-tests (-r) which will run tests on images in 'tests' folder placed alongside hydrus_ai_metadata_finder.py
Hashes of images with no AI metadata will be written to db/ignored_database.db.
Hashes of images with AI metadata will be written to db/processed_database.db.
If any AI metadata was found at all then 'meta:ai generation metadata' tag will be added. 'meta:ai generated' tag will be added as well.
The script is really slow. It's best to place the script onto SSD and compile it with Nuitka. Even in such conditions ~1.600.000 images were processing for around 2.5 days of non-stop working. Skipping already processed and ignored hashes is super fast at least.
However, if your Hydrus instance has a lot of AI generated hentai images you can download this 'ignored_database.db' (https://drive.google.com/file/d/1IcWZCffstvfnIhmfivNMBOa6nznjxf_m/view?usp=drive_link)
Put it into 'db' folder before your first start, might save you a couple of days of processing.