Pixiv downloading "Work cannot be displayed" image

[501stRookie]

Starting today, when I tried to download an image from Pixiv instead of downloading the image, it instead downloads this image with Japanese text that says "This work cannot be displayed".

It seems to only happen on posts that were recently posted, as images that were uploaded yesterday and older download fine.

[thatfuckingbird]

Can confirm, seeing this too. Refreshing the login token doesn't help and it doesn't seem to be related to whether the image is r18 or not.
The URL I get for this image is https://s.pximg.net/common/images/limit_sanity_level_360.png
The metadata json seems to be mostly fine, but it also contains a "sanity_level": 4, entry.

Update:
Tried PixivUtil2, it doesn't have this problem.

Update2:
Tried the official Pixiv app, these images also do not show up there. The last displayed images (if I short by "newest") are from yesterday. Except ugoira, those show for some reason.

Also might be related issue here: upbit/pixivpy#275

Update3:

If you check https://www.pixiv.net/info.php?cid=1&lang=en there are announcements about the suspension and reinstantiation of their mobile apps from app stores, related to content in the apps. I think they might be doing some kind of semi-manual filtering now which causes this lag between the mobile app API and the website.
This might mean we can no longer use that API, at least for downloading the images themselves.
Also for the future, it might be a good idea to detect that limit_sanity_level placeholder image and error on it.

Update4:

The metadata is incomplete too for these images (no tags).

[Slider-Whistle]

Doesn't appear to be happening on my end.

[kattjevfel]

I ran into this but I can no longer reproduce it, so must've been something temporary.

[thatfuckingbird]

It is still happening (as I'm writing this), but the lag between what is available on the mobile app API and what's visible on the site has decreased. Currently I see about a 8-10 minute lag until an image shows up on the mobile app (looking at the posting times). You can reproduce this if you go to the site, search for some very common tag like "illustration" and try to download the newest entry chronologically (check if it was posted in the last few mins).

The other question is, is there any content that won't be available on the mobile API at all? I haven't encountered anything like that yet but since this whole thing might be because the mobile app does some additional filtering due to appstore requirements then it can't be discounted.

For now I think a temporary solution would be to catch these cases when an invalid image is returned (easy from the URL) and either error or try to wait 5-10-15 mins like in the case of rate limits. If the lag between the site and image availability in the API remains low then this might be enough, maybe along with some informational message in these cases.

Ultimately if the time lag between the mobile API and the site keeps randomly increasing/decreasing or the mobile API becomes filtered in some other way then a switch to the non-mobile API (the one the website uses) might be needed.

[Dartkun]

Still happening on my end. Also only for basically brand new pictures.

[alleneko]

Happens for me when I'm downloading from my bookmarks, but doesn't happen when I use the search page or individual posts.

[mikf]

https://s.pximg.net/common/images/limit_sanity_level_360.png images now get ignored
(a45a17d (yes, that's the wrong issue number ...))

To manually ignore them, enable url-metadata and --filter them that way.

The other question is, is there any content that won't be available on the mobile API at all?

I've noticed that search results, and only those, do not include R-18G works.

[thatfuckingbird]

I've noticed that search results, and only those, do not include R-18G works.

Works for me, it might be your account settings (there is a separate toggle for r18g iirc)

[mikf]

These settings are enabled for all of my accounts.
It is working again, but it definitely wasn't when I posted #4327 (comment).

Are these "Work cannot be displayed" images still a thing or did Pixiv somehow fix whatever these were meant for?

(I've never encountered one of these or a "Skipping 'sanity_level' warning" logging message myself)

[thatfuckingbird]

There's still at least a few minutes of lag before images displayed on the website also appear in the app, so if you happen to download very recent image URLs those will still produce the sanity_level image. I think we will just have to live with this for the time being, since it probably not worth a rewrite to switch to the API that the website uses.

[mikf]

Yeah, I'd really want to avoid using the website API if at all possible. It is a lot slower, requires an extra request for each individual post, and, more importantly, would need exported cookies for authentication, which expire in a month or so.

I did try to rewrite the current extractor back when auth with username and password got disabled and it wasn't a "pleasant" experience, to say the least.

[AlttiRi]

Just a short resume of #4421 (comment)

In Pixiv's Android application, and therefore in gallery-dl too:

• There are shadow banned images, they are hidden from profiles entirely (/v1/user/illusts?user_id= just does not return any information for them), but they are still listed in bookmarks (/v1/user/bookmarks/illust?user_id=) with a dummy thumbnail in the app. Can't be downloaded even with an artwork url (/v1/illust/detail?illust_id=). The endpoint returns a trimmed response with "visible": false.
• There are "soft" shadow banned images. As usual images, but only with the missed caption (description).

[AlttiRi]

Seems, the caption "bug" is "fixed".
But some images are still with "visible": False, gallery-dl does not see them when it downloads a profile's images.

---

Upd 2023.10.08: The "bug" was returned.