[Pixiv] Doesn't want to download images past a certain point.

[Deejay85]

Whenever I try to download anything from Pixiv, it works for a while, up until it hits a certain number of images, and then stops. Running this in verbose mode this is the result I got before the error hit.

l:\Pixiv\Bloated Belly\134489967_p14 - 吉田優子,吉田清子.png
l:\Pixiv\Bloated Belly\134489967_p15 - 吉田優子,吉田清子.png
l:\Pixiv\Bloated Belly\134489897_p0 - mermaid girl. fish interspecies.jpg
[urllib3.connectionpool][debug] https://app-api.pixiv.net:443 "GET /v1/search/illust?word=%E3%83%9C%E3%83%86%E8%85%B9+r-18&search_target=partial_match_for_tags&sort=date_desc&offset=5010 HTTP/1.1" 400 147
[pixiv][debug] {'error': {'user_message': '', 'message': '{"offset":["offset must be no more than 5000"]}', 'reason': '', 'user_message_details': {}}}
[pixiv][error] API request failed: {'user_message': '', 'message': '{"offset":["offset must be no more than 5000"]}', 'reason': '', 'user_message_details': {}}

I have no idea what the "offset must be no more than 5000" means. Is it referring to the number of images downloaded, or something else? Also when I tried using the range command as a test (70001-80000), I get the same error message, usually in the exact same spot.

[mikf]

Search results are limited to the most recent 5000 results.

[Deejay85]

I checked the config file and didn't see anything mentioning a limit, so I imagine that you mean this is on the API side, right? Is there a way around it since using --range doesn't produce any better results? AFAIK I do not think there is a way to input a certain date to help narrow things down, is there?

[mikf]

As it turns out, there is: The scd & ecd query parameters:

https://www.pixiv.net/en/tags/女の子/artworks?scd=2025-07-07&ecd=2025-07-14

You can get them by selecting a "Period" under "Search option":
(or you add them manually)

[Deejay85]

Just to make sure, I did some quick math, and at 60 images a page, 5000/60 = 83.3, so grab the date from the end of page 83, and start all over again? I think I can do that. One question though...why is searches limited to 5000? Is it a change Pixiv made to their site?

[mikf]

Is it a change Pixiv made to their site?

This limit has always existed: #1686

[Deejay85]

Uh oh. One slight problem here, chief: the link isn't working for some reason. The following error message reads as follows:

'ecd' is not recognized as an internal or external command,
operable program or batch file.
's_mode' is not recognized as an internal or external command,
operable program or batch file.

I'm not sure what happened to cause this, but here is what I typed in the command prompt to cause this:

gallery-dl -d "l:" https://www.pixiv.net/en/tags/ボテ腹%20r-18/artworks?scd=2025-08-10&ecd=2025-09-01&s_mode=s_tag

I removed the --range command to see if that would change anything for testing purposes, and no, it didn't...same error message.

EDIT: Nevermind...added some quotation marks to the link, and it started chugging right along. I'll leave this faux pas up because I know I'm not the only one who will have this problem. Do have one last quick question though...why is it that logging in takes so many retries? I tried running multiple instances of gallery-dl, and it all worked fine until I opened the fourth window. Yeah it worked...eventually, but it's still annoying when your trying to run multiple instances of the same program to save on time.

[mikf]

602c84e should make it possible to go past offset 5000 without having to manually update any query parameters.