Replies: 3 comments 4 replies
-
|
#8472 doesn't help in this situation?
There is no "duplicate" option.
It doesn't. gallery-dl processes items on the fly. |
Beta Was this translation helpful? Give feedback.
-
|
Apologies, not the 'duplicate' but the skip option is already set. As to processing things on the fly, I can't see how that's working with the question. If I have an account to update (already did the preliminary download), and it has only about 500 entries, with 6 new items, that takes only a couple of minutes. If I'm on the 20K account, and there are only 25 new items, it takes upwards of an hour or more before it starts downloading anything. |
Beta Was this translation helpful? Give feedback.
-
|
Closing topic as the fix is to remove the variables mentioned above. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm having to not be able to check first again because I'm not sure how to phrase this inquiry.
Is there a config option to limit the number of return headers on one of the API extractors? I listed it for DA above because it's the one I know about, but I'm sure there could be others that might apply here.
What I'm trying to do is to reduce, sharply, the amount of time needed for the API. In DA, there are several accounts that I have marked for downloading that have literally 10s of thousands of media entries. I already set the duplicate option to 5, but it seems like the API call must first download the list of every single item first. If it could be set to only get 100 entries at a time, that would definitely accelerate the download delays. (I'm currently using Task Til Dawn to automate the firing off of the various extracts/sites. But because of how long it takes to just start comparing the returned URLs, I had to break it into 3 different jobs. At least two of those jobs take well over 14 hours each time it runs.)
Any insights would be greatly appreciated.
Ssurgul
Beta Was this translation helpful? Give feedback.
All reactions