[tiktok] First draft of a challenge resolver#8850
Conversation
|
I'm noticing that after challenges are "resolved," I begin to encounter a few 403 responses. The solution needs a bit of tweaking. Maybe a small sleep after resolving the challenge is required? Or maybe there's a piece of the puzzle I'm missing. Edit: on second thoughts, the 403 responses may be from the better quality URLs (I'm currently testing with the old PR that doesn't provide fallbacks). To verify this I'll make sure to start testing with the latest fallback changes soon. Edit 2 (11 Jan): I have now began testing with all of my latest changes, I'll see if I keep running into the 403 responses. |
|
I do still run into the 403 errors, but much less frquently. Manual testing locally (as opposed to testing my PRs on a dedicated server overnight) allows me to get a clearer understanding of what happens when we do encounter a challenge response, and the resolver has worked every time I've encountered said response, with no 403 responses. So I am guessing that the 403s are due to excessive requests being made by the server. Edit to say that I have ended up seeing 403 responses a couple of times locally, too, but re-running the download commands always worked for me. |
|
If this PR ends up being unsuitable, then apparently the approach in this yt-dlp PR also fixes the issue, but it's a bit more work. |
|
I've done a bit of cleanup. I can't get a challenge myself, but it runs without exceptions on synthetic data. |
|
I will test again tonight, but looks good to me! I do still run into it occasionally so I'll let you know once everything works correctly. |
|
❗ The changes to range predicates have broken account extraction, btw (specifically around line 1191). I'll need to bring up another PR for that: #8931. |
Full disclosure, I largely copied/translated the JavaScript code to resolve the challenge ourselves. There are potentially a couple of problems with this:
I am currently in the processing of performing thorough testing on this PR (as well as all of my others), I will report soon whether or not this approach works.
Closes #8772.