Abort when losing proxy connection

[ss71651]

Hi,

I use a proxy for downloading from TikTok, occasionally it fails to connect to the proxy. Would it be possible to add an option that aborts the process when that happens? Otherwise it will get my ip blocked by half the websites if I don't use a proxy for TikTok.

After some time without proxy:

Unable to download webpage: Failed to perform, curl: (28) Connection timed out after 30005 milliseconds. See https://curl.se/libcurl/c/libcurl-errors.html first for more details. (caused by TransportError('Failed to perform, curl: (28) Connection timed out after 30005 milliseconds. See https://curl.se/libcurl/c/libcurl-errors.html first for more details.'))

After some time with proxy:

Unable to download webpage: Failed to perform, curl: (28) Failed to connect to xxx.xxx.xxx.xxx port 59100 after 21044 ms: Could not connect to server. See https://curl.se/libcurl/c/libcurl-errors.html first for more details. (caused by TransportError('Failed to perform, curl: (28) Failed to connect to xxx.xxx.xxx.xxx port 59100 after 21044 ms: Could not connect to server. See https://curl.se/libcurl/c/libcurl-errors.html first for more details.'))

Both lead to connection problems with many websites, as soon as I close the command line the connection problems stop.

Best regards

[AlttiRi]

If you use something like

gallery-dl.exe --proxy localhost:12345 https://www.tiktok.com/@user1 https://site2.com/@user2 https://site3.com/@user3

The (not working) proxy will affect on all listed sites.

You need to set the proxy only for the needed extractor:

-o "extractor.tiktok.proxy=localhost:12345"

(the errors above are from yt-dlp, not from gallery-dl, btw.)

However, it seems there is no option to terminate gallery-dl after N failed extractors.

---

Also, you can set the proxy only for yt-dlp:

-o "downloader.ytdl.raw-options.proxy=localhost:12345"

[mikf]

It should be possible to detect logging messages regarding (proxy) errors and calling abort() when they happen using https://gdl-org.github.io/docs/configuration.html#extractor-actions

{
    "extractor": {
        "tiktok": {
            "actions": {
                "*:PROXY MESSAGE HERE": "abort"
            }
        }
    }
}

(replace PROXY MESSAGE HERE with the actual message)

[ss71651]

It should be possible to detect logging messages regarding (proxy) errors and calling abort() when they happen using https://gdl-org.github.io/docs/configuration.html#extractor-actions

{
"extractor": {
"tiktok": {
"actions": {
"*:PROXY MESSAGE HERE": "abort"
}
}
}
}
(replace PROXY MESSAGE HERE with the actual message)

Can I do it like this or how exactly does the syntax work with that?

"actions": {
"*:%Failed to connect to%": "abort"
},

[mikf]

Why did you add % characters?

 "*:Failed to connect to": "abort"

[ss71651]

Why did you add % characters?

"*:Failed to connect to": "abort"

Thank you! And when I want multiple different error messages to abort?

[mikf]

Update the regex with | to add alternatives,

            "actions": {
                "*:Failed to connect to|error 2|error 3": "abort"
            }

or just add multiple entries to actions:

            "actions": {
                "*:Failed to connect to": "abort",
                "*:error 2": "abort",
                "*:error 3": "abort"
            }

[ss71651]

Thanks for your help so far but I can't get it to abort when the error occurs

[mikf]

Then why didn't you mention this in #8753 (comment)?

[AlttiRi]

Try to use terminate or exit instead of abort.

[mikf]

abort has been broken since 2eb5e52, as it turns out.

[mikf]

Fixed in fd6f193

For the time being, #8753 (comment) is your best bet.

[ss71651]

@AlttiRi @mikf Thank you both for your help!