How to make sure your twitter/X account don't get suspended while using cookies?

[rarelygoeshere]

Hello, for some reason, while I was using cookies, Twitter/X now suspends your account if you scrape for a while, typically while scraping large accounts. Thanks to this, I got my main twitter account and an alt suspended while scraping. I don't know if it's because I was using an older version, because when I upgraded my gallery-dl, I found out I was using gallery_dl-1.27.4 rather than 1.27.7. If that wasn't the cause of my problem, can anyone give me tips on how to make sure I can scrape Twitter/X more smoothly in a way that won't get my accounts suspended? Thank you.

[rarelygoeshere]

Well, in the time after I asked this, I found out that apparently Twitter/X really hates scraping now, with doing it being an insta-ban. So I guess it's probably not the fault of my old Gallery-dl version that got me suspended.

[MikeRich88]

I am using cookies and I am not being banned.

[rarelygoeshere]

Really? How did you manage it? Is it related to any config you set up?

[Jeddostotle7]

This was exactly what stopped my attempt to continue mass-downloading Twitter late last year, and I've been procrastinating on trying again because I don't know if there's a solution/workaround or not.

[Illusivehacks]

This was exactly what stopped my attempt to continue mass-downloading Twitter late last year, and I've been procrastinating on trying again because I don't know if there's a solution/workaround or not.

Use different IP addresses (proxy rotation) and Implement a persistent queue system that resumes after failures.

[Jeddostotle7]

Oh, at this point I've used some of the other advice for Twitter downloading and managed to download most everything I was planning to, but those are some good additional ideas I'll take into consideration.

[Illusivehacks]

Oh, at this point I've used some of the other advice for Twitter downloading and managed to download most everything I was planning to, but those are some good additional ideas I'll take into consideration.

Great 🫱🏻‍🫲🏼

[rarelygoeshere]

Oh, at this point I've used some of the other advice for Twitter downloading and managed to download most everything I was planning to, but those are some good additional ideas I'll take into consideration.
May I ask if you can share those advice? I've been stalling on mass Twitter scraping for a while now so I'd really love to get in on some working methods that wouldn't get my account banned

[Jeddostotle7]

May I ask if you can share those advice? I've been stalling on mass Twitter scraping for a while now so I'd really love to get in on some working methods that wouldn't get my account banned

I amalgamated some advice from discussions/issue threads like these:
#5775
#5532
#7162
and others

Stuff like: use an account that has behaved like a normal user for a while (if it's a new account try using it like a normal account for a couple weeks, I just used my longtime main account), and keep using it like a normal account while doing the downloading; add sleep delays to slow down the rate of accessing new stuff, and in ranges of delays so it's not the same delay duration every time, so it's less likely to think it's a bot; taking breaks (which I did by just manually stopping gallery-dl and restarting it in a certain range of time every day; had it running ~3 PM to ~11 PM, then ~2 AM to ~1 PM, but varied the on and off times a bit every day); etc.

My current twitter setup looks like this:

"twitter":
{
    "cookies": "path:\\to\\cookie\\file.txt",
    "sleep": [1.6,11.4],
    "sleep-request": [2.1,15.2],
    "sleep-extractor": [2.8,17.7],

    "postprocessors": [
        {multiple metadata postprocessors to grab .jsons and .txts for all tweets, including text tweets; not relevant to the "not getting banned" question}
    ],
    "image-filter": "quote_id or retweet_id or (user is author)",
    "filename":{
        "locals().get('retweet_id')": "{retweet_id}_{num}.{extension}",
        ""                            : "{tweet_id}_{num}.{extension}"
    },
    "directory": ["twitter", "{author[name]}"],
    "conversations": "accessible",
    "text-tweets": true,
    "replies": true,
    "quoted": true,
    "retweets": true,
    "metadata": true

},