[TikTok] could not decode response for this page, trying again / failed to retrieve post/item_list when using cookies

[docholllidae]

verbose output:

PS C:\Users\me\Downloads\!tools> .\gallery-dl.exe https://www.tiktok.com/@user938556219072 --verbose
[gallery-dl][debug] Version 1.31.2 - Executable (stable/windows)
[gallery-dl][debug] Python 3.13.7 - Windows-11-10.0.26100-SP0
[gallery-dl][debug] requests 2.32.5 - urllib3 2.6.2
[gallery-dl][debug] Configuration Files ['%APPDATA%\\gallery-dl\\config.json']
[gallery-dl][debug] Starting DownloadJob for 'https://www.tiktok.com/@user938556219072'
[tiktok][debug] Using TiktokUserExtractor for 'https://www.tiktok.com/@user938556219072'
[tiktok][debug] Sleeping 2.25 seconds (extractor)
[tiktok][debug] Using TiktokAvatarExtractor for 'https://www.tiktok.com/@user938556219072/avatar'
[tiktok][debug] cookies: Loading cookies from 'X:/My Drive/scrapers/TikTok/cookies.tiktok.txt'
[tiktok][debug] Sleeping 1.55 seconds (extractor)
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.tiktok.com:443
[urllib3.connectionpool][debug] https://www.tiktok.com:443 "GET /@user938556219072 HTTP/1.1" 200 None
X:\My Drive\scrapers\TikTok\Downloads\7394905381433099310-user938556219072\avatars\han-user938556219072_avatar-7394905381433099310.jpg
[tiktok][debug] Using TiktokStoriesExtractor for 'https://www.tiktok.com/@user938556219072/stories'
[tiktok][debug] cookies: Loading cookies from 'X:/My Drive/scrapers/TikTok/cookies.tiktok.txt'
[tiktok][debug] Sleeping 2.71 seconds (extractor)
[tiktok][info] https://www.tiktok.com/@user938556219072: retrieving story/item_list page 1
[urllib3.connectionpool][debug] https://www.tiktok.com:443 "GET /api/story/item_list/?aid=1988&app_language=en&app_name=tiktok_web&browser_language=en-US&browser_name=Mozilla&browser_online=true&browser_platform=Win32&browser_version=5.0%20%28Windows%29&channel=tiktok_web&cookie_enabled=true&device_id=7267925111982554594&device_platform=web_pc&focus_state=true&from_page=user&history_len=2&is_fullscreen=false&is_page_visible=true&language=en&os=windows&priority_region=&referer=&region=US&screen_height=1080&screen_width=1920&tz_name=UTC&verifyFp=verify_63466bf&webcast_language=en&cursor=0&authorId=7394905381433099310&loadBackward=false&count=5 HTTP/1.1" 200 4607
[tiktok][debug] Sleeping 2.48 seconds (request)
[urllib3.connectionpool][debug] https://www.tiktok.com:443 "GET /@user938556219072/video/7591275290408668430 HTTP/1.1" 200 None
X:\My Drive\scrapers\TikTok\Downloads\7394905381433099310-user938556219072\stories\han-@user938556219072_-_TikTok-7591275290408668430_0.mp4
[tiktok][debug] Using TiktokPostsExtractor for 'https://www.tiktok.com/@user938556219072/posts'
[tiktok][debug] cookies: Loading cookies from 'X:/My Drive/scrapers/TikTok/cookies.tiktok.txt'
[tiktok][debug] Sleeping 1.70 seconds (extractor)
[tiktok][info] https://www.tiktok.com/@user938556219072: retrieving post/item_list page 1
[tiktok][debug] Sleeping 1.22 seconds (request)
[urllib3.connectionpool][debug] https://www.tiktok.com:443 "GET /api/post/item_list/?aid=1988&app_language=en&app_name=tiktok_web&browser_language=en-US&browser_name=Mozilla&browser_online=true&browser_platform=Win32&browser_version=5.0%20%28Windows%29&channel=tiktok_web&cookie_enabled=true&device_id=7285918122557929049&device_platform=web_pc&focus_state=true&from_page=user&history_len=2&is_fullscreen=false&is_page_visible=true&language=en&os=windows&priority_region=&referer=&region=US&screen_height=1080&screen_width=1920&tz_name=UTC&verifyFp=verify_8ac9d36&webcast_language=en&cursor=0&secUid=MS4wLjABAAAAdBT1fG-9QbIGt89ZpYBKF0y-qkDvBeHHW3BYX1-Ae2ZlaPf3NHV3TMx4nWs0w9KY&post_item_list_request_type=0&count=15&needPinnedItemIds=false HTTP/1.1" 200 0
[tiktok][warning] Could not decode response for this page, trying again
[tiktok][debug] Sleeping 2.04 seconds (request)
[urllib3.connectionpool][debug] https://www.tiktok.com:443 "GET /api/post/item_list/?aid=1988&app_language=en&app_name=tiktok_web&browser_language=en-US&browser_name=Mozilla&browser_online=true&browser_platform=Win32&browser_version=5.0%20%28Windows%29&channel=tiktok_web&cookie_enabled=true&device_id=7285918122557929049&device_platform=web_pc&focus_state=true&from_page=user&history_len=2&is_fullscreen=false&is_page_visible=true&language=en&os=windows&priority_region=&referer=&region=US&screen_height=1080&screen_width=1920&tz_name=UTC&verifyFp=verify_63d8666&webcast_language=en&cursor=0&secUid=MS4wLjABAAAAdBT1fG-9QbIGt89ZpYBKF0y-qkDvBeHHW3BYX1-Ae2ZlaPf3NHV3TMx4nWs0w9KY&post_item_list_request_type=0&count=15&needPinnedItemIds=false HTTP/1.1" 200 0
[tiktok][warning] https://www.tiktok.com/@user938556219072: failed to retrieve post/item_list page 1
[tiktok][debug] Sleeping 30.00 seconds (retry)

when trying to download a user's profile/wall, if I have cookies set I get these two errors, which loops through 10 retries (presumably or coincidentally the number of retries i have set in my config) then quits
[tiktok][warning] Could not decode response for this page, trying again
[tiktok][warning] https://www.tiktok.com/@user938556219072: failed to retrieve post/item_list page 1

downloading individual posts from the user while cookies are set is successful
downloading the user's profile/wall when cookies are commented out is successful

I've tried logging out, clearing my cookies, logging in, then recapturing and using the fresh cookies
I've tried using cookies for another login
I've tried .\gallery-dl.exe --clear-cache TikTok

it is a public profile, and one of the accounts I used the cookies from even follows the profile

[pamelianda]

I have same problem. I've tried using yt-dlp, but dont get same problem

[Horladoyin]

I'm experiencing the same issue, although yt-dlp seems to be working fine.

[alvvaysu]

I wanted to say that before the nightly version gallery-dl@a6c845b, using cookies.txt and entering any TikTok user’s homepage would work fine for extraction and downloading, except for some incomplete warnings from the built-in yt-dlp.
I even suspect the current new version hasn’t undergone any reliability testing.

For me, yt-dlp cannot download photo posts, but gallery-dl could make up for that. I still mainly use yt-dlp now—if possible, I hope to only extract photo posts and ignore the rest.

[docholllidae]

photo posts have downloaded for me since many versions ago, certainly while it was still using yt-dlp

[KristyAnon]

I have the same issue, is there any solution for it?

[CasualYT31]

@docholllidae What region are you extracting with?

It looks as though the post/item_list endpoint does not work, but the creator/item_list endpoint does (which is the one yt-dlp uses, and the one gallery-dl uses if no cookies are provided). In my experience the endpoint fails when no or incorrect cookies are provided but there could be other reasons, too.

We should use creator/item_list as a fallback in case post/item_list doesn't work.

[CasualYT31]

You can skip downloading videos by passing -o videos=false to the command line. And you can do the same with audio: -o audio=false.

[docholllidae]

i don't have any region set in my config, but i'm in the US and not using any VPN

[CasualYT31]

Not sure why post/item_list doesn't work for others. But if we can detect that that endpoint is failing, we can use creator/item_list as a fallback.

[CasualYT31]

Huh, we already do that.
I see the problem, the fallback mechanism doesn't work, I'll fix that in my PR.
It seems like a good idea to allow the user to choose which endpoint to use, too.

[CasualYT31]

Okay, final solution is to just always use the creator/item_list endpoint, unless a non-descending extraction order is requested with the order-posts option and cookies are provided. If the latter fails, we will now properly fallback to the creator/item_list endpoint.