[CivitAI] 400 Bad Request. mb just something on their side?

[Ikkoru]

Only happens on some users.

[urllib3.connectionpool][debug] https://civitai.com:443 "GET /api/trpc/post.getInfinite?input=%7B%22json%22%3A%7B%22browsingLevel%22%3A31%2C%22period%22%3A%22AllTime%22%2C%22periodMode%22%3A%22published%22%2C%22sort%22%3A%22Newest%22%2C%22followed%22%3Afalse%2C%22draftOnly%22%3Afalse%2C%22pending%22%3Atrue%2C%22include%22%3A%5B%22cosmetics%22%5D%2C%22username%22%3A%22Nicows%22%2C%22cursor%22%3A%222024-08-14T19%3A10%3A27.421Z%7C5427833%22%7D%2C%22meta%22%3A%7B%22values%22%3A%7B%22cursor%22%3A%5B%22Date%22%5D%7D%7D%7D HTTP/1.1" 400 705
[civitai][error] HttpError: '400 Bad Request' for 'https://civitai.com/api/trpc/post.getInfinite?input=%7B%22json%22%3A%7B%22browsingLevel%22%3A31%2C%22period%22%3A%22AllTime%22%2C%22periodMode%22%3A%22published%22%2C%22sort%22%3A%22Newest%22%2C%22followed%22%3Afalse%2C%22draftOnly%22%3Afalse%2C%22pending%22%3Atrue%2C%22include%22%3A%5B%22cosmetics%22%5D%2C%22username%22%3A%22Nicows%22%2C%22cursor%22%3A%222024-08-14T19%3A10%3A27.421Z%7C5427833%22%7D%2C%22meta%22%3A%7B%22values%22%3A%7B%22cursor%22%3A%5B%22Date%22%5D%7D%7D%7D'
[civitai][debug]
Traceback (most recent call last):
  File "gallery_dl\job.py", line 158, in run
  File "gallery_dl\job.py", line 209, in dispatch
  File "gallery_dl\extractor\civitai.py", line 82, in items
  File "gallery_dl\extractor\civitai.py", line 888, in _pagination
  File "gallery_dl\extractor\civitai.py", line 864, in _call
  File "gallery_dl\extractor\common.py", line 287, in request_json
  File "gallery_dl\extractor\common.py", line 278, in request
gallery_dl.exception.HttpError: '400 Bad Request' for 'https://civitai.com/api/trpc/post.getInfinite?input=%7B%22json%22%3A%7B%22browsingLevel%22%3A31%2C%22period%22%3A%22AllTime%22%2C%22periodMode%22%3A%22published%22%2C%22sort%22%3A%22Newest%22%2C%22followed%22%3Afalse%2C%22draftOnly%22%3Afalse%2C%22pending%22%3Atrue%2C%22include%22%3A%5B%22cosmetics%22%5D%2C%22username%22%3A%22Nicows%22%2C%22cursor%22%3A%222024-08-14T19%3A10%3A27.421Z%7C5427833%22%7D%2C%22meta%22%3A%7B%22values%22%3A%7B%22cursor%22%3A%5B%22Date%22%5D%7D%7D%7D'
[civitai][debug] Using CivitaiUserImagesExtractor for 'https://civitai.com/user/Nicows/images'
[civitai][debug] Using tRPC API
[civitai][debug] Using api_key authentication
[civitai][debug] Sleeping 2.06 seconds (request)
[urllib3.connectionpool][debug] https://civitai.com:443 "GET /api/trpc/image.getInfinite?input=%7B%22json%22%3A%7B%22useIndex%22%3Atrue%2C%22period%22%3A%22AllTime%22%2C%22sort%22%3A%22Newest%22%2C%22withMeta%22%3Afalse%2C%22fromPlatform%22%3Afalse%2C%22browsingLevel%22%3A31%2C%22include%22%3A%5B%22cosmetics%22%5D%2C%22types%22%3A%5B%22image%22%5D%2C%22username%22%3A%22Nicows%22%2C%22cursor%22%3Anull%7D%2C%22meta%22%3A%7B%22values%22%3A%7B%22cursor%22%3A%5B%22undefined%22%5D%7D%7D%7D HTTP/1.1" 200 None

[mikf]

Definitely something one their side. It is working (again?) for me:

$ gallery-dl https://civitai.com/user/Nicows/images
./civitai/Nicows/images/111079043.png
./civitai/Nicows/images/111078997.png
./civitai/Nicows/images/110578719.png

[Ikkoru]

./civitai/Nicows/images/111079043.png

Sorry didn't mention, it only happens after a while; not right away.

[mikf]

OK, I can replicate the 400 Bad Request error after 100 posts from https://civitai.com/user/nicows/posts, but the website itself doesn't load posts after this point either even though this user apparently has 171.

[mikf]

Here's the formatted error message from the 400 response:

  {
    "code": "invalid_union",
    "errors": [
      [
        {
          "expected": "number",
          "code": "invalid_type",
          "path": [],
          "message": "Invalid input: expected number, received Date"
        }
      ],
      [
        {
          "expected": "string",
          "code": "invalid_type",
          "path": [],
          "message": "Invalid input: expected string, received Date"
        }
      ]
    ],
    "path": [
      "cursor"
    ],
    "message": "Invalid input"
  }

[mikf]

Found the problem:

diff --git a/gallery_dl/extractor/civitai.py b/gallery_dl/extractor/civitai.py
index 34983967..ce45c4f8 100644
--- a/gallery_dl/extractor/civitai.py
+++ b/gallery_dl/extractor/civitai.py
@@ -798,3 +798,2 @@ class CivitaiTrpcAPI():
         endpoint = "post.getInfinite"
-        meta = {"cursor": ("Date",)}
 
@@ -813,4 +812,3 @@ class CivitaiTrpcAPI():
         params = self._type_params(params)
-        return self._pagination(endpoint, params, meta,
-                                user=("username" in params))
+        return self._pagination(endpoint, params, user=("username" in params))
 

---

but the website itself doesn't load posts after this point either even though this user apparently has 171.

That happened because I wasn't logged in...