Issue: Bulk Export takes very long time with small number of traces

[dast-draftwise]

Hi,
I am using the bulk export feature as per documentation.
After creating S3 destination I check the status of the export job using this script

curl --request GET \
  --url 'https://api.smith.langchain.com/api/v1/bulk-exports/{export_id}' \
  --header 'Content-Type: application/json' \
  --header 'X-API-Key: YOUR_API_KEY' \
  --header 'X-Tenant-Id: YOUR_WORKSPACE_ID'

the result are as follows

{
  "bulk_export_destination_id": "ceb0XXXX",
  "session_id": "9a04XXXXXXX",
  "start_time": "2025-10-28T00:00:00+00:00",
  "end_time": "2025-10-29T23:59:59+00:00",
  "filter": "and(eq(is_root, true), eq(feedback_key, \"prompt_intent\"))",
  "format": "Parquet",
  "compression": "gzip",
  "interval_hours": null,
  "id": "4a09bXXXXXX",
  "tenant_id": "6bd3XXXXXX",
  "status": "Running",
  "created_at": "2025-10-29T10:55:36.008282+00:00",
  "updated_at": "2025-10-29T10:56:01.616411+00:00",
  "finished_at": null,
  "source_bulk_export_id": null
}

I am exporting 1 day worth of traces (~600 traces in this case), I can see that the traces are present in the S3 bucket (not sure if all of them) but whenever I query the status with the details it returns Running state.
It's been 1h and the status doesn't change.
This causes 2 issues:

• Even if all the traces are in the S3 how they should, as long as the status is not Completed, there is no visibility to whether task finished or not. This is very problematic for proper data pipeline orchestration.
• I am testing on 1 day (~600) traces, eventually would like to export all the traces, which is more than 50K. If it takes so long with small number, the export of 50K is not feasible.

Can someone please look into it, and check if that might be a bug on LangSmith side?
Thank you

[dast-draftwise]

Here you can see, its running super long, even though each parquet file is very small - few MB

[bvs-langchain]

Hi @dast-draftwise, I see ~900k rows exported for this bulk export, from 6:55am-8am ET today. this makes me think each trace must have many runs, as each row corresponds to a run, not a trace, in the export. can you confirm whether or not that is the case or if there is any duplicate data in the export?

[dast-draftwise]

I did two, exports. The first one I canceled but the second one started at 7:17 AM ET time, and it hasn't completed until now (20h+), exportId: c1d64fad-786a-4b53-bad6-4204c54296f6
The data is consists of agentic traces, so yes, they do contain sub-runs.
For that specific export, we are talking about ~600 agentic runs each contains on average 20 sub-runs, which gives us total of 2400 runs, no where near 900k.

From aws s3:

• first parquet write: October 29, 2025, 13:18:09 (UTC+01:00)
• latest (and still going): October 30, 2025, 09:06:34 (UTC+01:00)

[dast-draftwise]

@bvs-langchain
There are duplicates, please see details below.
I looked at one of the parquet files and here is what I see.

Analysis of single parquet

>>df.shape 
(500, 29)

>>df.id.nunique()
82

>>df.id
0      8d8f93b5-0590-4337-90eb-572d6d9a7241
1      d559ec00-acdd-4763-b3df-104593c7cd8b
2      198a0254-f719-4314-a6e3-368527038788
3      307c00c9-4b36-4079-8fad-d06f8bc6887e
4      32b9529c-176a-41b1-94b1-d6f0d7b01e95
                       ...                 
495    c1541bff-da7e-4c8c-98f0-c36d4d0a9a95
496    c1541bff-da7e-4c8c-98f0-c36d4d0a9a95
497    c1541bff-da7e-4c8c-98f0-c36d4d0a9a95
498    c1541bff-da7e-4c8c-98f0-c36d4d0a9a95
499    c1541bff-da7e-4c8c-98f0-c36d4d0a9a95
Name: id, Length: 500, dtype: object

>>df[df.id == "c1541bff-da7e-4c8c-98f0-c36d4d0a9a95"].shape
(419,29)

Conclusion
Given 500 rows I would expect 500 unique id i.e. run_id instead I see 82. The export details do not correspond to what I see in UI.

Within that run (screenshot) there are 8 sub-runs (its agent workflow run)

I tested also 2 other parquet files. One of them had all the id unique, so 500. But third one was full of duplicates, out of 500 rows of data all of them had same id and trace_id and had exactly the same content.

[dast-draftwise]

Hi,
Any update on this?