Skip to content

Commit 711abbc

Browse files
looputDDVD233
authored andcommitted
[data] fix: update parquet_files type check to support multi-file input (volcengine#3211)
1 parent e3d8a51 commit 711abbc

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

verl/utils/dataset/multiturn_sft_dataset.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
import numpy as np
2323
import pandas as pd
2424
import torch
25+
from omegaconf import ListConfig
2526
from torch.utils.data import Dataset
2627
from transformers import PreTrainedTokenizer
2728

@@ -60,7 +61,7 @@ def __init__(self, parquet_files: str | list[str], tokenizer, config=None):
6061
self.apply_chat_template_kwargs = config.get("apply_chat_template_kwargs", {})
6162
assert self.truncation in ["error", "left", "right"]
6263

63-
if not isinstance(parquet_files, list):
64+
if not isinstance(parquet_files, list | ListConfig):
6465
parquet_files = [parquet_files]
6566

6667
self.parquet_files = parquet_files

0 commit comments

Comments
 (0)