The labeled_token_indices

[weiguo-li]

The following is from example.py

if args.model_size == '7B':
    adjusted_token_ids = [1] + adjusted_token_ids # Adjusting to recover the first token_ids of the sentences
    adjusted_token_ids=torch.tensor([adjusted_token_ids])
    labeled_token_indices = labeled_token_indices[2:]  # Adjusting to skip the first two separators (beginning and ending of the problems)

My question is : when you add [1] to the "adjusted_token_ids", this will make every token in "the adjusted_token_ids" shift to the right and thus change the corresponding indices of "labeled_token_indices". So "labeled_token_indices" will shift right by one position. The above is my confusion.

[Shijie-Xia]

My apologies for the delay. Yes, after reviewing the code, my understanding is that from an inference perspective, this change makes the 7B models judge the correctness of the reasoning step without its last token. This should be fine to keep as is. The reason is that the last token is often a punctuation mark, so excluding it shouldn't impact the correctness judgment and thus shouldn't significantly affect performance.