Skip to content

Conversation

@HarvestStars
Copy link
Contributor

@HarvestStars HarvestStars commented Oct 28, 2025

Closes #345

Hi @abdulfatir
Here is the bugfix about the function "validate_and_prepare_single_dict_task", which had 2 issue points:

  1. Originally, one of this func return, the "task_n_future_covariates", will return the ["past only" + "future known"]covariates number, by task_n_future_covariates = len(task_future_covariates_list) as task_future_covariates_list is filled by for key in task_covariates_keys
  2. The code seems not to guarantee the last "future known" rows are atcually what we expected, even there is a sorted option.

So, this PR fixed them by separating "past only" and "future known" covs from the "past_covariates" input, and explicitly put the "past only" covs rows above "future known" cov rows, supported by a temp list "ordered_covariate_keys".

Copy link
Contributor

@abdulfatir abdulfatir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@HarvestStars Thanks a lot for catching and fixing this! Left a few minor comments. Could you please address those? I will also run some evals on my side before merging to ensure that we did not break anything accidentally.

@HarvestStars
Copy link
Contributor Author

Hi @abdulfatir
Thanks for replying so fast, as well the comment for name standarization.
I think they make much sense, so I fixed them following your comment.
Please double check.

@abdulfatir
Copy link
Contributor

abdulfatir commented Oct 29, 2025

@HarvestStars Thanks that looks good to me. Could you also please format the code? You can do that by running.

uvx ruff format src

See: https://docs.astral.sh/uv/guides/tools/

@abdulfatir
Copy link
Contributor

I re-ran the evals and numbers match. In any case, this issue mainly affects fine-tuning. Thank you for finding this and please feel free to open other issues you encounter with fine-tuning. We have not rigorously tested fine-tuning support yet, that's why it has been marked as experimental.

@HarvestStars
Copy link
Contributor Author

Hi @abdulfatir
I've already done the format by running
uvx ruff format src
And also update the branch, please check.

@abdulfatir abdulfatir force-pushed the bugfix_dataset_future_cov branch from 226ebd5 to a673a12 Compare October 29, 2025 12:27
@abdulfatir abdulfatir added the run-eval Run evaluation CI workflow label Oct 29, 2025
@abdulfatir abdulfatir merged commit 6c69155 into amazon-science:main Oct 29, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-eval Run evaluation CI workflow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect task_n_future_covariates Counting and Misordered Covariate Rows in validate_and_prepare_single_dict_task

2 participants