feat: saving and re-using fine-tuned models

[jmoralez]

Adds the following:

• NixtlaClient.finetune to fine tune and save a model.
• NixtlaClient.finetuned_models to list the models a user has fine tuned.
• NixtlaClient.delete_finetuned_model to delete an existing model.
• The finetuned_model_id argument to all methods to use a saved fine tuned model to compute forecasts.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[github-actions]

Experiment Results

Experiment 1: air-passengers

Description:

variable	experiment
h	12
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	12.6793	11.0623	47.8333	76
mape	0.027	0.0232	0.0999	0.1425
mse	213.936	199.132	2571.33	10604.2
total_time	1.3671	0.817	0.0043	0.0032

Plot:

Experiment 2: air-passengers

Description:

variable	experiment
h	24
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	58.1031	58.4587	71.25	115.25
mape	0.1257	0.1267	0.1552	0.2358
mse	4040.21	4110.79	5928.17	18859.2
total_time	0.7608	1.185	0.0037	0.0033

Plot:

Experiment 3: electricity-multiple-series

Description:

variable	experiment
h	24
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	178.293	268.13	269.23	1331.02
mape	0.0234	0.0311	0.0304	0.1692
mse	121589	219485	213677	4.68961e+06
total_time	0.7417	2.5719	0.0046	0.0041

Plot:

Experiment 4: electricity-multiple-series

Description:

variable	experiment
h	168
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	465.497	346.972	398.956	1119.26
mape	0.062	0.0436	0.0512	0.1583
mse	835021	403760	656723	3.17316e+06
total_time	1.0131	0.9656	0.0048	0.0042

Plot:

Experiment 5: electricity-multiple-series

Description:

variable	experiment
h	336
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	558.673	459.757	602.926	1340.95
mape	0.0697	0.0565	0.0787	0.17
mse	1.22723e+06	739114	1.61572e+06	6.04619e+06
total_time	1.228	1.0667	0.0049	0.0044

Plot:

[AzulGarza]

very cool pr!

when using distributed frameworks, we would expect an error. should we raise a particular error in that case?

[jmoralez]

Trying to fine tune on a distributed df?

[marcopeix]

Unsolicited comments/questions:

• Why is num_partitions not supported for the finetune method? Would this limit the finetuning process on small-ish datasets?
• Is it relevant to add a test that compares finetuning in the forecast method and finetuning separately and using the finetuned model to ensure the predictions are the same?

[jmoralez]

• Supporting partitioning would fine tune num_partitions models on unknown portions of the original data. I don't think that's a good use case.
• We already have that test in the API.

[AzulGarza]

lgtm:)

[elephaint]

Nice, pretty awesome! output_model_id (and thus later on finetuned_model_id) can be any user chosen string, right? I suspect users will typically want to name their models with a custom string. I think we'll have to emphasize that in the docs following this PR

[jmoralez]

output_model_id (and thus later on finetuned_model_id) can be any user chosen string, right?

Yes, if provided that one is used, otherwise we generate a random UUID.

[farukalamai]

I am finetuning the model using my custom dataset. Now how can I save the model like in pkl file or joblib? Here is my code

'''
forecast_df = nixtla_client.forecast(
df=train_df,
X_df=future_ex_vars_df,
h=len(future_ex_vars_df),
id_col='unique_id',
target_col='y',
time_col='ds'
)
'''

@jmoralez

[jmoralez]

@farukalamai https://nixtlaverse.nixtla.io/nixtla/docs/tutorials/reusing_finetuned_models.html