Skip to content

[FIX] Deprecate insample endpoint#712

Merged
elephaint merged 10 commits intomainfrom
fix/deprecate_insample_endpoint
Dec 4, 2025
Merged

[FIX] Deprecate insample endpoint#712
elephaint merged 10 commits intomainfrom
fix/deprecate_insample_endpoint

Conversation

@elephaint
Copy link
Copy Markdown
Contributor

  • Deprecates in-sample endpoint in favor of using cross-validation endpoint
  • Fixes some warning messages to use the logger instead of warnings.warn

Note:
Tests fail until ltm-api that supports feature_contributions for cross-validation has been released.

@elessar-for-github
Copy link
Copy Markdown

Changes made by this pull request:

nixtla/nixtla_client.py
[Error processing diff]

nixtla_tests/conftest.py
[Error processing diff]

nixtla_tests/nixtla_client/test_nixtla_client.py
[Error processing diff]

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 29, 2025

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 0.9358 0.6865 0.0043 0.0034

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 0.7334 0.7031 0.0039 0.0034

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 178.293 268.13 269.23 1331.02
mape 0.0234 0.0311 0.0304 0.1692
mse 121589 219485 213677 4.68961e+06
total_time 0.5972 1.4545 0.0049 0.0044

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 465.497 346.972 398.956 1119.26
mape 0.062 0.0436 0.0512 0.1583
mse 835021 403760 656723 3.17316e+06
total_time 0.8713 0.5964 0.0053 0.0044

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 558.673 459.757 602.926 1340.95
mape 0.0697 0.0565 0.0787 0.17
mse 1.22723e+06 739114 1.61572e+06 6.04619e+06
total_time 0.7416 2.0815 0.0055 0.0045

Plot:

Comment thread nixtla/nixtla_client.py
model_input_size,
)
in_sample_payload = _forecast_payload_to_in_sample(payload, insample_h, n_windows)
logger.info("Calling Historical Forecast Endpoint...")
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fwiw keeping the same logging message here to not confuse users

("cross_validation", {"h": 7, "n_windows": 2}, False),
("forecast", {"h": 7, "add_history": True}, False),
("cross_validation", {"h": 7, "n_windows": 2}, True),
("forecast", {"h": 7, "add_history": True}, True),
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these two tests should be "True", otherwise they are identical to the ones above it.

@elephaint elephaint force-pushed the fix/deprecate_insample_endpoint branch from 0964a72 to c7b3af7 Compare December 3, 2025 18:59
@elephaint elephaint merged commit fac0fd5 into main Dec 4, 2025
28 of 29 checks passed
@elephaint elephaint deleted the fix/deprecate_insample_endpoint branch December 4, 2025 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants