[BUG] Lags can only be applied to target variables

[manmeet3591]

My data is as follows:

I want to train a model which takes previous 10 time steps of all 4 variables rainfall_, dmi_, nino12_ and nino34_ and gives the 11th step of rainfall_ as target. My training dataset is as follows:

Create time_idx and group_id

df = train_df.copy()
df['time_idx'] = df.index
df['group_id'] = 0  # Only one time series

# Define the TimeSeriesDataSet
max_encoder_length = 10  # Number of time steps for the encoder
max_prediction_length = 1  # Number of time steps for the prediction

training_dataset = TimeSeriesDataSet(
    data=df,
    time_idx="time_idx",
    target="rainfall_",
    group_ids=["group_id"],
    max_encoder_length=max_encoder_length,
    min_encoder_length=max_encoder_length,
    max_prediction_length=max_prediction_length,
    # time_varying_known_reals=["dmi_", "nino34_", "nino12_"],
    time_varying_unknown_reals=["dmi_", "nino34_", "nino12_", "rainfall_"],
    lags={"rainfall_": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 
          "nino12_":  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]},  # Include previous 10 time steps of rainfall_
    # lags={
    #     "rainfall_": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    #     "dmi_": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    #     "nino34_": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    #     "nino12_": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    # }
)

# Convert dataset to a dataloader for model training
train_dataloader = training_dataset.to_dataloader(train=True, batch_size=64)

This is throwing up the following error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-22-0a8da5b66b64>](https://localhost:8080/#) in <cell line: 10>()
      8 max_prediction_length = 1  # Number of time steps for the prediction
      9 
---> 10 training_dataset = TimeSeriesDataSet(
     11     data=df,
     12     time_idx="time_idx",

6 frames
[/usr/local/lib/python3.10/dist-packages/sklearn/base.py](https://localhost:8080/#) in _check_feature_names(self, X, reset)
    479                 )
    480 
--> 481             raise ValueError(message)
    482 
    483     def _validate_data(

ValueError: The feature names should match those that were passed during fit.
Feature names unseen at fit time:
- nino12__lagged_by_1
Feature names seen at fit time, yet now missing:
- nino12_

[benHeid]

With

train_df = pd.DataFrame(
    {
        "rainfall": [0.1] * 100,
        "nino34": [0.1] * 100,
        "nino12": [0.5] * 100,
        "dmi": [0.1] * 100,
        "rainfall_": [0.1] * 100,
        "nino34_": [0.1] * 100,
        "nino12_": [0.1] * 100,
        "dmi_": [0.1] * 100,
    }
)

I can confirm this error.
The lags seem only to work for the target currently.

@manmeet3591, according to your use-case above, I assume that you do not need the lags, since the model is getting the past ten values of rainfall and your other unknown variables as input. Lags mean that for each input step the past ten values would be considered.