TslibDataModule#

class pytorch_forecasting.data._tslib_data_module.TslibDataModule(time_series_dataset: TimeSeries, context_length: int, prediction_length: int, freq: str = 'h', add_relative_time_idx: bool = False, add_target_scales: bool = False, target_normalizer: TorchNormalizer | EncoderNormalizer | NaNLabelEncoder | str | list[TorchNormalizer | EncoderNormalizer | NaNLabelEncoder] | tuple[TorchNormalizer | EncoderNormalizer | NaNLabelEncoder] | None = 'auto', scalers: dict[str, StandardScaler | RobustScaler | TorchNormalizer | EncoderNormalizer] | None = None, shuffle: bool = True, window_stride: int = 1, batch_size: int = 32, num_workers: int = 0, train_val_test_split: tuple[float, float, float] = (0.7, 0.15, 0.15), collate_fn: Callable | None = None, **kwargs)[source]#

Bases: LightningDataModule

Experimental data module for integrating tslib time series into PyTorch Forecasting.

This module serves as the D2 layer for tslib models including transformer-based architectures like Informer, AutoFormer, TimeXer and other model deep learning model architectures.

Parameters:

  • time_series_dataset (TimeSeries) – The time series dataset to be used for training and validation. This is the newly implemented D1 layer.

  • context_length (int) – The length of the context window for the model. This is the number of time steps used as input to the model.

  • prediction_length (int) – The length of the prediction window for the model. This is the number of time steps to be predicted by the model.

  • freq (str, default = "h") – The frequency of the time series data. This is used to determine the time steps for the model.

  • features (str = "MS") –

    Feature combination mode:

    • ”S”: Single variable forecasting (target only)

    • ”M”: Multivariate forecasting, using all variables

    • ”MS”: Multivariate to single, using all variables to predict target

  • add_relative_time_idx (bool = False) – Whether to allow the relative time index to be used with the model.

  • add_target_scales (bool = False) – Whether to add target scaling info.

  • target_normalizer –

    Union[NORMALIZER, str, list[NORMALIZER], tuple[NORMALIZER], None],

    default=”auto”

    Normalizer for the target variable. If “auto”, uses RobustScaler.

  • scalers (Optional[dict[str, Union[StandardScaler, RobustScaler, TorchNormalizer]]], default=None #noqa: E501) – Dictionary of feature scalers.

  • shuffle (bool, default=True) – Whether to shuffle the data at every epoch.

  • window_stride (int, default=1) – The stride for the sliding window. This is used to create overlapping windows for the data.

  • batch_size (int, default=32) – Batch size for dataloader.

  • num_workers (int, default=0) – Number of workers for dataloader.

  • train_val_test_split (tuple, default=(0.7, 0.15, 0.15)) – Proportions for train, validation, and test dataset splits.

  • collate_fn (Optional[callable], default=None) – Custom collate function for the dataloader.

prepare_data_per_node#

If True, each LOCAL_RANK=0 will call prepare data. Otherwise only NODE_RANK=0, LOCAL_RANK=0 will prepare data.

allow_zero_length_dataloader_with_multiple_devices#

If True, dataloader with zero length within local rank is allowed. Default value is False.

Inherited-members:

Methods

collate_fn(batch)

Custom collate function for the dataloader.

from_datasets([train_dataset, val_dataset, ...])

Create an instance from torch.utils.data.Dataset.

load_from_checkpoint(checkpoint_path[, ...])

Primary way of loading a datamodule from a checkpoint.

load_state_dict(state_dict)

Called when loading a checkpoint, implement to reload datamodule state given datamodule state_dict.

on_after_batch_transfer(batch, dataloader_idx)

Override to alter or apply batch augmentations to your batch after it is transferred to the device.

on_before_batch_transfer(batch, dataloader_idx)

Override to alter or apply batch augmentations to your batch before it is transferred to the device.

on_exception(exception)

Called when the trainer execution is interrupted by an exception.

predict_dataloader()

Create the prediction dataloader.

prepare_data()

Use this to download and prepare data.

remove_ignored_hparams(ignore_list)

Remove ignored hyperparameters from the stored state.

save_hyperparameters(*args[, ignore, frame, ...])

Save arguments to hparams attribute.

setup([stage])

Setup the data module by preparing the datasets for training, testing and validation.

state_dict()

Called when saving a checkpoint, implement to generate and save datamodule state.

teardown(stage)

Called at the end of fit (train + validate), validate, test, or predict.

test_dataloader()

Create the test dataloader.

train_dataloader()

Create the train dataloader.

transfer_batch_to_device(batch, device, ...)

Override this hook if your DataLoader returns tensors wrapped in a custom data structure.

val_dataloader()

Create the validation dataloader.

Attributes

CHECKPOINT_HYPER_PARAMS_KEY

CHECKPOINT_HYPER_PARAMS_NAME

CHECKPOINT_HYPER_PARAMS_TYPE

hparams

The collection of hyperparameters saved with save_hyperparameters().

hparams_initial

The collection of hyperparameters saved with save_hyperparameters().

metadata

name

static collate_fn(batch)[source]#

Custom collate function for the dataloader.

Parameters:

batch (list[tuple[dict[str, Any]]]) – The batch of data to be collated.

Returns:

A tuple containing the collated data and the target variable. If the dataset has multiple targets, a list of tensors each of shape (batch_size, prediction_length,). Otherwise, a single tensor of shape (batch_size, prediction_length).

Return type:

tuple[dict[str, torch.Tensor], torch.Tensor or list of torch.Tensor]

predict_dataloader() → DataLoader[source]#

Create the prediction dataloader.

Returns:

The prediction dataloader.

Return type:

DataLoader

setup(stage: str | None = None) → None[source]#

Setup the data module by preparing the datasets for training, testing and validation.

Parameters:

stage (Optional[str]) – The stage of the data module. This can be “fit”, “test” or “predict”. If None, the data module will be setup for training.

test_dataloader() → DataLoader[source]#

Create the test dataloader.

Returns:

The test dataloader.

Return type:

DataLoader

train_dataloader() → DataLoader[source]#

Create the train dataloader.

Returns:

The train dataloader.

Return type:

DataLoader

val_dataloader() → DataLoader[source]#

Create the validation dataloader. :returns: The validation dataloader. :rtype: DataLoader

property metadata: dict[str, Any]#

” Compute the metadata via the _prepare_metadata method. This method is called when the metadata property is accessed for the first. :returns: Metadata for the data module. Refer to the _prepare_metadata method for

the keys and values in the metadata dictionary.

Return type:

dict