Getting started#
Installation#
If you are working windows, you need to first install PyTorch with
pip install torch -f https://download.pytorch.org/whl/torch_stable.html.
Otherwise, you can proceed with
pip install pytorch-forecasting
Alternatively, to installl the package via conda:
conda install pytorch-forecasting pytorch>=1.7 -c pytorch -c conda-forge
PyTorch Forecasting is now installed from the conda-forge channel while PyTorch is install from the pytorch channel.
To use the MQF2 loss (multivariate quantile loss), also install pip install pytorch-forecasting[mqf2]
Usage#
The library builds strongly upon PyTorch Lightning which allows to train models with ease, spot bugs quickly and train on multiple GPUs out-of-the-box.
Further, we rely on Tensorboard for logging training progress.
The general setup for training and testing a model is
Create training dataset using
TimeSeriesDataSet.Using the training dataset, create a validation dataset with
from_dataset(). Similarly, a test dataset or later a dataset for inference can be created. You can store the dataset parameters directly if you do not wish to load the entire training dataset at inference time.Instantiate a model using the its
.from_dataset()method.Create a
lightning.Trainer()object.Find the optimal learning rate with its
.tuner.lr_find()method.Train the model with early stopping on the training dataset and use the tensorboard logs to understand if it has converged with acceptable accuracy.
Tune the hyperparameters of the model with your favourite package.
Train the model with the same learning rate schedule on the entire dataset.
Load the model from the model checkpoint and apply it to new data.
The Tutorials section provides detailled guidance and examples on how to use models and implement new ones.
Example#
import lightning.pytorch as pl
from lightning.pytorch.callbacks import EarlyStopping, LearningRateMonitor
from lightning.pytorch.tuner import Tuner
from pytorch_forecasting import TimeSeriesDataSet, TemporalFusionTransformer
# load data
data = ...
# define dataset
max_encoder_length = 36
max_prediction_length = 6
training_cutoff = "YYYY-MM-DD" # day for cutoff
training = TimeSeriesDataSet(
data[lambda x: x.date < training_cutoff],
time_idx= ...,
target= ...,
# weight="weight",
group_ids=[ ... ],
max_encoder_length=max_encoder_length,
max_prediction_length=max_prediction_length,
static_categoricals=[ ... ],
static_reals=[ ... ],
time_varying_known_categoricals=[ ... ],
time_varying_known_reals=[ ... ],
time_varying_unknown_categoricals=[ ... ],
time_varying_unknown_reals=[ ... ],
)
# create validation and training dataset
validation = TimeSeriesDataSet.from_dataset(training, data, min_prediction_idx=training.index.time.max() + 1, stop_randomization=True)
batch_size = 128
train_dataloader = training.to_dataloader(train=True, batch_size=batch_size, num_workers=2)
val_dataloader = validation.to_dataloader(train=False, batch_size=batch_size, num_workers=2)
# define trainer with early stopping
early_stop_callback = EarlyStopping(monitor="val_loss", min_delta=1e-4, patience=1, verbose=False, mode="min")
lr_logger = LearningRateMonitor()
trainer = pl.Trainer(
max_epochs=100,
accelerator="auto",
gradient_clip_val=0.1,
limit_train_batches=30,
callbacks=[lr_logger, early_stop_callback],
)
# create the model
tft = TemporalFusionTransformer.from_dataset(
training,
learning_rate=0.03,
hidden_size=32,
attention_head_size=1,
dropout=0.1,
hidden_continuous_size=16,
output_size=7,
loss=QuantileLoss(),
log_interval=2,
reduce_on_plateau_patience=4
)
print(f"Number of parameters in network: {tft.size()/1e3:.1f}k")
# find optimal learning rate (set limit_train_batches to 1.0 and log_interval = -1)
res = Tuner(trainer).lr_find(
tft, train_dataloaders=train_dataloader, val_dataloaders=val_dataloader, early_stop_threshold=1000.0, max_lr=0.3,
)
print(f"suggested learning rate: {res.suggestion()}")
fig = res.plot(show=True, suggest=True)
fig.show()
# fit the model
trainer.fit(
tft, train_dataloaders=train_dataloader, val_dataloaders=val_dataloader,
)
Main API#
Special Normalizer that is fit on each encoding sequence. |
|
Normalizer that scales by groups. |
|
Normalizer for multiple targets. |
|
Labelencoder that can optionally always encode nan and unknown classes (in transform) as class |
|
PyTorch Dataset for fitting timeseries models. |
|
|
DistributionLoss base class. |
|
Abstract class for defining metric for a multihorizon forecast |
Metric that can be used with muliple metrics. |
|
|
Beta distribution loss for unit interval data. |
|
Implicit Quantile Network Distribution Loss. |
|
Log-normal loss. |
|
Multivariate quantile loss based on the article Multivariate Quantile Function Forecaster. |
|
Multivariate low-rank normal distribution loss. |
|
Negative binomial loss, e.g. |
|
Normal distribution loss. |
Cross entropy loss for classification. |
|
Mean average absolute error. |
|
Mean absolute percentage. |
|
Mean absolute scaled error |
|
Poisson loss for count data. |
|
Root mean square error |
|
Symmetric mean absolute percentage. |
|
Quantile loss, i.e. a quantile of |
|
|
Model with additional methods for autoregressive models. |
|
Model with additional methods for autoregressive models with covariates. |
BaseModel from which new timeseries models should inherit from. |
|
|
Model with additional methods using covariates. |
Baseline model that uses last known target value to make prediction. |
|
DeepAR Network. |
|
MLP on the decoder. |
|
Initialize NBeats Model - use its |
|
Initialize N-HiTS Model - use its |
|
|
Embedding layer for categorical variables including groups of categorical variables. |
|
GRU that can handle zero-length sequences |
LSTM that can handle zero-length sequences |
|
Get LSTM or GRU. |
|
Recurrent Network. |
|
|
Temporal Fusion Transformer for forecasting timeseries - use its |
|
Apply function to a list of objects or directly if passed value is not a list. |
Computes the autocorrelation of samples at dimension |
|
|
Create boolean masks of shape len(lenghts) x size. |
Detach object |
|
Determine empirically good embedding sizes (formula taken from fastai). |
|
Groupby apply for torch tensors |
|
Create histogram of integers in predefined range |
|
Move object to device. |
|
|
Profile a given function with |
Convert value or list to list of values. |
|
Unpack RNN sequence. |