DeepAR¶

class pytorch_forecasting.models.deepar.DeepAR(cell_type: str = 'LSTM', hidden_size: int = 10, rnn_layers: int = 2, dropout: float = 0.1, static_categoricals: List[str] = [], static_reals: List[str] = [], time_varying_categoricals_encoder: List[str] = [], time_varying_categoricals_decoder: List[str] = [], categorical_groups: Dict[str, List[str]] = {}, time_varying_reals_encoder: List[str] = [], time_varying_reals_decoder: List[str] = [], embedding_sizes: Dict[str, Tuple[int, int]] = {}, embedding_paddings: List[str] = [], embedding_labels: Dict[str, numpy.ndarray] = {}, x_reals: List[str] = [], x_categoricals: List[str] = [], n_validation_samples: Optional[int] = None, n_plotting_samples: Optional[int] = None, target: Optional[Union[str, List[str]]] = None, target_lags: Dict[str, List[int]] = {}, loss: Optional[pytorch_forecasting.metrics.DistributionLoss] = None, logging_metrics: Optional[torch.nn.modules.container.ModuleList] = None, **kwargs)[source]¶

Bases: pytorch_forecasting.models.base_model.AutoRegressiveBaseModelWithCovariates

DeepAR Network.

The code is based on the article DeepAR: Probabilistic forecasting with autoregressive recurrent networks.

Parameters

cell_type (str, optional) – Recurrent cell type [“LSTM”, “GRU”]. Defaults to “LSTM”.
hidden_size (int, optional) – hidden recurrent size - the most important hyperparameter along with rnn_layers. Defaults to 10.
rnn_layers (int, optional) – Number of RNN layers - important hyperparameter. Defaults to 2.
dropout (float, optional) – Dropout in RNN layers. Defaults to 0.1.
static_categoricals – integer of positions of static categorical variables
static_reals – integer of positions of static continuous variables
time_varying_categoricals_encoder – integer of positions of categorical variables for encoder
time_varying_categoricals_decoder – integer of positions of categorical variables for decoder
time_varying_reals_encoder – integer of positions of continuous variables for encoder
time_varying_reals_decoder – integer of positions of continuous variables for decoder
categorical_groups – dictionary where values are list of categorical variables that are forming together a new categorical variable which is the key in the dictionary
x_reals – order of continuous variables in tensor passed to forward function
x_categoricals – order of categorical variables in tensor passed to forward function
embedding_sizes – dictionary mapping (string) indices to tuple of number of categorical classes and embedding size
embedding_paddings – list of indices for embeddings which transform the zero’s embedding to a zero vector
embedding_labels – dictionary mapping (string) indices to list of categorical labels
n_validation_samples (int, optional) – Number of samples to use for calculating validation metrics. Defaults to None, i.e. no sampling at validation stage and using “mean” of distribution for logging metrics calculation.
n_plotting_samples (int, optional) – Number of samples to generate for plotting predictions during training. Defaults to n_validation_samples if not None or 100 otherwise.
target (str, optional) – Target variable or list of target variables. Defaults to None.
target_lags (Dict[str, Dict[str, int]]) – dictionary of target names mapped to list of time steps by which the variable should be lagged. Lags can be useful to indicate seasonality to the models. If you know the seasonalit(ies) of your data, add at least the target variables with the corresponding lags to improve performance. Defaults to no lags, i.e. an empty dictionary.
loss (DistributionLoss, optional) – Distribution loss function. Keep in mind that each distribution loss function might have specific requirements for target normalization. Defaults to NormalDistributionLoss.
logging_metrics (nn.ModuleList, optional) – Metrics to log during training. Defaults to nn.ModuleList([SMAPE(), MAE(), RMSE(), MAPE(), MASE()]).

Methods

`construct_input_vector`(x_cat, x_cont[, …])	Create input vector into RNN network
`decode`(input_vector, target_scale, …[, …])	Decode hidden state of RNN into prediction.
`decode_all`(x, hidden_state[, lengths])
`encode`(x)	Encode sequence into hidden state
`forward`(x[, n_samples])	Forward network
`from_dataset`(dataset[, …])	Create model from dataset.
`log_metrics`(x, y, out)	Log metrics every training/validation step.
`log_prediction`(x, out, batch_idx)	Log metrics every training/validation step.
`plot_prediction`(x, out, idx[, …])	Plot prediction of prediction vs actuals
`predict`(data[, mode, return_index, …])	predict dataloader
`validation_step`(batch, batch_idx)	Operates on a single batch of data from the validation set.

construct_input_vector(x_cat: torch.Tensor, x_cont: torch.Tensor, one_off_target: Optional[torch.Tensor] = None) → torch.Tensor[source]¶

Create input vector into RNN network

Parameters: one_off_target – tensor to insert into first position of target. If None (default), remove first time step.

decode(input_vector: torch.Tensor, target_scale: torch.Tensor, decoder_lengths: torch.Tensor, hidden_state: Union[Tuple[torch.Tensor, torch.Tensor], torch.Tensor], n_samples: Optional[int] = None) → Tuple[torch.Tensor, bool][source]¶: Decode hidden state of RNN into prediction. If n_smaples is given, decode not by using actual values but rather by sampling new targets from past predictions iteratively

encode(x: Dict[str, torch.Tensor]) → Union[Tuple[torch.Tensor, torch.Tensor], torch.Tensor][source]¶: Encode sequence into hidden state

forward(x: Dict[str, torch.Tensor], n_samples: Optional[int] = None) → Dict[str, torch.Tensor][source]¶: Forward network

classmethod from_dataset(dataset: pytorch_forecasting.data.timeseries.TimeSeriesDataSet, allowed_encoder_known_variable_names: Optional[List[str]] = None, **kwargs)[source]¶

Create model from dataset.

Parameters

dataset – timeseries dataset
allowed_encoder_known_variable_names – List of known variables that are allowed in encoder, defaults to all
**kwargs – additional arguments such as hyperparameters for model (see __init__())

Returns

DeepAR network

log_metrics(x: Dict[str, torch.Tensor], y: torch.Tensor, out: Dict[str, torch.Tensor]) → None[source]¶

Log metrics every training/validation step.

Parameters

x (Dict[str, torch.Tensor]) – x as passed to the network by the dataloader
y (torch.Tensor) – y as passed to the loss function by the dataloader
out (Dict[str, torch.Tensor]) – output of the network

log_prediction(x, out, batch_idx) → None[source]¶

Log metrics every training/validation step.

Parameters

x (Dict[str, torch.Tensor]) – x as passed to the network by the dataloader
out (Dict[str, torch.Tensor]) – output of the network
batch_idx (int) – current batch index

plot_prediction(x: Dict[str, torch.Tensor], out: Dict[str, torch.Tensor], idx: int, add_loss_to_title: Union[pytorch_forecasting.metrics.Metric, torch.Tensor, bool] = False, show_future_observed: bool = True, ax=None) → matplotlib.figure.Figure[source]¶

Plot prediction of prediction vs actuals

Parameters

x – network input
out – network output
idx – index of prediction to plot
add_loss_to_title – if to add loss to title or loss function to calculate. Can be either metrics, bool indicating if to use loss metric or tensor which contains losses for all samples. Calcualted losses are determined without weights. Default to False.
show_future_observed – if to show actuals for future. Defaults to True.
ax – matplotlib axes to plot on

Returns

matplotlib figure

predict(data: Union[torch.utils.data.dataloader.DataLoader, pandas.core.frame.DataFrame, pytorch_forecasting.data.timeseries.TimeSeriesDataSet], mode: Union[str, Tuple[str, str]] = 'prediction', return_index: bool = False, return_decoder_lengths: bool = False, batch_size: int = 64, num_workers: int = 0, fast_dev_run: bool = False, show_progress_bar: bool = False, return_x: bool = False, n_samples: int = 100)[source]¶

predict dataloader

Parameters

dataloader – dataloader, dataframe or dataset
mode – one of “prediction”, “quantiles” or “raw”, or tuple ("raw", output_name) where output_name is a name in the dictionary returned by forward()
return_index – if to return the prediction index
return_decoder_lengths – if to return decoder_lengths
batch_size – batch size for dataloader - only used if data is not a dataloader is passed
num_workers – number of workers for dataloader - only used if data is not a dataloader is passed
fast_dev_run – if to only return results of first batch
show_progress_bar – if to show progress bar. Defaults to False.
return_x – if to return network inputs
n_samples – number of samples to draw. Defaults to 100.

Returns

some elements might not be present depending on what is configured: to be returned

Return type

output, x, index, decoder_lengths

validation_step(batch, batch_idx)[source]¶

Operates on a single batch of data from the validation set. In this step you’d might generate examples or calculate anything of interest like accuracy.

# the pseudocode for these calls
val_outs = []
for val_batch in val_data:
    out = validation_step(val_batch)
    val_outs.append(out)
validation_epoch_end(val_outs)

Parameters

batch (Tensor | (Tensor, …) | [Tensor, …]) – The output of your DataLoader. A tensor, tuple or list.
batch_idx (int) – The index of this batch
dataloader_idx (int) – The index of the dataloader that produced this batch (only if multiple val dataloaders used)

Returns

Any of.

Any object or value

None - Validation will skip to the next batch

# pseudocode of order
out = validation_step()
if defined('validation_step_end'):
    out = validation_step_end(out)
out = validation_epoch_end(out)

# if you have one val dataloader:
def validation_step(self, batch, batch_idx)

# if you have multiple val dataloaders:
def validation_step(self, batch, batch_idx, dataloader_idx)

Examples:

# CASE 1: A single validation dataset
def validation_step(self, batch, batch_idx):
    x, y = batch

    # implement your own
    out = self(x)
    loss = self.loss(out, y)

    # log 6 example images
    # or generated text... or whatever
    sample_imgs = x[:6]
    grid = torchvision.utils.make_grid(sample_imgs)
    self.logger.experiment.add_image('example_images', grid, 0)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    val_acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs!
    self.log_dict({'val_loss': loss, 'val_acc': val_acc})

If you pass in multiple val dataloaders, validation_step() will have an additional argument.

# CASE 2: multiple validation dataloaders
def validation_step(self, batch, batch_idx, dataloader_idx):
    # dataloader_idx tells you which dataset this is.

Note

If you don’t need to validate you don’t need to implement this method.

Note

When the validation_step() is called, the model has been put in eval mode and PyTorch gradients have been disabled. At the end of validation, the model goes back to training mode and gradients are enabled.

deepar mlp