{ "cells": [ { "cell_type": "markdown", "id": "b5d44943", "metadata": {}, "source": [ "# TSLib for v2 - Example notebook for full pipeline" ] }, { "cell_type": "markdown", "id": "b7d27b55", "metadata": {}, "source": [ "## Basic imports for getting started\n", "\n", "This notebook is a basic vignette for the usage of the `tslib` data module on the `TimeXer` model for the v2 of PyTorch Forecasting. This is an experimental version and is an unstable version of the API.\n", "\n", "Feedback and suggestions on this pipeline - PR [#1836](https://github.com/sktime/pytorch-forecasting/pull/1836)" ] }, { "cell_type": "code", "execution_count": null, "id": "550a3fbf", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "from sklearn.preprocessing import StandardScaler\n", "import torch\n", "\n", "from pytorch_forecasting.data.data_module import TslibDataModule\n", "from pytorch_forecasting.data.encoders import (\n", " NaNLabelEncoder,\n", " TorchNormalizer,\n", ")\n", "from pytorch_forecasting.data.timeseries import TimeSeries\n", "from pytorch_forecasting.models.timexer._timexer_v2 import TimeXer" ] }, { "cell_type": "markdown", "id": "2625ed3d", "metadata": {}, "source": [ "## Construct a time series dataset\n", "\n", "This step requires us to build a `TimeSeries` object for creating a time series dataset, which identifies the features from a raw time series dataset. As you can see below, we are initialising a sample time series dataset." ] }, { "cell_type": "code", "execution_count": 2, "id": "a0058487", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
series_idtime_idxxycategoryfuture_known_featurestatic_featurestatic_feature_cat
0000.1776580.18112401.0000000.4095810
1010.1811240.31408100.9950040.4095810
2020.3140810.60193400.9800670.4095810
3030.6019340.73380500.9553360.4095810
4040.7338050.76884300.9210610.4095810
\n", "
" ], "text/plain": [ " series_id time_idx x y category future_known_feature \\\n", "0 0 0 0.177658 0.181124 0 1.000000 \n", "1 0 1 0.181124 0.314081 0 0.995004 \n", "2 0 2 0.314081 0.601934 0 0.980067 \n", "3 0 3 0.601934 0.733805 0 0.955336 \n", "4 0 4 0.733805 0.768843 0 0.921061 \n", "\n", " static_feature static_feature_cat \n", "0 0.409581 0 \n", "1 0.409581 0 \n", "2 0.409581 0 \n", "3 0.409581 0 \n", "4 0.409581 0 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "num_series = 100\n", "seq_length = 50\n", "data_list = []\n", "for i in range(num_series):\n", " x = np.arange(seq_length)\n", " y = np.sin(x / 5.0) + np.random.normal(scale=0.1, size=seq_length)\n", " category = i % 5\n", " static_value = np.random.rand()\n", " for t in range(seq_length - 1):\n", " data_list.append(\n", " {\n", " \"series_id\": i,\n", " \"time_idx\": t,\n", " \"x\": y[t],\n", " \"y\": y[t + 1],\n", " \"category\": category,\n", " \"future_known_feature\": np.cos(t / 10),\n", " \"static_feature\": static_value,\n", " \"static_feature_cat\": i % 3,\n", " }\n", " )\n", "data_df = pd.DataFrame(data_list)\n", "data_df.head()" ] }, { "cell_type": "markdown", "id": "c7c04ff5", "metadata": {}, "source": [ "## Feature Categories and Definitions\n", "\n", "### **`time_idx`**\n", "- **Definition**: The temporal index column that orders observations chronologically\n", "- **Example**: Sequential time steps (0, 1, 2, ...) or timestamps\n", "- **Usage**: Identifies the temporal ordering of data points within each time series\n", "\n", "### **`target`** \n", "- **Definition**: The variable you want to predict/forecast\n", "- **Example**: Sales volume, stock price, temperature readings\n", "- **Usage**: The dependent variable that the model learns to forecast\n", "\n", "### **`group`**\n", "- **Definition**: Categorical variables that identify different time series entities\n", "- **Example**: `series_id`, `store_id`, `product_id`, `customer_id`\n", "- **Usage**: Distinguishes between multiple time series in the dataset\n", "\n", "### **`num`**\n", "- **Definition**: Numerical/continuous features used as model inputs\n", "- **Example**: Price, quantity, weather data, economic indicators \n", "- **Usage**: Continuous variables that provide numerical context for predictions\n", "\n", "### **`cat`**\n", "- **Definition**: Categorical features that represent discrete classes or labels\n", "- **Example**: Product category, day of week, seasonal indicators, region\n", "- **Usage**: Discrete variables that provide categorical context for predictions\n", "\n", "### **`known`**\n", "- **Definition**: Future values that are known at prediction time (exogenous variables)\n", "- **Example**: Holidays, planned promotions, scheduled events, calendar features\n", "- **Usage**: Information available for both historical and future periods\n", "\n", "### **`unknown`**\n", "- **Definition**: Variables only available during training/historical periods\n", "- **Example**: Past weather conditions, historical prices, competitor actions\n", "- **Usage**: Features that help with training but aren't available for future predictions\n", "\n", "### **`static`**\n", "- **Definition**: Time-invariant features that remain constant for each time series\n", "- **Example**: Store size, product attributes, geographic location, customer demographics\n", "- **Usage**: Entity-specific characteristics that don't change over time" ] }, { "cell_type": "code", "execution_count": 3, "id": "89a5adbe", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/aryan/pytorch-forecasting/pytorch_forecasting/data/timeseries/_timeseries_v2.py:105: UserWarning: TimeSeries is part of an experimental rework of the pytorch-forecasting data layer, scheduled for release with v2.0.0. The API is not stable and may change without prior warning. For beta testing, but not for stable production use. Feedback and suggestions are very welcome in pytorch-forecasting issue 1736, https://github.com/sktime/pytorch-forecasting/issues/1736\n", " warn(\n" ] } ], "source": [ "dataset = TimeSeries(\n", " data=data_df,\n", " time=\"time_idx\",\n", " target=\"y\",\n", " group=[\"series_id\"],\n", " num=[\"x\", \"future_know_feature\", \"static_feature\"],\n", " cat=[\"category\", \"static_feature_cat\"],\n", " known=[\"future_known_feature\"],\n", " unknown=[\"x\", \"category\"],\n", " static=[\"static_feature\", \"static_feature_cat\"],\n", ")" ] }, { "cell_type": "markdown", "id": "f8753a6a", "metadata": {}, "source": [ "## Initialise the `TslibDataModule` using the dataset\n", "\n", "This steps initialises a basic data module built specially for `tslib` modules and provides all the metadata required to train and implement the `tslib` of your choice!\n", "You can refer the implementation for `TslibDataModule` for more information." ] }, { "cell_type": "code", "execution_count": 4, "id": "5eae9035", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/aryan/pytorch-forecasting/pytorch_forecasting/data/_tslib_data_module.py:275: UserWarning: TslibDataModule is experimental and subject to change. The API is not stable and may change without prior warning.\n", " warnings.warn(\n" ] } ], "source": [ "data_module = TslibDataModule(\n", " time_series_dataset=dataset,\n", " context_length=30,\n", " prediction_length=1,\n", " add_relative_time_idx=True,\n", " target_normalizer=TorchNormalizer(),\n", " categorical_encoders={\n", " \"category\": NaNLabelEncoder(add_nan=True),\n", " \"static_feature_cat\": NaNLabelEncoder(add_nan=True),\n", " },\n", " scalers={\n", " \"x\": StandardScaler(),\n", " \"future_known_feature\": StandardScaler(),\n", " \"static_feature\": StandardScaler(),\n", " },\n", " batch_size=32,\n", ")" ] }, { "cell_type": "code", "execution_count": 5, "id": "b1843233", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'feature_names': {'categorical': ['category', 'static_feature_cat'],\n", " 'continuous': ['x', 'future_known_feature', 'static_feature'],\n", " 'static': ['static_feature', 'static_feature_cat'],\n", " 'known': ['future_known_feature'],\n", " 'unknown': ['x', 'category', 'static_feature', 'static_feature_cat'],\n", " 'target': ['y'],\n", " 'all': ['x',\n", " 'category',\n", " 'future_known_feature',\n", " 'static_feature',\n", " 'static_feature_cat'],\n", " 'static_categorical': ['static_feature_cat'],\n", " 'static_continuous': ['static_feature']},\n", " 'feature_indices': {'categorical': [1, 4],\n", " 'continuous': [0, 2, 3],\n", " 'static': [],\n", " 'known': [2],\n", " 'unknown': [0, 1, 3, 4],\n", " 'target': [0]},\n", " 'n_features': {'categorical': 2,\n", " 'continuous': 3,\n", " 'static': 2,\n", " 'known': 1,\n", " 'unknown': 4,\n", " 'target': 1,\n", " 'all': 5,\n", " 'static_categorical': 1,\n", " 'static_continuous': 1},\n", " 'context_length': 30,\n", " 'prediction_length': 1,\n", " 'freq': 'h',\n", " 'features': 'MS'}" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_module.metadata" ] }, { "cell_type": "markdown", "id": "dd9451ee", "metadata": {}, "source": [ "## Initialise the model\n", "\n", "We shall try out two versions of this model, one using `MAE()` and one with `QuantileLoss()`.\n", "\n", "Let us quickly import the required packages for the next steps." ] }, { "cell_type": "code", "execution_count": null, "id": "f6b568a5", "metadata": {}, "outputs": [], "source": [ "from pytorch_forecasting.metrics import MAE, SMAPE, QuantileLoss" ] }, { "cell_type": "code", "execution_count": 7, "id": "429b5f15", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/aryan/pytorch-forecasting/pytorch_forecasting/models/base/_base_model_v2.py:61: UserWarning: The Model 'TimeXer' is part of an experimental reworkof the pytorch-forecasting model layer, scheduled for release with v2.0.0. The API is not stable and may change without prior warning. This class is intended for beta testing and as a basic skeleton, but not for stable production use. Feedback and suggestions are very welcome in pytorch-forecasting issue 1736, https://github.com/sktime/pytorch-forecasting/issues/1736\n", " warn(\n", "/home/aryan/pytorch-forecasting/pytorch_forecasting/models/base/_tslib_base_model_v2.py:60: UserWarning: The Model 'TimeXer' is part of an experimental implementationof the pytorch-forecasting model layer for Time Series Library, scheduledfor release with v2.0.0. The API is not stableand may change without prior warning. This class is intended for betatesting, not for stable production use.\n", " warn(\n", "/home/aryan/pytorch-forecasting/pytorch_forecasting/models/timexer/_timexer_v2.py:133: UserWarning: TimeXer is an experimental model implemented on TslibBaseModelV2. It is an unstable version and maybe subject to unannouced changes.Please use with caution. Feedback on the design and implementation iswelcome. On the issue #1833 - https://github.com/sktime/pytorch-forecasting/issues/1833\n", " warn.warn(\n", "/home/aryan/pytorch-forecasting/pytorch_forecasting/models/timexer/_timexer_v2.py:179: UserWarning: Context length (30) is not divisible by patch length. This may lead to unexpected behavior, as sometime steps will not be used in the model.\n", " warn.warn(\n" ] } ], "source": [ "model1 = TimeXer(\n", " loss=MAE(),\n", " hidden_size=64,\n", " nhead=4,\n", " e_layers=2,\n", " d_ff=256,\n", " dropout=0.1,\n", " patch_length=4,\n", " logging_metrics=[MAE(), SMAPE()],\n", " optimizer=\"adam\",\n", " optimizer_params={\"lr\": 1e-3},\n", " lr_scheduler=\"reduce_lr_on_plateau\",\n", " lr_scheduler_params={\n", " \"mode\": \"min\",\n", " \"factor\": 0.5,\n", " \"patience\": 5,\n", " },\n", " metadata=data_module.metadata,\n", ")" ] }, { "cell_type": "code", "execution_count": 8, "id": "0aa21f48", "metadata": {}, "outputs": [], "source": [ "model2 = TimeXer(\n", " loss=QuantileLoss(quantiles=[0.1, 0.5, 0.9]), # quantiles of 0.1, 0.5 and 0.9 used.\n", " hidden_size=64,\n", " nhead=4,\n", " e_layers=2,\n", " d_ff=256,\n", " dropout=0.1,\n", " patch_length=4,\n", " optimizer=\"adam\",\n", " optimizer_params={\"lr\": 1e-3},\n", " lr_scheduler=\"reduce_lr_on_plateau\",\n", " lr_scheduler_params={\n", " \"mode\": \"min\",\n", " \"factor\": 0.5,\n", " \"patience\": 5,\n", " },\n", " metadata=data_module.metadata,\n", ")" ] }, { "cell_type": "code", "execution_count": 9, "id": "02605f9b", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "HPU available: False, using: 0 HPUs\n", "GPU available: True (cuda), used: True\n", "TPU available: False, using: 0 TPU cores\n", "HPU available: False, using: 0 HPUs\n" ] } ], "source": [ "from lightning.pytorch import Trainer\n", "\n", "trainer1 = Trainer(\n", " max_epochs=5,\n", " accelerator=\"auto\",\n", " devices=1,\n", " enable_progress_bar=True,\n", " enable_model_summary=True,\n", ")\n", "\n", "trainer2 = Trainer(\n", " max_epochs=4,\n", " accelerator=\"auto\",\n", " devices=1,\n", " enable_progress_bar=True,\n", " enable_model_summary=True,\n", ")" ] }, { "cell_type": "markdown", "id": "e22756b2", "metadata": {}, "source": [ "## Fit the trainer on the model and feed data using the data module" ] }, { "cell_type": "code", "execution_count": 10, "id": "6e9117d2", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "You are using a CUDA device ('NVIDIA GeForce RTX 4050 Laptop GPU') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n", "\n", " | Name | Type | Params | Mode \n", "----------------------------------------------------------------\n", "0 | loss | MAE | 0 | train\n", "1 | en_embedding | EnEmbedding | 320 | train\n", "2 | ex_embedding | DataEmbedding_inverted | 2.0 K | train\n", "3 | encoder | Encoder | 133 K | train\n", "4 | head | FlattenHead | 513 | train\n", "----------------------------------------------------------------\n", "136 K Trainable params\n", "0 Non-trainable params\n", "136 K Total params\n", "0.546 Total estimated model params size (MB)\n", "57 Modules in train mode\n", "0 Modules in eval mode\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "f26d868819404cb0a48cc030aefef48c", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Sanity Checking: | | 0/? [00:00