How to Master Hierarchical Time Series Forecasting: A Comprehensive Guide

 Time series forecasting is a technique which makes prediction for future values by utilizing historical data, but what if we need forecast from different parts of business, in such cases we can not only rely on simple time series models we need something unique.

Hierarchical Time Series (HTS) forecasting is an essential technique for organizations that can manage data across multiple levels, such as geographical regions, product categories, or organizational structures. HTS is highly used in retail sector where we need prediction for different types of categories and various regions.

The ultimate goal of HTS forecasting is to generate consistent and reliable predictions that make sense across all hierarchical levels. By ensuring these forecasts are aligned, decision-makers can better strategize and it helps in inventory management and resource allocation.

If you want to know more what is HTS and how to implement it, keep reading the article!

Hierarchical Time Series Model

What is Hierarchical Time Series Forecasting?

Hierarchical Time Series forecasting deals with data that is organized in a hierarchy. The hierarchy can be geographical (e.g., country → state → city), organizational (e.g., company → department → team), or any structure where data can be aggregated or disaggregated.

For example:

• Top Level: Total sales for a company.
• Middle Level: Sales by regions (e.g., North, South, East).
• Bottom Level: Sales by individual stores.

HTS forecasting ensures that the sum of forecasts at the bottom level matches the forecast at the top level. This process is called reconciliation, which maintains consistency across the hierarchy.

Why Use Hierarchical Time Series Forecasting?

Organizations benefit from HTS forecasting because it provides insights at various levels helping businesses in identifying trends on both macro and micro level. In lower levels of a hierarchy (e.g., individual stores), data can be sparse, leading to less accurate forecasts. By borrowing strength from higher-level data, HTS mitigates sparsity issues and improves forecast accuracy at granular levels.

Techniques like bottom-up, top-down, middle-out, and optimal reconciliation ensure consistent and reliable forecasts, making HTS invaluable for complex, multi-level datasets.

HTS can also facilitates "what-if" analysis across different hierarchy levels, enabling businesses to test the impact of decisions or external factors at varying scales.

For example:

• A retail chain can forecast overall sales to plan future inventory needs.
• At the same time, it can forecast store-level sales for precise stock allocation.

The consistency offered by HTS forecasting reduces the risk of misalignment and ensures better resource planning and decision-making.

Steps to Implement Hierarchical Time Series Forecasting

Step 1: Define the Hierarchy

Before forecasting, understand and define the structure of your hierarchy. For instance:

• Level 1 (Top Level): Total sales.
• Level 2: Regional sales.
• Level 3 (Bottom Level): Sales by individual stores.

This definition serves as the foundation for organizing your data and reconciling forecasts.

Step 2: Collect and Prepare Data

Historical data is crucial for HTS forecasting. Ensure your dataset includes:

• Time Period: Dates or time intervals for each observation.
• Features: Data points such as store location, region, or category.
• Target Variable: The value you want to predict, such as sales.

Here’s how to load and prepare a sample dataset in Python:

python
import pandas as pd

# Your hierarchical data
data = {
    "date": ["2024-11-01", "2023-12-01", "2022-10-01", "2021-01-01"],
    "region": ["North", "North", "South", "South"],
    "store": ["A", "B", "C", "D"],
    "sales": [100, 120, 80, 90],
}
df = pd.DataFrame(data)

# Ensure the date column is in datetime format
df["date"] = pd.to_datetime(df["date"])

# Aggregate data for each level
total_sales = df.groupby("date")["sales"].sum().reset_index()
regional_sales = df.groupby(["date", "region"])["sales"].sum().reset_index()

print(total_sales)
print(regional_sales)

Step 3: Choose a Forecasting Method

There are three primary methods for HTS forecasting:

1. Bottom-Up: Aggregate forecasts from the lowest level to predict higher levels.
2. Top-Down: Start with the top-level forecast and split it among lower levels proportionally.
3. Middle-Out: Forecast from a middle level and adjust predictions upward and downward.

Each method has its use cases:

• Bottom-Up works well for detailed data.
• Top-Down is suitable for data with strong aggregate trends.
• Middle-Out balances complexity and accuracy.

Step 4: Train Forecasting Models

Train forecasting models for each level of the hierarchy. Here’s an example using ARIMA(Autoregressive Integrated Moving Average) for time series forecasting:

python
from statsmodels.tsa.arima.model import ARIMA

# Train ARIMA model for total sales
model = ARIMA(total_sales["sales"], order=(1, 1, 1))
model_fit = model.fit()

# Forecast future sales
forecast = model_fit.forecast(steps=5)
print("Forecasted Total Sales:", forecast)

For more complex hierarchies, you can use machine learning models like Random Forest or Gradient Boosting for detailed forecasting at lower levels.

Step 5: Reconcile Forecasts

Reconciliation ensures that forecasts across levels align. You can use:

• Bottom-Up Reconciliation: Aggregate forecasts from the bottom to the top.
• Top-Down Reconciliation: Distribute the top-level forecast proportionally to lower levels.
• Optimal Reconciliation: Minimize errors using advanced statistical techniques.

Here’s an example of bottom-up reconciliation:

python code
# Bottom-Up Reconciliation
store_forecasts = {"A": 110, "B": 130, "C": 85, "D": 95}
regional_forecasts = {"North": sum(store_forecasts[s] for s in ["A", "B"]),
                      "South": sum(store_forecasts[s] for s in ["C", "D"])}

total_forecast = sum(store_forecasts.values())

print("Regional Forecasts:", regional_forecasts)
print("Total Forecast:", total_forecast)

Step 6: Validate Forecasts

Evaluate the accuracy of your forecasts using metrics like Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE).

python code

from sklearn.metrics import mean_absolute_error, mean_squared_error

# Actual vs Predicted
actual = [390, 400, 420]
predicted = [380, 410, 415]

mae = mean_absolute_error(actual, predicted)
rmse = mean_squared_error(actual, predicted, squared=False)

print("MAE:", mae)
print("RMSE:", rmse)

Tools and Libraries for HTS Forecasting

Several tools and libraries can simplify HTS forecasting:

1. Python Libraries:

   • Statsmodels: For ARIMA and exponential smoothing.
   • Prophet: For trend and seasonality-based forecasting.
   • HTS-Prophet: A library specifically designed for hierarchical forecasting.

2. R Packages:

   • hts: Designed for hierarchical time series forecasting.
   • forecast: A general time series forecasting package.

Challenges in HTS Forecasting

1. Data Quality: Missing or inconsistent data can significantly affect accuracy.
2. Complex Hierarchies: Large hierarchies require more computational power.
3. Reconciliation: Balancing forecasts across levels is challenging but critical.

Conclusion

Hierarchical Time Series forecasting is a powerful tool for organizations dealing with structured data. By following the steps outlined—defining the hierarchy, preparing data, selecting forecasting methods, training models, reconciling predictions, and validating results—you can implement a reliable HTS forecasting system. With Python and modern libraries, even complex hierarchical structures can be managed effectively, enabling better decision-making and resource allocation.