A Series on Time Series, Part II: Solving Data Problems and Understanding Classical Methods

Forecasting home prices is an intricate process that involves a blend of economic theory, data analysis, and statistical techniques. In Part I, we explored the reasons for building forecasts despite inherent challenges. Now, in Part II, we will delve into techniques for dealing with common data problems and introduce classical time series methods that are still commonly used by forecasters. These methods, including ARIMA and Holt-Winters, have stood the test of time; students are well advised to learn them.

Common Data Problems in Time Series Forecasting

Despite the robustness of classical time series methods, forecasters often encounter several data-related challenges that can complicate the modeling process. Understanding these issues is crucial for developing accurate and reliable forecasts.

Missing Data

Missing data is a common problem in time series analysis. Gaps in the data can occur for various reasons, such as data collection errors, reporting lags, or system failures. Missing values can disrupt the continuity of the time series and impact the performance of models like ARIMA and Holt-Winters.

Solutions

• Imputation: Replace missing values with estimated ones based on available data, such as using the mean, median, or interpolation methods.
• Model-Based Methods: Use models to predict missing values based on the observed data.

Outliers

Outliers are extreme values that deviate significantly from the rest of the data. They can distort the results of time series models and lead to inaccurate forecasts. Outliers may result from unusual events, data entry errors, or changes in market conditions.

Solutions

• Outlier Detection: Identify and analyze outliers to determine if they should be included or removed from the dataset.
• Robust Modeling: Use models that are less sensitive to outliers or apply transformations to mitigate their impact.

Non-Stationarity

Non-stationarity occurs when the statistical properties of a time series, such as the mean and variance, change over time. Many time series models, including ARIMA, require the data to be stationary for accurate forecasting.

Solutions

• Differencing: Apply differencing to the time series to remove trends and make it stationary.
• Transformation: Use logarithmic or power transformations to stabilize the variance.

Multicollinearity

Multicollinearity arises when two or more predictors in a model are highly correlated. This can cause instability in the model estimates and make it difficult to determine the individual effect of each predictor.

Solutions

• Feature Selection: Remove or combine correlated predictors to reduce multicollinearity.
• Regularization: Apply regularization techniques, such as Ridge or Lasso regression, to penalize large coefficients and mitigate multicollinearity. Sadly, we don’t have space to cover regularization here in-depth, but there are plenty of online resources describing them. 

Heteroskedasticity

Heteroskedasticity refers to the presence of non-constant variance in the error terms of a model. This can lead to inefficient estimates and unreliable confidence intervals.

Solutions

• Weighted Least Squares: Apply weighted least squares to give different weights to observations based on their variance.
• Transformations: Use transformations, such as logarithmic or square root, to stabilize the variance.

Data Granularity

The granularity of the data, or the level of detail captured, can affect the performance of time series models. High-frequency data may contain more noise, while low-frequency data may miss important patterns.

Solutions

• Aggregation: Aggregate high-frequency data to a lower frequency to reduce noise and reveal underlying patterns.
• Disaggregation: In some cases, disaggregating low-frequency data into higher-frequency components can provide more detailed insights.

Classical Methods: An Overview

Classical time series forecasting methods rely on historical data to identify patterns and project future values. These methods are grounded in statistical theory and are particularly effective when patterns in the data are stable and consistent over time. Below, we explore some of the most widely used classical methods.

ARIMA (AutoRegressive Integrated Moving Average)

ARIMA is a powerful and versatile forecasting technique that combines three components:

1. AutoRegression (AR): This component models the relationship between an observation and a number of lagged observations. For instance, in the context of home prices, the AR component might analyze how current prices relate to prices from previous months or years.
2. Integrated (I): The integration part of ARIMA involves differencing the data to make it stationary, meaning that its statistical properties do not change over time. Stationarity is crucial for the reliability of the model, and differencing helps in removing trends and seasonal effects.
3. Moving Average (MA): This component models the relationship between an observation and a residual error from a moving average model applied to lagged observations.

The general form of an ARIMA model is denoted as ARIMA(p,d,q), where:

• p is the number of lag observations in the model (AR component).
• d is the number of times that the raw observations are differenced (I component).
• q is the size of the moving average window (MA component).

ARIMA models are particularly useful for short-term forecasting and can handle various types of data patterns, including trends and cycles.

Holt-Winters Exponential Smoothing

The Holt-Winters method is another classical time series technique, particularly effective for data with seasonality. It extends simple exponential smoothing to capture trends and seasonal patterns. The method consists of three equations:

1. Level Equation: Estimates the average value in the series.
2. Trend Equation: Estimates the trend in the data.
3. Seasonal Equation: Estimates the seasonal component.

The Holt-Winters method comes in two variations: additive and multiplicative. The additive version is suitable for series where seasonal variations are roughly constant over time, while the multiplicative version is better for series where seasonal variations change proportionally with the level of the series.

Decomposition Methods

Decomposition methods separate a time series into its constituent components: trend, seasonality, and residuals. This approach allows forecasters to analyze each component individually and understand the underlying patterns. The two main types of decomposition are:

1. Additive Decomposition: Assumes that the components add together to form the observed data.
2. Multiplicative Decomposition: Assumes that the components multiply together to form the observed data.

Decomposition is particularly useful for visualizing and understanding the components that drive the time series, making it easier to develop accurate forecasts.

Simple Moving Average (SMA) and Weighted Moving Average (WMA)

Moving average methods smooth out short-term fluctuations and highlight longer-term trends or cycles. The two main types are:

1. Simple Moving Average (SMA): Calculates the average of a fixed number of past observations. It is simple and effective for series with no clear trend or seasonality.
2. Weighted Moving Average (WMA): Similar to SMA, but assigns different weights to past observations, giving more importance to recent data. This method is more responsive to changes in the data.

Applying Classical Methods to Home Price Forecasting

When applied to home price forecasting, these classical methods can provide valuable insights, especially in stable market conditions. For instance, ARIMA models can help capture the autoregressive nature of home prices, where past prices influence future prices. Holt-Winters can effectively model seasonal variations, such as increased home buying in the spring and summer months.

However, it is essential to recognize the limitations of these methods. They may struggle in highly volatile or rapidly changing markets, such as those influenced by sudden policy changes or economic shocks. In such cases, more advanced techniques, including machine learning methods, may offer better performance.

Conclusion

Classical time series methods like ARIMA and Holt-Winters are fundamental tools in the forecaster’s toolkit. They offer robust frameworks for understanding and predicting time series data, providing valuable insights into patterns and trends. However, these methods are not without their challenges, particularly when dealing with common data problems such as missing data, outliers, non-stationarity, multicollinearity, heteroskedasticity, and data granularity.

By addressing these data challenges, forecasters can enhance the accuracy and reliability of their models. In the next part of this series, we will explore modern machine learning-based methods, which have gained popularity for their ability to handle complex and non-linear relationships in data. These advanced techniques promise to further improve home price forecasts, addressing some of the limitations of traditional methods. Stay tuned for Part III: Modern Machine Learning Methods and Vector AutoRegression.

Franklin Carroll

Franklin Carroll is Kukun's VP of Modelling and Analytics; After finishing his studies at the University of Chicago, he worked at Fannie Mae where he helped build Collateral Underwriter, roll out the Value Verify initiative, and specialized in collateral related and credit related policy.

Published July 30, 2024