Choosing a Home Price Index for Our Kukun Investment Outlook

Choosing an HPI method for our Kukun Investment Outlook, a zip code level forecast of home price appreciation, led me to a variety of questions. One of which was, why aren’t home price indices ever published alongside their performance metrics? At the end of the day, every index is also a mark-to-market model (though not necessarily a good one). PPE_10[1], a standard metric of AVM[2] performance, can easily be produced for any index by taking prior transactions and using them to predict prices on recent sales.  The economic answer is that real estate dynamics are extremely local while these indices often attempt to track home price trends in large geographic areas like MSAs, counties, and cities. That said, all things considered, a home price index that performs better as an AVM (albeit only on properties that have a prior sale) should be considered a superior index: This philosophy inspired our KIO design. After careful consideration of the alternatives, we eventually settled on an approach we believed would maximize our forecast’s accuracy and reasonableness.

There are four main methods for building home price indices:

1. Median Sales Price Indices

These simply track the median sale price over time in a given geography.

2. Repeat Transactions Indices 

RTIs look at pairs of sales and assume that value changes can be attributed to changes in supply and demand over time.   

3. Autoregressive Price Indices

A method that is similar to the RTI, but that allows single sales to contribute information to the index.

4. Hedonic Price Indices

These use an AVM to produce the index. These can be extracted by looking at coefficients produced by the model or by using the model to predict prices for all the homes in a given area.

Hedonic and autoregressive methods that take the median home price prediction for all the homes in a particular location, not just recent sales, outperform more traditional repeat transaction methods (in fairness, though, they all are beaten by proper AVMs). This suggests that RTIs, long the industry standard, may have serious drawbacks.

Read more: Building home price indices

Here are just some of the problems with RTIs:

1. Limited Data Availability

With sales volumes dropping in this high interest-rate environment, we can expect RTIs to become less reliable. RTIs require not only sales, but—as the name suggests—repeat sales. As sales volumes drop, repeat sales, which are necessarily a fraction of sales, are even more likely to drop below the number needed to produce a robust estimate. Furthermore, new homes now make up a larger percentage of sales than they have in a very long time because home builders are not subject to the “locked-in” effect the same way regular homeowners are—a homeowner who sells his home, which has an older mortgage with a lower mortgage rate, must get a new mortgage at a higher rate. The popularity of new homes will further decrease the statistical reliability of RTIs.  

2. Sample Bias

RTIs are based on a sample of homes that have sold more than once, which can lead to sample bias. Homes that are more likely to be sold repeatedly, such as those in high turnover areas or those that have appreciated more rapidly, are over-represented in the sample. This can produce biased estimates of price changes over time. Median sales price indices, like that produced by the NAR, suffer even more from this problem. Autoregressive indices ameliorate this problem somewhat by utilizing single sales as well as repeat sales, but repeat sales are still given more weight implicitly simply because they appear more often in the series. If your sale appears three times, you will receive roughly three times the weight. Nevertheless, PPE_10 metrics for ARIs are higher than for traditional RTIs.  

It might not be obvious why differences in turnover rate can bias RTIs so much. However, imagine a young couple in their late twenties buying a starter home and then, once they reach their late thirties, buying a larger home. They may stay in this starter home for only a few years. Once their family grows larger, they are likely to buy a nicer home that they remain in until retirement. These starter homes, which may appreciate more quickly because builders tend to build nicer, larger homes with the handful of permits they are granted, might appreciate at a different rate than the nicer homes people move into during midlife. In fact, if you look at price appreciation by price tier, we often see lower-end homes appreciating more because of constricted supply. Home builders are generally building nicer homes, meaning the stock of “starter homes” remains relatively fixed.

3. Neglecting property characteristics, including falling demand for older housing stock

RTIs make an implicit assumption about the quality and the condition of the home remaining constant over time, something that is not the case.  

4. Inability to isolate demand shifts

RTIs do not account for changes in the composition of homes that are sold over time, which may be influenced by shifts in consumer preferences. For example, if there is an increase in demand for homes with multiple bathrooms (imagine you have an increase in the birth rate and higher family formation), this could change home prices without implying that a two-bedroom house experienced an equivalent amount of appreciation.  An RTI would wrongly attribute these changes entirely to a general increase in home prices when the increase in demand was just for a certain segment of the housing stock.  

We should move the industry away from RTIs, which underperform predictively. HPIs must be judged, at least in part, by their performance metrics. Some may object claiming that the purpose of the index is not to predict home prices but to anticipate the theoretical price appreciation of a well-maintained home; namely, its purpose is to aid economists, not modelers. Fair enough, but even here an objective test is needed. We can meet that objection by employing another method. We can test home price indices by using them to make the time adjustments within an existing AVM, be it a neural net model or a comparable sales model, instead of directly incorporating a time component. The index that returns the highest PPE_10 within the given AVM framework can be judged the best. Of course, different AVM methods might return different results—in which case the one that works the best within the model that otherwise performs the best should probably be chosen. My own experience is that autoregressive and hedonic indices perform the best whether you are benchmarking them by marking-to-market or by incorporating them into an AVM. Economists may reject this suggestion, but they ought to respond by putting up another one rather than turning their backs on empirical testing entirely.    

Having rejected RTIs, we were left to choose between an autoregressive index and a hedonic index, which perform similarly. Here we chose a hedonic index because it flows naturally out of our existing AVM process—and assures consistency between our models. RTIs and autoregressive indices, due to the implicit waiting issues we discussed above, can often tell a story that is not consistent with the time series of values your AVM has been producing. Ultimately it was this fact that broke what was essentially a tie and made us go with a hedonic approach.

While business and economic considerations are important, we need to begin testing our home price indices. While I can readily accept that different indices might be used to gauge subtly different things, these subtleties should be something we can account for by employing different testing methods. There is no justification for accepting these indices, which often paint quite dissimilar pictures of the housing market, without subjecting them to more rigorous testing. In developing our KIO, a zip code level forecast of home prices, we were careful to use performance metrics when deciding which home price indices to incorporate—which significantly improved its predictive ability.

[1] PPE_10, proportional performance error, is the percentage of the time an AVM (see below) predicts a price that is within ten percent of the observed sales price.

[2] Automated Valuation Model, a model that predicts a home’s price based upon its features and the time of sale.

Read more: Agent based models

Franklin Carroll

Franklin Carroll is Kukun's VP of Modelling and Analytics; After finishing his studies at the University of Chicago, he worked at Fannie Mae where he helped build Collateral Underwriter, roll out the Value Verify initiative, and specialized in collateral related and credit related policy.

Published August 18, 2023