Follow Insights Blog


CoreLogic Econ


Channeling Deep Blue Versus Garry Kasparov in Home Valuations

It’s time to standardize how real estate appraisers make their adjustments

Michael G. Bradley    |    Property Valuation

It might be time to reengineer the process appraisers use to make adjustments to comparable homes. The current approach does not appear to be defensible.

Completion of a uniform residential appraisal report by a licensed or certified appraiser is the dominant process used to determine the value of residential real estate in the United States. Fed Economist Leonard Nakamura reported in his 2010 Business Review article that roughly $4 billion was spent on appraisals in the U.S. in 2005 alone, backing a lending process with trillions of dollars at stake. Given its importance, it is quite surprising that very little research exists examining the current appraisal process with the goal to improve longstanding practices.

During the 2014 CoreLogic RiskSummit, we reported on a rather unique test we performed to determine how appraisers go about making adjustments to comparable properties. The test was fashioned after “Deep Blue versus Garry Kasparov,” a pair of six-game human–computer chess matches that pitted machine against man. The matches were played between the IBM supercomputer Deep Blue and World Chess Champion Garry Kasparov. Though Garry was able to outsmart the machine in the first round of matches, the machine ended up winning the rematch. At that point, with the help of a team of IBM engineers who fine-tuned and reprogrammed the machine, the number of possible moves it could then consider was simply impossible for any human to process.

An appraiser faces a similar problem in that comparable properties often differ from the property being appraised in a wide variety of dimensions, with the number of possible adjustments being quite large. How exactly does the appraiser determine which property features to adjust and how much those adjustments should be? Machines, aided by huge amounts of data and advanced statistical models, do a much better job of performing these tasks.

To investigate this issue, we obtained a set of relocation appraisals from an appraisal management company and conducted our own analysis. (As background, for relocation deals, two or more appraisals are ordered at the same time; and the two appraisers often choose an identical comparable property).

For the experiment, we:
1) Obtained a set of relocation appraisals—two appraisals on each property prepared at roughly the same date for which the appraisers choose an identical comparable.
2) Looked at the adjustments made to that comparable and noted discrepancies in the amount of the adjustments, and even the direction (positive or negative).

Although our full analysis involved a larger set of paired results, to keep things manageable, I’ll present just two examples. For privacy purposes, we do not show address information.

For property 1, Appraiser A made an $18,700 negative adjustment for “site area” while Appraiser B made a $33,020 negative adjustment for this same attribute. Appraiser A made no adjustment for “quality of construction” while Appraiser B made a $15,000 negative adjustment. After making all adjustments, the adjusted sales price is $831,250 for Appraiser A ,while it is $796,010 for Appraiser B.

For property 2, Appraiser C made a $11,000 adjustment for “age” while Appraiser D made no adjustment for age. For “condition,” Appraiser C made no adjustment while Appraiser D made a $20,000 adjustment.

These examples serve to illustrate the major result from our fuller analysis – namely, that there seems to be no defensible rationale for the adjustments appraisers make to comparable properties when performing their work. Indeed, even for adjustments that should be fairly easy to quantify, e.g., for “gross living area,” appraiser adjustments were quite inconsistent. For some features, at times, even the direction of the adjustments went in opposite directions.

It’s time that we re-engineer the process so that appraisers can focus on the things that may benefit from the human element, such as defining the neighborhood and selecting comparables. But for the pieces of the puzzle that need to be standardized, like adjustments to comparables, we should be harnessing the power of our machines. Too much is at stake to maintain the status quo. Appraisers would benefit by being allowed to focus on things they do well; loan applicants would benefit from a less-costly, streamlined process; and lenders would benefit from valuations that were standardized and more reliable.

© 2014 CoreLogic, Inc. All rights reserved.