Internal Validation of Boxwood's Commercial Evaluation Reports Shows Solid Results

Commercial evaluations are an increasingly important tool for financial institutions that require valuations of small commercial properties. However, unlike appraisals, there is no significant quantitative measures or research regarding the accuracy of evaluations or the risk of relying on them. With recent regulatory changes that allow for use of evaluations on higher-value transactions, it is more important than ever that commercial lenders gain a solid understanding of the risks and accuracy when using these valuation tools.

However, measuring accuracy of evaluations versus appraisals is difficult for a number of reasons, including:

• Large data collections of appraised values stretching back to the 1970s are available for analysis. These data, mostly from the NCREIF National Property Index, provide the basis of most academic studies of appraisal accuracy. No such dataset exists for commercial evaluation products.
• Appraisal licensing requirements result in practitioners with fairly consistent training and education regardless of his/her place of employment. Analysts conducting evaluations have more variable skills and experience.
• Evaluations are less standardized and rigorous than USPAP Appraisals, thereby reducing the comparability of evaluation products among different firms offering them.

Boxwood overcomes these research challenges by using proprietary datasets of commercial evaluations conducted solely by in-house Boxwood analysts and appraisers. As a result, Boxwood's FieldSmart commercial evaluations reflect uniform content and formats. Additionally, all reports are performed by senior and well-qualified Boxwood staff. By statistically eliminating "noise" from these complicating factors, we can more easily measure evaluation accuracy. Though the results are not generalizable to the commercial evaluation industry as a whole, the findings do indicate how, under ideal conditions, evaluations can be quite reliable.

Briefly, we collected sales prices from public records for all properties for which, over the previous two years, we had prepared a commercial evaluation. When matching our evaluation report's Market Value Estimate (MVE) to the subsequent sale, we required that the sale occur within four months of completion of the evaluation. Also, we required a perfect match on address and all assessor parcel numbers listed in both the sale and the evaluation report in order to ensure an appropriate comparison. In total, we found 313 properties that matched these criteria.

The median difference between the market value estimates from these 313 properties and the subsequent sales prices was 2%, with the average error equal to 4.3%. Of the MVEs from these 313 commercial evaluations, 70% (or 219) were within 10% of the actual sales price as recorded by the public records.

How do these results compare to appraisals? In a word, favorably. Published studies of appraisal data over varying time periods are quite consistent in their findings and cited average errors on par with Boxwood’s evaluation reliability.

As Table 1 nearby shows, the fit of Boxwood's reports is fairly consistent across the different types of FieldSmart evaluations with trivial differences identified. (LTE follows the sales comparison approach with limited narrative and no listings; standard FS includes the sales comparison approach with listings and market narrative; SI includes both the sales and income approach, rent and sales listings, reconciliation and market narrative.) Each report type shows a median ratio of MVE to sales price of 1 to 1.039. Median errors are 5% or below across the board, and the percentage of reports within 10% of the sale price range from 65% to 74%.

At first glance, it may seem surprising that Boxwood’s evaluation reports have roughly equivalent reliability irrespective of their level of detail. However, our research findings indicate that clients tend to choose the more comprehensive reports (SI and standard FS) where properties are complex or difficult to value. When these factors are held constant, the more detailed FieldSmart SI report tends to reduce risks for clients.

Boxwood's dataset allows us to examine report performance from a variety of angles. For example, some voices in the industry were concerned that employing commercial evaluations on transactions above the original $250,000 threshold increased risk - as well as potentially jeopardizing the safety and soundness of financial institutions. Table 2 provides some quantitative insight into this question. Here we see little difference between the accuracy of the 182 properties valued less than $250,000 versus the 131 valued above $250,000. What differences are observed indicate that the higher-priced properties were valued a bit more accurately. Our findings suggest that this effect does not derive from clients ordering only easy-to-value properties above $250,000, nor does the effect vanish with even higher-valued valuations. In short, higher-priced properties are valued just as reliably as lower-priced assets.

We have only touched the surface in terms of reporting on our evaluation dataset and intend to update clients on the accuracy of our evaluation products in the future.

You can view a summary of the study's results with this infographic.