The ultimate way to disrobe Zestimates | Inman Philip Gray, a San Leandro, California-based appraiser, is taking this approach. Bringing up the Zestimate of the property Rascoff recently offloaded will help him deal with the frequent pushback he receives from homeowners “who think Zillow is the magic 8-ball,” he said. ‘We missed’  Zestimates on Rascoff’s former home have certainly been overstating the property’s value, said Zillow Chief Analytics Officer Stan Humphries. “The fact that we missed and there are empirical reasons we missed — that’s a great conversation that real estate agents should have” with consumers, he said, citing the property’s irregular lot and location on a busy road as partly responsible for its Zestimate’s inaccuracy. But he expressed hope that, in the same discussion, agents also won’t instill “data nihilism” in consumers, and that they acknowledge that humans also can miss the mark. Smaller gap at start In July, the Zestimate of Rascoff’s former property wouldn’t have raised the eyebrows of anyone who’s familiar with automated valuation models (AVMs). At $1.388 million, the property’s Zestimate was 7.3 percent higher than its listing price of $1.295 million at the time. Since Zillow only shows revised historical Zestimate data on property pages, the home’s property page currently indicates that the property’s Zestimate was around $1.6 million in July 2015, somewhere in the neighborhood of $200,000 more than the Zestimate that actually appeared on its property page on July 17, 2015. Screen shot showing historically revised February Zestimate of Rascoff’s former property.

For all anyone knew in July 2015, the property might have eventually sold at a price closer to its Zestimate than its listing price. But that didn’t happen. The home later sold for $1.05 million, 19 percent below its July listing price. Undergoing a number of price cuts, the property was listed and de-listed several times between when it was originally listed on July 7, 2015 and when it sold on February 29, 2016. If Rascoff thought his home was worth its July listing price, the outcome of the sale might have come as a disappointment. But if the success of the transaction were judged by the property’s Zestimate, it was a failure. The home’s Zestimate was $1,750,405 on March 1, the day after the property sold for $1,050,000.

If that Zestimate were accurate, it would mean the chief of the biggest name in real estate and the recent co-author of a book about “the new rules of real estate” would have sold his home for 40 percent less than it was worth. Automated valuations vary In addition to highlighting the shortcomings of Zestimates, the Zestimate of Rascoff’s home also brings into focus the potential for some automated valuations to be more accurate than others. Unlike Zillow’s property page on the home the day after it sold, Redfin’s page on the home showed that the sale had occurred. At the time, it displayed a valuation of $1.1 million — much closer to the property’s sales price of $1.05 million. Redfin property page of Rascoff’s former property shows a home value estimate of $1.1 million the day after it sold.

On Thursday, May 5, Redfin’s estimate of the home’s value was $1.3 million. So while Zillow’s estimate had come down by around $140,000 since the home sold, Redfin’s had increased by about $200,000. Both differed from the price the home sold for a little over two months ago by hundreds of thousands of dollars. Zillow has since added the sales price of Rascoff’s former home to its property page. The property’s Zestimate had slipped from $1,750,405 the day after it sold to $1,608,670 on May 5, but its Zestimate on May 5 still only represented 65 percent of what the home sold for a little over two months before. To judge the Zestimate’s accuracy based solely on the gap between the sales price of Rascoff’s former home and its Zestimate would probably be unfair. The discrepancy is unusually wide, according to what Zillow says is the Zestimate’s median error rate. Zillow puts the Zestimate’s national median error rate at 7.9 percent, meaning half of Zestimates nationwide are within 7.9 percent of a home’s sales price and half are off by more than 7.9 percent. The listing portal claims an even higher level of accuracy in Seattle, where Rascoff’s former home is located. There, Zestimates for half of homes are supposed to be within 6.1 percent of their sales price, while half are supposed to be off by more than 6.1 percent. This suggests that the Zestimate of Rascoff’s home missed by much more than normal in Seattle. Why was that? One reason is that the home’s Zestimate was comparing Rascoff’s former home, which is located on a triangular lot, to recently sold homes located on rectangular lots, according to Humphries. Since rectangular lots provide more utility than triangular lots, he said, that meant the Zestimate was

overvaluing the plot of Rascoff’s home. Another reason was that Rascoff’s home was located on an “arterial” road while nearby recently sold homes sat on quieter streets. Zillow continues to research how to program Zestimates to account for such factors, but “we haven’t fully cracked the nut on that one” yet, Humphries said. ‘The classic luxury homes problem’ Zillow Senior Economist Skylar Olsen added that the Zestimate of Rascoff’s home represents “the classic luxury homes problem.” Zestimates can’t take into account “non-quantifiable facts,” such as layout design or lighting, and these facts can have much more of an effect on the values of luxury homes than less expensive properties, she said. Real estate agents can see how special features impact a property’s value, but the “Zestimate algorithm can’t know” and “at this point in time, it’s not designed to know,” she said. The reason why the Zestimate of Rascoff’s former property hasn’t dropped dramatically since selling at a much lower price than Zestimates leading up to the sale is that the Zestimates have a “smoothing function” designed to keep them from overreacting to recent property sales. The Zestimate on the Rascoff’s former property will gradually come down to more closely resemble its sales price. And upcoming updates to the Zestimate’s algorithms will adjust the smoothing function so that the Zestimate of a home that sells will come to more closely mirror its sales price much faster. Also worth noting is that Zillow does not have access to sold listing data from the Northwest Multiple Listing Service, the MLS that covers Seattle. Automated valuation models (AVMs) that crunch sold MLS data can have an advantage over AVMs that only use public sales records — which are the only sales records used by Zestimates covering Seattle. While Zillow says on its website that most consumers understand that Zestimates truly are only estimates, the listing portal concedes that, sometimes, “someone will come along that insists on setting the price they are willing to buy or sell for based solely on the Zestimate.” Zillow goes on to say that “education is the key” and that, armed with knowledge of how Zestimates are calculated along with their local median error rate, agents can explain “why the Zestimate is a good starting point as well as a historical reference, but it should not be used for pricing a home.” While Zestimates can create hassles for agents, some agents would certainly agree with Zillow’s assertion that understanding how a Zestimate is calculated, along with its strengths and weaknesses, “can provide the real estate pro with an opportunity to demonstrate their expertise.”

The gap between the Zestimate of Rascoff’s former property and its sales price may have made it easier for agents to seize that opportunity. Zillow’s Humphries’ hopes that, when putting Zestimates in perspective for consumers, agents will also acknowledge that Zestimates do have a scientific basis, and that nobody’s perfect — even trained professionals. He noted that a study released by Zillow in 2012 showed that the typical gap between a home’s Zestimate and its sales price wasn’t that much larger than the typical gap between a home’s initial list price — which is often set based on a real estate agent’s recommendation — and its sales price. “We acknowledge humans are great at this, and we’re great too — but they’re greater,” Humphries said. Editor’s note: A previous version of this story incorrectly stated that realtor.com has access to the Seattle MLS, and that realtor.com showed status changes to a property that weren’t displayed on Zillow.