New Research Highlights Motivations of Reviewers

Study Looked at Airbnb’s Review System and Found that Holding Reviews Until Both Guest and Host Submitted Their Feedback Increased Review Rates

Key Takeaways:

• Experiment at Airbnb reveals a more effective way to generate online reviews.
• Holding access to reviews for both the seller and the customer generates an increase in reviews, as well as increases accuracy of reviews.

CATONSVILLE, MD, November 3, 2021 – Online sellers and marketers are quite familiar with reputation systems that seek to understand and shape online reputations, oftentimes in the form of online reviews.

Stay at an Airbnb property, and you’re likely to be asked for feedback about your stay in the form of a review. This helps Airbnb and the host better learn what they did right and what they could improve for future guests. It also helps other consumers make purchase decisions based on the reviews they see.

But if you’re the consumer, you’re not the only one who can give a review. The host may submit a review of the guest. This helps Airbnb track and monitor the behaviors of its own customers, which can help the platform avoid problems with future stays involving the same guest. It can also help the platform create marketing incentives for ‘model’ guests.

A challenge for online marketers is that not all reputation systems work the same way, and the right strategy may depend on the reputation system’s rules. For example, reputation systems differ in who is allowed to review and what information is displayed (e.g., star ratings and text).

A group of researchers sought to evaluate one of the more common reputation system design decisions, which is centered on the timing of when the user can see feedback written about themselves.

The study, to be published in the November issue of the INFORMS journal Marketing Science, “Reciprocity and Unveiling in Two-Sided Reputation Systems: Evidence from an Experiment on Airbnb,” is authored by Andrey Fradkin of Boston University and the MIT Initiative on the Digital Economy; Elena Grewal of Yale University; and David Holtz of the University of California, Berkeley, and the MIT Initiative on the Digital Economy.

“Online reviews provide valuable data to platforms and marketers,” says Fradkin. “But the problem is, for many platforms, reviews are underprovided, leaving both platform and marketer in the dark on what went well and what did not.” 

The researchers decided to use a large-scale experiment on Airbnb to study a potential way to reduce the problem of missing and biased reviews.

The treatment in the experiment featured a simultaneous reveal review system where the guest and the host each had the opportunity to review one another before reviews were revealed. In the control group of the study, reviews were revealed to users and to the public immediately after they were submitted.

“The design in the control group left open the possibility that the second reviewer could reciprocate or retaliate against the first review,” says Fradkin. “In the treatment group, this was impossible because first reviews were hidden to the second reviewer.”

“Ultimately, we found that when reviews were hidden until both parties had submitted their reviews there was a shortening of the time to receive reviews and reviews were more likely to be accurate,” Fradkin says. The authors argue that these effects are driven by curiosity and business interest. Users want to see what was written about them right away, and quickly submit their review as a result. Users may also want others on Airbnb to see these reviews.

Link to Study

About INFORMS and Marketing Science

Marketing Science is a premier peer-reviewed scholarly marketing journal focused on research using quantitative approaches to study all aspects of the interface between consumers and firms. It is published by INFORMS, the leading international association for the decision and data sciences. More information is available at www.informs.org or @informs.

# # #