Posted 18 December 2009
UCD computer scientists develop technique to improve helpfulness of user-generated online reviews
Irish computer scientists have developed a system to improve the helpfulness of online customer reviews. In recognition of the quality of their work, the group received a distinguished paper award at the British Computer Society’s annual Artificial Intelligence Conference.
“Our technique automatically analyses online user-generated reviews based on four key factors - reputation, social, sentiment and content - and determines how helpful the reviews are likely to be to customers before presenting the most helpful first – including both positive and negative reviews,” says UCD Professor Barry Smyth, Director of CLARITY, one of the researchers involved in the work.
User-generated reviews have become increasingly important when it comes to users making buying decisions. While online product or service reviews have become more common, they tend to vary greatly in their quality and helpfulness.
“Some online user-generated reviews can be biased or poorly written, while others can be balanced and insightful,” says Professor Smyth. “That is why we worked to develop an automated technique of accurately identifying helpful product or service reviews for the user.”
“Some online services are already trying to do this by allowing users to rate the helpfulness of a review, but this type of feedback tends to be sparse and varied. In fact, if the online reviews received enough helpfulness feedback from users there would be little need for our work in this area.”
According to the UCD computer scientists, after sampling 225,000 reviews by 45,000 distinct reviewers on 70,000 hotels on TripAdvisor, they discovered that about 65% of reviews did not receive sufficient feedback to allow for a straightforward determination of their helpfulness.”
“The infrequent helpfulness ratings that we identified would not give users a lot of confidence in the reviews,” explains Dr Michael O’Mahony, another of the UCD researchers involved in the work. “It’s about identifying which reviews are the most informative about the product or service. After all TripAdvisor currently hosts over 30 million user-generated reviews, so there is an amazing amount of latent value in existing online reviews.”
The system devised by the researchers applies machine learning techniques to discover the factors that make for a helpful review. It analyses reviews that have received feedback, and then applies the resulting models to evaluate the helpfulness of the majority of reviews that have little or no feedback.
The technique applies four key attributes to predict the helpfulness of a review: reputation, social, sentiment, and content. When added together, these elements offer a greatly improved helpfulness classification for online reviews.
Reputation captures a reviewer’s reputation as regards all previous reviews written by the same author. Social takes account of the fact that when a user writes their own reviews they also tend to be more likely to respond to existing reviews and thereby improving the quality of their own review. Sentiment considers that the score attributed to a product in the review is an indicator of review helpfulness, as users tend to respond more to positive reviews. Content accounts for how well the review is written, by way of form and length - poorly authored reviews are considered less favourable by users.
"One interesting result that we noticed is that users tend to be drawn more towards positive reviews and often ignore negative reviews," says Dr O'Mahony, "even though the negative reviews can be very revealing and informative. Thus one of the benefits of our system is that it can be used to promote helpful negative reviews that might otherwise be hidden within a mass of review content."
“Ultimately the user benefits by having access to more helpful reviews. And websites like TripAdvisor will benefit from a more satisfied user-base, and even the service providers (the hotel owners in this case) will benefit because the good services or products will have a better chance of attracting customers, while the poorer ones will have to be improved in order to attract better reviews.”
At the British Computer Society’s annual Artificial Intelligence Conference on 17 December 2009, CLARITY researchers also claimed the award for the Best Technical Paper for their paper entitled "Coping with Noisy Search Experiences" which describes new research on the use of online search engines. The research describes the HeyStaks technology which works with Google and Bing to help users better collaborate while they search. A new start-up company, HeyStaks Technologies Ltd is working to bring this social search technology to the marketplace. The researchers involved in this work include: Professor Barry Smyth, Dr Maurice Coyle, Dr Peter Briggs, from CLARITY (and co-founders of Heystaks Technologies Limited), and by Dr. Pierre-Antoine Champin, a visiting researcher from the University of Lyon.
CLARITY is a Science Foundation Ireland (SFI) Centre for Science, Engineering and Technology (CSET). CLARITY focuses on the intersection between two important research areas: adaptive sensing and information discovery. CLARITY is a partnership between University College Dublin and Dublin City University, supported by research at the Tyndall National Institute, Cork. This ground breaking research centre focuses on the Sensor Web, which captures the intersection between two important researches areas-Adaptive Sensing and Information Discovery. CLARITY brings together software and electronic engineers, material scientists and sports scientists to meet joint research objectives. CLARITY is based on a far-reaching academic-industry partnership consisting of academic and industry partners/collaborators, as well as national agencies and social partners.
(Produced by UCD University Relations)
