According to McKinsey, 35% of Amazon’s revenue is generated by its recommendation engine. How is it able to contribute to such a huge percentage of the sales?
Because recommendation engines deliver personalized suggestions, and personalization enhances customer experience. In fact, a survey by Invesp found that 56% of consumers would return to a site that recommends products because it delivers a personalized experience.
So it’s clear that implementing a recommender system will greatly enhance an e-commerce store. A question most businesses ask is if they should buy a ready-to-use recommender engine or build one from scratch.
We get to this question towards the end, but first, it's important to understand how a recommendation engine works and the different filtering models that can be used to make it faster, more efficient and accurate.
In this article, we cover the four popularly used types of recommender systems based on the filtering model:
- Content-based filtering
- Collaborative-based filtering
- Popularity-based filtering
- Hybrid filtering
And we also cover three performance measurement metrics:
- Measure of personalization
- Coverage
- Hit rate
What is a recommendation system?
Simply put, a recommendation system is a filter that selects specific items from the complete dataset, items that a user is very likely to find interesting, and presents them to the user as a recommendation.
Netflix uses a recommendation system to recommend movies, Amazon uses it to recommend products, YouTube uses it to prioritize videos, and Tinder uses it to filter matches. Every top aggregator of items and products uses a recommender engine to personalize the suggestions being made to their users.
The image above shows Amazon's recommender engine at work. The 'Frequently Bought Together' and 'Customers Who Bought This Item Also Bought' suggestions are being made by its recommendation system. The former is made based on the item that is added to the cart, and the latter is made based on the user's attributes. In both cases, specific books have been filtered out from their entire catalog of choices and presented to the user.
The types of recommendations systems listed in the next section will help you understand the working of this engine better.
Types of recommender systems
Content-based filtering
Content-based filtering compares product features to match similar products and then makes recommendations.
When a user shows interest in some content (which can be a product, a movie, a brand, and so on), the recommender system uses its features to find other, similar content and then recommends it to the user. Thus the name, content-based filtering. The recommendation happens based on the content the user interacts with:
For example, if a user purchases historical fiction like “A Thousand Splendid Suns” by Khaled Hosseini on Amazon, the recommender system looks for other books that have these key features as well - fiction, historical, Khaled Hosseini, and then recommends them to the user when they next visit Amazon.
Advantages
- Since recommendations are made based on content features and not user-based features, high volumes of user-specific data is not required for it to operate.
- A content-based filtering system can start recommending products as soon as the user has an interaction.
- This system caters to specific preferences because it takes into consideration a user's individual interactions only.
Disadvantages
- Product features should be extracted and categorized for the system to compute similarity, and this can be a time-consuming task.
- Since the system only considers the user's preferences to make recommendations, its power is limited and it continues to make recommendations within a closed boundary. Users can find themselves in a bubble and never experience different products they might have enjoyed.
- The system relies on continuous user activity to enhance the accuracy of recommendations for that particular user.
Collaborative filtering
Collaborative filtering groups (or collaborates) people together based on their similarities and then makes a recommendation to one user based on items the other users in the group have shown interest in. The assumption is that people who are similar will also like similar items.
If user A and user B are similar, and if A purchased product 1, this product is recommended to B because B has similar interests as A and will most likely want to buy product 1 too.
Recommendations using collaborative filtering are carried out in one of two ways:
- User-based
- Item-based
User-based
In user-based collaborative filtering, the system may first group users A and B based on the similarity in their purchase history - they bought the same two books. The book 'The Kite Runner' is then recommended to user A because it was purchased by user B. The features of the book itself are irrelevant; the recommendation is made by matching user interactions, hence the name user-based recommendations. Anytime you see a “Users similar to you also liked” section, it is a user-based collaborative filtering recommender system in action. Let’s look at item-based filtering.
Item-based
In item-based filtering, the recommender system looks for similarities between two items in order to make recommendations. Although it sounds similar to content-based filtering, there is one major difference. In item-based collaborative filtering, the system does not match item features to tag products as similar, which is what happens in content-based filtering.
Similarity calculation is done based on item interactions by users who have been grouped together by the collaborative filter. In the image above, users A, B, and C are grouped as similar by the collaborative filter. All three users purchased the book To Kill A Mockingbird and also the book Fahrenheit 451, so the system tags the two books as similar. The next time a user from this group buys To Kill A Mockingbird, Fahrenheit 451 is recommended to them.
Advantages
- Recommendations are fairly accurate even with smaller datasets
- Gets users out of the recommendation 'bubble' that happens with content-based filtering by helping them discover products that may be out of their usual area of interest (but purchased by similar users).
- Domain knowledge isn't necessary; the system is able to self-learn as interactions increase.
Disadvantages
- Newly added items will not enter the recommendation loop until they are interacted with.
- It's hard to include product features into the filtering algorithm.
- There needs to be a good amount of user interactions for the recommendations to be accurate.
Popularity based filtering
Here, products are recommended based on how popular they are. Items that are interacted with by a lot of users are considered to be 'trending' and these are recommended to other users.
There is no weightage given to item or user features, recommendations are solely based on trends.
Advantages
- It does not suffer from a cold start and can start delivering immediately.
- User historic data is not needed so new users can also get recommendations.
Disadvantages
- Recommendations are not personalized to the user.
- All users will get the same or similar recommendations and other products will not be promoted.
Hybrid filtering
In this method, both user attributes and item features are considered when making recommendations.
For each user, products are given a binary value (0 or 1) that indicates if the user is interested in the product or not.
User attributes like age, gender, geography, past interactions, etc., are matched with item features like pricing, size, availability, etc., to assign the binary value to the product. Products that the user is more likely to be interested in are then recommended.
Advantages
- Results will be personalized to the user
- As interactions increase, the accuracy of recommendations also increases.
Disadvantages
- Huge amounts of data need to be collected and classified since both user and item features are being considered.
Measuring the Performance of a Recommendation System
Once a recommender system is in place, it is important to measure its performance, that is, how successful its recommendations are. The following three metrics will help you determine the accuracy and performance of the recommender system:
Personalization
The first metric to measure the performance of a recommender system is the level of personalization. Is the system making unique suggestions to each user that are personal to them, or is it making generalized suggestions to all users? Personalization score can be calculated using the 1-cosine similarity method.
Coverage
Coverage is the percentage of items from the training dataset that are being recommended on a test set. If the training set has 100 items and the recommender engine suggested 10 items, the coverage is 10 percent.
The quality of coverage depends on the type of filtering model and the percentage. A popularity-based recommender will generally suggest the same trending products to all users, so the coverage is expected to be low. A random recommender can easily have coverage of 100% because it recommends products randomly. Therefore, a KPI is defined based on the filtering model and the business, and then coverage is measured.
Hit rate
The 'hit rate' is a measure of how accurate each recommendation was for a user. It is calculated by measuring how many recommended items the user interacted with, where the interaction is an end goal (like a purchase or a watched movie).
You can generate the top 'n' recommendations made for a user, and then check the items the user has watched, purchased, rated, liked, etc. For each positive interaction, incrementally increase the rate by 1. The final number divided by 'n' is the hit rate.
Getting started
As owners of a brand with expertise in business, wrapping your head around the technical nuances of a recommender system can be terrifying. Fortunately, there are plug-and-play systems available today that have done all the groundwork, and are available to integrate with your business and its data. .
Argoid's recommender system, for example, can be used across different platforms - e-commerce stores, OTT platforms, Shopify stores, and so on, without needing any technical expertise from your end. This allows you to reap the benefits of personalization without any of the hassles. In fact, our customers see great results within just a few weeks of implementation.
If you’d like to see if your platform and our recommendation engine are a good fit, drop us a message, and we can schedule a demo to show you just how easy we’ve made it.
.png)
.png)
