train recommendation algorithm based on sparse data

We were challenged to create a custom predictive analytics and recommender system to enhance an existing e-commerce platform and improve sales. And building a decent recommender system, ideally, requires having a large and diverse dataset of user behavior that can include purchase history, product page views, likes, ratings, reviews, etc.

At initial stages, we faced data limitations. We had only purchase history available at that point. To add more, a substantial part of users had quite a small number of purchases. And that was the only type of user data that we could work with. So, the data shortage complicated our task of tailoring an e-commerce analytics system and training a recommendation model. The challenge was to tap into the limited data on users’ purchasing history and harness it for predictive modeling.

Predictive analytics module for e-commerce platform

We based our development approach on collaborative filtering technique based on matrix factorization used in recommender system. We didn’t use any specific data about the users or products. We gathered only historical data on user-product interactions and also retrieved information about positive preferences of users to item. So we decided to stick with a simple yet well-proven implementation called implicit ALS to train our model on sparse data. Also, we had a reasonable amount of data on the number of user transactions that was provided by the Client. The collected data was used to train the ML model that would power a custom recommender system.



We used the confidence metric to train the model to emphasize items purchased several times over items purchased only once. Also, we had a significant number of users with few purchases, which was not enough for the recommendation model to give a reliable prediction. So, we implemented several additional techniques to enhance the recommendations given by the model. As a result, we made the algorithm to better understand user preferences and avoid recommending the same items to all users.

We applied several filters to make the model more accurate and relevant to the business goals As a result, relevant categories of users will get recommendations that will make them feel more satisfied with the services.

ML-based solution to automate processes and improve customer experience

We delivered the model to fit in with the business needs, which can be utilised in completing the following tasks:

• Recommend items to a given user
• Find similar users based on items preferences
• Recommend most likely users to purchase a given item
• Recommend similar items

Our solution perfectly met the following business needs:

• Make the search process highly personalized
• Automating routine tasks of shop assistants
• Ensure awesome online shopping experience
• Boost customer loyalty