Forget A/B testing: Order elements using personalized machine learning recommendations in JavaScript

There are some great tools out there for A/B testing that let you automatically present a selection of different experiences — whether layout, colors, fonts, or content. They measure which is most effective, and you make a final decision on which one to go with.

But what if we didn’t have to settle for one experience? What if we could use Machine Learning to learn which experience each user is most likely to engage with, and present different experiences to different kinds of people.

Personalised experience

Let’s say we have a list of choices to present to the user. We want to present the most relevant content first so that our user doesn’t get bored and instead engages with our app or website.

The choices could be anything — news articles, songs, pictures, products, tweets, or whatever your particular data is.

The most important thing about applying Machine Learning is to think carefully about your goals — what do you want to achieve? While it can be fun, applying new tech for its own sake is not awesome.

Given these choices, we want to change the order based on the persona of the visitor.

For choices A to F, they would be displayed differently for these three groups of users:

Machine Learning can learn about your users

The first thing we need to do is learn about our users.

We can start by randomly changing the order of the choices and tracking engagement (clicks probably) just like A/B testing does.

The difference is we will also capture some measurable data about the user too, which can start to form a model of their interests. For example, we might know their age, or location, or previous purchasing history. The goal here is to think about what properties might be important when tailoring the experience.

Things don’t have to get creepy — use data that the user knows you have and doesn’t mind you using. And always let them at least opt-out, if not having everyone opt-in.

We might also decide to give the model other inputs, such as the time of day, or even the current weather, if they are likely to influence our users’ decisions.

The learning applies to other users too

The insights we get from our users can be applied to other users, and even to brand new users who have never used our app before.

For example, given the following simple table of data about some users:

Name    City     AgeGroup   Loved
--------------------------------------------------------------------
Mat     London   30-40      The Matrix + 28 Days Later
David   London   30-40      The Matrix + 28 Days Later
Piotr   Warsaw   20-30      The Matrix + Jack Strong
Paweł   Warsaw   20-30      The Matrix + Jack Strong
Bill    London   60-70      Jack Strong + Das Boot

What films would you suggest to Piotr and Paweł?

Given their age group, and the fact that they liked The Matrix, it is probably sensible to recommend 28 Days Later to them.

If a new user comes along with the following properties, which films would you suggest?

Name    City     AgeGroup   Loved
--------------------------------------------------------------------
Peter   London   60-70      ?

Since he lives in London and is in the same age group as Bill, we might decide to recommend Jack Strong and Das Boot.

Of course, in the real world it’s nowhere near this simple and the patterns are likely to be much more nuanced — never mind when you introduce any kind of big data scale.

This is where Machine Learning can do a better job than humans.

Reward the model

Once we can make predictions, we need to track whether they’re successful or not.

Whenever we get something right (like a user clicks a choice, or watches a movie, or buys a product) we will reward the model to reinforce the learning.

Our model will then notice patterns, and make better predictions in the future.

Meet Suggestionbox

Suggestionbox is a tool from Machine Box that provides a Machine Learning model for this very use case. You use a simple JSON API to interact with the box.

Typing this single line into a terminal will download and run Suggestionbox for you (assuming you have installed Docker):

docker run -p 8080:8080 -e "MB_KEY=$MB_KEY" machinebox/suggestionbox

If you don’t have an MB_KEY — which you need to unlock the box — you can grab one from the Machine Box website.

We will use it to build a little demo to show personalization working.

Create a model

While you can create models via the API, it’s much easier to head to https://localhost:8080 and create the model using the Console.

I am going to create a model called Genres, with five choices.

Make predictions

Our model is now ready to start making predictions.

Of course, they’re not going to be very well informed initially, but very quickly Suggestionbox will notice patterns in the rewards, and we’ll start to see the predictions get better and better.

Try the free simulator

Suggestionbox ships with a built-in simulator, so you can actually simulate real user activity to see how the model might take shape. If you want to try this, you can do so from the Console at https://localhost:8080/console.

Wiring up our web page

There are two things our JavaScript needs to do:

1. Ask the model for a prediction, and use that prediction to decide on the order of elements on our page,
2. When the user successfully interacts with an element, reward the model so it can learn.

Assuming we had a user object that contained some relevant properties:

var user = {
    age: 30,
    city: "London",
    interests: ["music", "movies", "politics"]
}

CAUTION: The JavaScript on this page is simple and bearbones — you should use whatever UI technology you’re most familiar with instead.

Make a prediction

To make a prediction, we might do something like this:

This code turns our user object into a prediction request and makes an AJAX request to the /suggestionbox/models/{model_id}/predict endpoint.

The makeRequest helper can be replaced with the $.ajax call in jQuery, or whatever remote data API your UI framework provides.

The results will come back looking something like this:

Each choice is mentioned with an order and a score. The order is what we care about most, because that is the order we need to present the choices to the user.

The score is useful for debugging, but you shouldn’t order by it. Occasionally Suggestionbox will try novel things to see if there is any more learning it can do — in these cases, the first element in the array won’t necessarily have the highest score.

The reward_id values are used to issue rewards if the user engages with any of these options.

Reorder the elements

Assuming we have the elements in a container, we can just append them back to it to control the order.

If elements are being appended to the container they’re currently in, they’ll essentially just move to the end. We can use this to easily set the order just by iterating over the choices:

At the same time, we are going to capture the reward IDs in an object keyed by the choice ID. This will make it easier to lookup reward IDs later.

Reward the model

When the user clicks one of the choices, we’ll call this function which will make an AJAX request to reward the model:

This function just creates a reward request object that contains the appropriate ID (we look it up via our rewardIDs object) and a value which is 1 for most cases.

We make a POST request to /suggestionbox/models/{model_id}/rewards, before going about our business.

The model will learn that it did a good thing when it suggested that choice, and this is how it improves over time.

Full example

See a full example of this by checking out the suggestpage toy on GitHub.

Need help implementing something like this?

Our team hang out all day in the Machine Box Community Slack which you can get an invitation to today.