Here’s a useful process for coming up with product improvements to a/b test. It’s for sites using the trial -> upsell -> monthly payments model.

Segment

Which segment of your customers do you want to target? If you are a startup, you probably want to target your most interested users: people who know they have the problem your app tries to solve, who have tried makeshift solutions, and who are willing to pay for a better solution.

What are they doing now?

Use your current dataset to figure out what this segment currently does from the second they arrive on your site to the second they leave.

Find the most common patterns.

What do they want? What are their incentives / pain points / goals?

Using your current survey dataset, validate your assumptions about what the segement wants to do when they come to your site. What are their immediate goals that drove them to your site in the first place? What are their longer term goals in life?

What do I want

1) Hopefully the first thing you want should be to solve your customers’ immediate goals.

1.5) The intermediate thing you want is probably going to be something related to showing the customer that you can also solve their larger goal / help them with their main pain point.

2) And of course you also want them to pay you.

How do I measure what I want?

Figure out a way to measure whether or not you are actually solving your customers’ immediate goals.

Figure out a way to measure whether you are properly using their longer-term incentives to lead to longer-term engagement with your app.

Where can I intercept them?

Now that you’ve sketched out what you/your customers want, as well as what your customers are doing, choose a point in their current login -> logout process to intercept them.

Then improve a current feature or build a new feature in order to intercept them before logout and solve their immediate goal and also show them that their longer-term goals are solvable with your app.

Do it in a way that is so minimal that you’re a) embarrassed to release it b) confidant that it will allow you to learn whether your ideas are on the right track.

Conclusion

I realize that this is all too abstract. E-mail me at f.mischa@gmail.com if you want to complain or comment.