App analytics

How to A/B test your app in app stores

How you present your app in your app store account will have a dramatic effect on how users will react to it, especially when you’re competing with dozens, scores and hundreds of other apps that offer similar or identical services. Your app’s icon, screenshots, description, colors… can mean the difference between the success and failure of your user acquisition strategy.

But how can you choose between different icons, or decide whether you should start your app’s description with “Best-selling image editing app” or a simple list of features?

The traditional way is to research other apps, scrutinize your competitors, make an educated guess or rely on your gut feeling. But the scientific and more accurate method to make such choices is to do A/B testing, in which you weigh user response on different variations of an element and choose the one that gives the best results.

There are many tools that allow you to perform A/B testing on your app, but app stores have no such features and options. Therefore you’ll have to make up for the lack by mimicking the app store experience. In this article, I’ll guide you through the steps to A/B test your app’s icon. The same procedure and guidelines can be applied to all aspects of your app store definition.

The landing page

The first thing to do is to create a mobile landing page to showcase your app. When building the site, we’ll use the same elements that a customer sees when visiting the app in major app stores, including an icon, description, and screenshots. You’ll also need a call-to-action button, such as “Buy Now” or “Download,” which will be the ultimate test of success.


The goal is to replicate the app store experience in a controlled environment that will allow you to capture and examine user behavior in reaction to different icons.

There are many tools that can help create mobile landing pages, such as Pijnz and Landr. Alternatively, you can get a template from sites such as ThemeForest, and if you have no web coding experience, you can always hire a freelancer from Upwork or eLance.

The A/B test

After creating the landing page, you need to create several variations of the page, each one displaying a different icon. However, take care to leave all other elements intact in order to make sure the metrics you’re receiving are precisely reflecting changes in the icon.

Tools like Optimizely and VWO can help you in creating the variations if you’ve built the landing page using a template or html.

Take note that the smaller the number of variations, the easier it will be to obtain and evaluate results. If you have several variations, you’ll have to direct more traffic and users to the landing page in order to get more precise results.

The testers

Once you’ve setup your landing page, you’ll have to start directing traffic toward it. If you have a strong social media presence, you can try free promotion by sharing the link to your page with your audience in Twitter, Facebook and other social media platforms. If not, I advise you to make a small investment (as little as $50) to run a paid advertisement campaign and send a few hundred visitors to your landing page.

The number of users you’ll have to test your app icon against will depend on the number of variations that you have. If you have four or less icons, you can run the a few hundred visits will be fine. But if you add more, you’ll need larger amounts of traffic to obtain precise results, which is OK if you don’t mind spending a few more dollars.

There are several ad networks that can get you started on mobile ads, including Google AdWords, and AdMob.

The results

As the ad campaign starts to send traffic to your landing page, you’ll get a clearer picture of which icon is performing better and is more likely to persuade users to download or purchase your app.

If you’ve built the landing page with custom code or a template, you’re going to need to use a service to track the number of clicks on your call-to-action button. Google Analytics and KISSMetrics are two analytics tools that can help you setup goals and events, and gather information on user behavior.


Let the experiment run for a couple of days until you get enough clicks on your call-to-action buttons to be able to make a correct assessment about which icon is producing better conversion rates.

Even little differences can mean a lot when multiplied by the number of users who will eventually visit your app in app stores. It will also affect your app store ranking and possibly help you nail one of the coveted spots in app search results, which could lead to a sky rocket effect on your app sales and downloads.

You can never say that the results from A/B testing are 100% accurate, but they’re based in science and are much more precise than what you would get if you just surmised and relied on your gut feeling.

From here on

The same approach you used with your app icon can be applied to your app name, description, price and screenshots. Pricing in particular warrants a little elaboration. When testing different prices for your app, you can get a more precise assessment of how an increase or decrease in price would affect both the download and revenue of your app.

For instance, say you want to test the price of your app at $0.99, $1.99 and $2.99. Obviously, you can expect your download rates to decrease as your price increases. But when you have a relatively-accurate assessment of fluctuations in download rates based on price variations, you can quickly multiply the revenue by the number of downloads and choose the price that yields the best results. Setting up values for goals in tools like Google Analytics will help a lot in this regard.

How do you optimize you app store definition? Share with us in the comments.

View Comments
There are currently no comments.