Lesson 5

A/B Testing Your Creative Assets

There are so many variations on icons, screenshots and videos in the app stores that it can be hard to know what’s truly delivering results. That’s where A/B testing comes into play; it can give you the hard data you need to prove that what may seem like a small variation — a new color treatment in your app icon, for example — can make a significant difference in conversion rates.

A/B testing is, simply put, a method of comparing two versions against each other to see which performs better. As a larger method, A/B testing is used to test everything from headlines to new features, but in the context of user acquisition, it applies to creative assets and advertising (which we’ll get to shortly).

Chances are, your first asset designs won’t be the final ones — and they probably never should be. By testing new designs and messaging, you’ll be able to zero in on how to attract more high-quality users.

A/B testing can also be a powerful lesson in how important it is to not make assumptions about what will or won’t work. Testing often proves us wrong.

Everything Improves With Testing

Ideally, all of your assets, from icons to screenshots to color palette and messaging, should be A/B tested. Testing can reveal insights that translate directly into significant gains. You may not get a 100% increase, but a better screenshot can get you 5% more users here, a better app icon 3% more user there. These small changes can really add up..

In fact, Google Play includes a comprehensive A/B testing tool for developers that’s surprisingly comprehensive for a free service. When it’s that easy, and free, there is no excuse not to experiment and improve.

Instagram recently took users behind the scenes to share the months-long process of designing its new logo. It took many (many) iterations of design and qualitative testing to ensure that it would work across platforms and align with other apps in the same family.

Insert screenshot here: Instagram logos side by side, old and new (AA)

Introduce Users to the App Experience

Another thing to keep an eye on is whether your creative assets are setting users up for the actual in-app experience. Onboarding, for example, doesn’t begin when users open your app for the first time — it begins with their first glimpse of your app store page.

Providing the right mix of app metadata (e.g., your name, description and creative assets) helps to reduce friction during discovery, download and first use. By testing, you can make sure your assets are driving home the right messages and matching user expectations.

Use Competitive Examples for Inspiration

Competitive analysis can be a valuable supplement to A/B testing. You can see competitors who made a change, either in an ad campaign or creative assets, and see how that affected their download and conversation rates. This kind of insight allows you to adjust your ASO plan accordingly.

It’s an Ongoing Process

The most successful app publishers constantly challenge themselves to improve performance on different parts of an ASO campaign. They continually ask themselves questions like, “If our app icon is performing well now, how can we make it even better?”

Chances are you haven’t really examined every aspect of your app store icon — after all, few publishers have the same time and resources to invest in such an endeavor the way Instagram did. But apps need updates, and user preferences change. A/B testing can help you set performance benchmarks to reach and even exceed.

Keep Learning

In the next lesson, learn about the global market: what works in some countries won’t work in others.

Next LessonPrevious Lesson