“A/B testing should be the same for both Android and iOS apps. How different can it be?” This assumption may have popped into our minds several times in the process of learning how app store testing works. But truth be told, they are two very different things. Learning the specifications of different platforms will allow you to build a better mobile user acquisition strategy through experimentation.
What is A/B Testing for Mobile Apps
To begin with, Mobile App A/B testing is the practice of using different test variants to measure different user experiences within the app and figuring out which one leads to higher conversions.
Just to recap, let’s remember the main ingredients of App Store Optimization: Keyword Optimization & Conversion Optimization.
Consistent and persistent optimizations to your app are important but knowing what optimizations have a significant impact on conversion rates can be worthwhile. Collecting data will allow you to make data-driven decisions to your app marketing strategy, increasing the effectiveness of your app’s growth efforts.
Moreover, A/B testing helps optimize in-app engagements, helps you understand what works for different target audiences, helps comprehend the impact of a new feature, and gives a deeper analysis of human behavior.
Now let’s jump to the main part of unraveling differences between A/B testing for Apple Play Store and Google Play Store.
A/B Testing in the App Store
With Apple App Store you can test the app’s icon, screenshots, previews, and promotional text. Not to forget but the most impactful news has been the announcement of iOS 15 which sparked excitement and apprehension in the app marketing community.
Furthermore, at WWDC 2021, Apple announced the release of Product Page Optimization where one can run tests on their product pages for organic traffic and determine which one is more powerful in convincing users to download the app.
This feature is expected to influence the conversion rates and so allow the Conversion Rate Optimization (CRO) on the App Store to match Google Play.
Before iOS 15, the presence of only one product page meant that all the organic traffic was directed here. However, with iOS 15, we will have one default product page and three variant tests where we can modify the icon, screenshots, and video previews. After the testing goes live, we will be able to trace impressions, downloads, and conversion rates.
In addition to Product Page Optimization, iOS 15’s advent allows the landing of organic and paid traffic on different app listings and enables mobile marketers to monitor campaigns more effectively, leading to accelerated conversions and growth.
With custom product pages, each user coming on the app store will see different product pages based on the advertising campaign they are coming from. Unlike PPO, the app icon will remain constant but we will be able to test different combinations of screenshots, video previews, and promotional text. Under custom product pages, we can devise one default page and 35 different variants showcasing different features to the relevant users.
Lastly, the release of iOS 15 has induced the display of in-app events which will assist users to discover events associated with an app on the Apple App Store.
However, in the Apple App Store, if you want to test the different alternatives of your app’s icon, you will have to include all the variants in the binary of your published app.
Moreover, the metadata of your app speaks volumes of how your app will or will not be ranked in the App Store. The 30 character App Title and Subtitle field are crucial for your organic rankings. This being said, it is very important that apps put in only the most profitable and well-performing keywords in the limited keyword fields.
Google Play A/B Testing
Meanwhile, on Google Play, one can test the app’s icons, screenshots, video previews, featured graphs, and short and long descriptions. Which means that on Google Play all localizable assets are eligible for A/B testing except the App title/name.
Now, let’s talk about the good and bad that comes with A/B testing performed through Google Play. Firstly, Google Play enables the testing of all localizable assets, except the app’s title/name, and includes both textual and visual elements.
Unlike the Apple App Store, you can always upload new app icons to test them. Moreover, each time that you change the metadata of your android app, you won’t have to release a new app version. Not just this but developers can run default graphic tests for the app’s creative elements or the app’s localized experiments in up to 5 different languages.
Moreover, the split testing in Google Play allows for split testing of 1 default and 3 different variants. Check out these very insightful results that Tapps games have achieved with their store listing experiments:
However, one major downside in Google Play Store is that you can’t know what derived the increase in conversion or why any variant of the app’s A/B test won as it doesn’t showcase any on-page behavior insights. Hence, it is suggested to test one creative at a time to eliminate any biased results and effectively monitor the success rate. It is also suggested that you run tests for at least a week to eliminate any day-to-day effects.
A/B testing in both app stores comes with certain advantages and disadvantages. However, A/B testing in itself is a great practice and helps optimize your core app quality and enhance your discoverability. And since your app is being tested repeatedly with the right audiences, you are able to find the most effective graphics and text that work for each audience. Lastly, you are able to run product and marketing tets on a category of users before actually publishing those changes as live.
Hopefully, this article sheds a light on the basics of A/B testings for Google Play and iOS. If you want to see how major apps are succeeding in user acquisition, sign up for free and start exploring!
