A/B Test your App using Firebase Remote Config

Last week I was in Nairobi and Cape Town giving a talk on Remote Config and Test Lab in Firebase. I had such a great time and thought I should share some of the content I presented online.

What is A/B Testing?

A/B testing is the process of experimentally testing your UI on different audiences in order to determine the best (or most profitable) user experience.  Firebase Remote Config enables us to do A/B testing by allowing us to randomly segment our app audiences easily.

Example A/B Test

In this example, we are going to test out a different version of a checkout button that we have in our app.

A/B test we will run using Firebase Remote config.

For variant A of the experiment, we want to show a pink “Checkout” button. For variant B we want to show a blue “Cart” button. For the rest of our users, we will show the original “Finished” button.

Now for the fun part, setting up the experiment![adwords_square]

Set up Firebase Remote Config for A/B Testing

  1. Create a Firebase Project.
  2. Head to Remote Config section.
  3. Create a parameter – let’s call this parameter experiment_variant. Make the default value return no_experiment.Setting up parameter in Firebase Remote Config
  4. Click “Add value for Condition”. Then click “Define a new Condition”. Give it the name “Variant A”. We will select “User in Random Percentile” as the conditional. We will then choose the users from 0% – 10% that will receive variant A as their experiment. Click “Create Condition“.Adding a A/B Testing conditional to use with Firebase Remote Config
  5.  Now you will see that conditional appear for the experiment_variant parameter. Assign  variant_a to the value that should be returned for that variant.Conditional in Firebase Remote Config
  6. We will repeat the same for variant B, by creating a conditional for the users from 10% – 20%. This means that only 20% of the entire user base will be running the experiment and 80% will see no change in the standard UI. The percentage of the experiment is totally up to you. Variant B for Firebase Remote ConfigThen return variant_b for the parameter value:All variants in Firebase Remote Config
  7. Make sure to update and publish the changes. We have completed the server side setup of our experiment.

Setting up the A/B Test on Android Devices

Once you have configured the server, the hard part is done! The next section will cover how to get setup for Android but the code is really similar for iOS too – so don’t be deterred!

  1. In your android app, make sure you have the Firebase dependencies set up. Place the following in your top level build.gradle:
  2. Add the following in your app level build.gradle:
  3. Download google-services.json file from Firebase console and add it to your app/ folder.
  4. Create a remote config defaults file in the res/xml folder like the one below. This file will be used if the user has no internet connection when loading up the app for the first time.
  5. Initialise Firebase remote config. This is also where we set the defaults to the file we created in the previous step.
  6. Then fetch the remote config values. Firebase will automatically cache the values for the specified time that you provide. By default, it caches for 12 hours.
  7. In order to run our experiment, we should fetch the experiment_variant parameter from our remote config and use it in our app.

    In this example, we fetch the parameter and set a custom analytics user property with the variant that has been assigned to the current device. This will allow us to properly track how our A/B test is performing.  In the conditional, we are free to make the changes we want to the app. We can set the text to different things and change the colours depending on the variant.
  8. Now that we have the experiment running, we need to track when a user clicks on the “Checkout” button in order to determine which button variant a user is more likely to tap on. We then log the event to Firebase analytics which will include the user property that we set previously.

    It is worth noting that there are a bunch of default events and parameters that you can make use of with Firebase Analytics. [adwords_square]
  9. In order to determine the success of your A/B test, you need to analyse your data to decide which UI is the most successful. After logging into the console, we can see there were 24 counts of the “CheckoutClicked” event.firebase_analytics_ab_testingAfter applying the filter for the user property “Experiment = Variant A”, we can see that 19 of the 24 clicks were from variant A of the experiment (granted this was just me clicking around in my sample app – but you get the idea).

Variant A filters - Remote config

For larger data sets where the result of the experiment may not be obvious at first, you can export your events to BigQuery. This will allow you to make more complex deductions from your experiments.

For now at least, we know the clear winner (in this very large test 😬) is Variant A – the pink checkout button wins!

AB Testing variant A winner

There you have it, an easy way to experiment with your UI using Firebase Remote Config.

Sample code can be found here (It is not very complex as all the hard work is done by the Firebase APIs.) 😊

Happy AB Testing!


Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

ConstraintLayout 101 & the new Layout Builder in Android Studio

I was lucky enough to attend Google I/O and I went to the talk about the new Layout – ConstraintLayout . I was really excited about this feature that I couldn’t wait till I got home to try it out.

What is ConstraintLayout?

ConstraintLayout  is a new type of layout that you can use in your Android App, it is compatible down to API level 9 and is part of the support library. Its aim is to reduce layout hierarchies (and improve performance of layouts) and it also reduces the complexity of trying to work with RelativeLayouts .  It is compatible with other layouts, so they can live in harmony together without having to choose one or the other.

Getting Started with ConstraintLayout:

You need to download Android Studio 2.2 Preview 3 to try it out.  With this new layout comes a new layout editor (which I am told was designed from the ground up, *huge sigh of relief* as the old one was really unusable).

To get started using ConstraintLayout , you can add the following to your build.gradle  dependencies (or Android Studio will automatically add the  ConstraintLayout dependency to your build.gradle when you create a new layout using it)

To use the new ConstraintLayout , click on res/layout  folder. Select New > Layout Resource Folder. In the root element option, remove LinearLayout and start typing ConstraintLayout. .  You will then create a new file with your ConstraintLayout , the default view includes the Blueprint mode Blueprint Mode and Design Mode Design Mode  which will look something like this:

ConstraintLayout

How does this new layout builder work?

Constraints:

Constraints are rules that you define for a widget, much like RelativeLayouts  “toLeftOf , toRightOf”, constraints offer similar options. They can also specify how widgets are aligned.
[adwords_square]

Types of Constraints:

  • Resize Handle Resize Handle -Constraint Layout Resize the widget by dragging its corners with the square icon. Resize Handle - Constraint Layout
  • Size Constraint Handle Size Constraint Handle– To specify the location of the widget, they are round circles on the side of each widget.  Constraints
  • Baseline Constraint Handle – Allows you to align two baselines of text in your widgets. Baseline ConstraintLayout
  • Bias– Specifies which direction you want your widget to have bias towards. You can specify vertical or horizontal bias.Bias - ConstaintLayout

Widget Sizing

In the side properties view, you will see an image depicting the constraints of the view size itself. Here you will be able to specify margins and decide how you want the the widget to resize, such as:

  • Any Size ConstraintLayout - Any Size – Similar to match_parent  except it obeys constraints – effectively 0dp.
  • Wrap Content Wrap Content - ConstraintLayout– The widget will only be as big as it needs to be for all the content inside.
  • Fixed Size  Fixed Size - Constraint Layout – Specifying a size of the widget in dp 

To toggle between these, simply click on the current option in the properties window and it will toggle between all of them.View Properties - ConstraintLayout

You will also be able to specify the bias by dragging the slider on the horizontal axis or vertical, depending on the bias you want (as shown above in the bias section).

Autoconnect

Autoconnect is a feature that automatically as you place a new widget in the layout, will start to calculate the best constraints to give a widget from where you have placed it.

As soon as you drag a widget into the layout, you will see automatic lines starting to draw from the sides of the widget, that is Autoconnect doing its job. This feature is enabled by default, and it can be toggled on and off, by clicking the magnet icon. Autoconnect ConstraintLayout

autoconnect - constraintlayout

Manual Constraints

To create your own constraints, click a circle on the side of the widget you wish to position. Drag the line towards the other widget you want to align it to, once the widget turns green, it will create the constraint. You can delete a constraint by clicking on it when it is highlighted red around it.

Manual constraints - ConstraintLayout

Inference

Inference creates constraints between all different widgets in your layout, it acts like Autoconnect, but instead it works for the entire layout and not just the widget being edited. It uses crazy mathematics to determine what widgets it should constrain to others, based on where you have placed it on the screen.

To use inference on a whole layout, click the lightbulb icon to see it work. Inference - ConstraintLayout

inference - ConstraintLayout

Other things to note:

  • The new layout editor does not depend on you using a ConstraintLayout , it works for older layouts too – some of the features like automatic inference won’t work.
  • Using this layout editor generates XML for you, so if you want to be hardcore and write the XML yourself, you are welcome to.
  • You can use an array of other layout properties, such as:
    • Setting default margin
    • Align a group of widgets right, centre, equal distribution just like a design program would offer you.

But what about my old layouts? Should I migrate them all?

I wouldn’t jump on the train of changing everything at once if it’s not necessary. If you find your layouts are slow or you are battling to make changes then perhaps you should consider changing it to use ConstraintLayout . It is still in early alpha.

An advertised feature of the new Android Studio is that you can automatically convert old layouts to new ones using the ConstraintLayout . When you navigate to your layout file and are in the design view, you can right click > “Convert …Layout to ConstraintLayout”.

I am not sure if I was doing something wrong but for all three times I tried, the ConstraintLayout  was added to the view but results were not as expected. I suspect this feature is still going to take a while before it works for every use case –  it is still in alpha after all 😊.

My thoughts:

I am excited about this new layout editor and ConstraintLayout , I really think we needed something like this in Android. After playing around a bit it seems relatively simple to understand and quite powerful.

I hope that this could be turned into a standalone design tool that designers could ultimately use to create screens.

What are your thoughts? Have you tried out ConstraintLayout?
[adwords_square]
Links:

https://codelabs.developers.google.com/codelabs/constraint-layout/index.html

http://tools.android.com/tech-docs/layout-editor

Android Layouts – A new world – YouTube video from Google I/O


If you enjoyed this blog post – please consider buying me a cupcake!
[buy_cupcake]

Optimizing Layouts in Android – Reducing Overdraw

You have a great idea and you have launched your application into the wild. Now you hear people complaining how your app is slow and horrible to use. Sad face.

One such step to improve the rendering time of your application is to have a look at the GPU Overdraw tool.

What is Overdraw?

Overdraw happens every time the application asks the system to draw something on top of something else. The “Debug GPU Overdraw” tool overlays colours on top of your screen to indicate how many times a pixel has been redrawn.
[adwords_square]

How do I enable the tool Debug GPU Overdraw?

  1. Go to Settings on your device.
  2. Go to Developer Options
  3. Select “Debug GPU Overdraw”.
  4. Select “Show overdraw areas”

You will notice your screen change colour – don’t panic. Navigate to your application and now we can begin to understand how to improve our layouts.

What do the different colours mean?

Screenshot_2016-02-01-11-08-40The colours mean the following things:

Original colourno overdraw – The pixels have been painted once on the screen.

Blue1x Overdraw –  The pixels have been painted 2 times on the screen.

Green –  2x Overdraw – The pixels on the screen have been painted 3 times on the screen.

Pink3x Overdraw – The pixels on the screen have been painted 4 times on the screen.

Red – 4x Overdraw – The pixels on the screen have been painted 5 times on the screen.

GPU Overdraw

You can see from my Book Dash application, that my initial layout was doing a lot of overdraw. 🙁

[adwords_square]

How do you fix overdraw?

In the example above, I removed the background colour that was set on the RelativeLayout and let the theme draw the background.

So going from this:

to this:

Modifying the layout yielded the following result 😊:

After removing the background colour.As you can see, the overdraw has been minimized. The red overdraw has been reduced.

This could be further improved to the layout showing mostly its true colour and a slight amount of blue overdraw. Some overdraw is inevitable.

Not all cases of overdraw are related to background colours. Other problems can be present too, like very complex layout hierarchies or too many views.

You should aim for a maximum overdraw of 2x (Green).

You can also use other tools to see why overdraw is happening, tools such as Hierarchy Viewer and GL Tracer.

How do you debug overdraw issues? Do you have any other tips to share?

Also, you should totally Download the Book Dash App or contribute on Github!

Links:

http://developer.android.com/tools/performance/debug-gpu-overdraw/index.html
Get it on Google Play