How to Stage Rollout Features using Firebase Remote Config (Android & iOS)

The Google Play Store has a great feature called “Staged Rollout“. It allows you to slowly release your app to a percentage of your user base. This feature is not available on the iTunes App Store. Using Firebase Remote Config, we can do much more than just stage rollout an entire app – we can stage rollout individual features within an iOS or Android app. 

By using the Google Play Store staged rollout feature, we can see how well a new version of our app is performing in the field without impacting our entire user base.

This is how it works: When uploading an APK on the Google Play Console, selecting the dropdown caret next to the giant Publish Now button allows publishing your app as a staged rollout. You can then choose a percentage of your user base that will receive the update. Random users will be chosen from this percentage and will have the update available to them for download.Staged Rollout in Google Play Store

The staged rollout feature on the Google Play Store has helped my team detect issues that weren’t found during the testing or development phase.

After using the staged rollout feature for a while, I assumed this was a standard feature and when asking iOS developers, it became clear to me that this isn’t available on iTunes.

When I was using Firebase Remote Config, I found something extremely handy. By leveraging the User in Random Percentile conditional, you can easily achieve a similar type of staged rollout in your iOS and Android apps without relying on the app stores to do it for you.

In this blog post, we will cover how to implement a staged rollout feature in your apps, focusing on the Android implementation (but iOS is very similar). If you are interested, check out my previous post on “A/B Testing your app using Firebase Remote Config“.[adwords_square]

Create feature flag and staged rollout conditional

  1. To set up your Firebase Project – follow the instructions here for Android and here for iOS.
  2. Navigate to Remote Config on the Firebase Dashboard. Create a conditional and use the “User in Random Percentile” to target 0 – 10% of your users. Name this conditional relative to the  parameter that you are going to define next.Staged Rollout percentage in Firebase Conditional
  3.  Create a new parameter called “feature_add_friends“, set the default value to false and the value for the above created conditional to true. Staged Rollout conditional

Use the feature flag

Within your app, you can now consume the parameter as you would any other variable from Firebase Remote Config and the server will decide if you should see the feature or not.

The part you should be interested in, is line 38 – 45, where we fetch the parameter and use it in the app. [adwords_square]

Rollout to more users

Once your feature has been released onto the app stores and users start to download it, you will be able to gauge how well it is performing. When you gain more confidence with your feature, you can slowly increase the percentage to more users.Increase staged rollout percentage.

When you are happy for all your users to use the new feature, you can simply remove the conditional altogether.

That is all you need to do in order to stage rollout a feature using Firebase Remote Config! Let me know what you think on Twitter – @riggaroo.


If you enjoyed this blog post, please consider buying me a cupcake!

[buy_cupcake]


 

A/B Test your App using Firebase Remote Config

Last week I was in Nairobi and Cape Town giving a talk on Remote Config and Test Lab in Firebase. I had such a great time and thought I should share some of the content I presented online.

What is A/B Testing?

A/B testing is the process of experimentally testing your UI on different audiences in order to determine the best (or most profitable) user experience.  Firebase Remote Config enables us to do A/B testing by allowing us to randomly segment our app audiences easily.

Example A/B Test

In this example, we are going to test out a different version of a checkout button that we have in our app.

A/B test we will run using Firebase Remote config.

For variant A of the experiment, we want to show a pink “Checkout” button. For variant B we want to show a blue “Cart” button. For the rest of our users, we will show the original “Finished” button.

Now for the fun part, setting up the experiment![adwords_square]

Set up Firebase Remote Config for A/B Testing

  1. Create a Firebase Project.
  2. Head to Remote Config section.
  3. Create a parameter – let’s call this parameter experiment_variant. Make the default value return no_experiment.Setting up parameter in Firebase Remote Config
  4. Click “Add value for Condition”. Then click “Define a new Condition”. Give it the name “Variant A”. We will select “User in Random Percentile” as the conditional. We will then choose the users from 0% – 10% that will receive variant A as their experiment. Click “Create Condition“.Adding a A/B Testing conditional to use with Firebase Remote Config
  5.  Now you will see that conditional appear for the experiment_variant parameter. Assign  variant_a to the value that should be returned for that variant.Conditional in Firebase Remote Config
  6. We will repeat the same for variant B, by creating a conditional for the users from 10% – 20%. This means that only 20% of the entire user base will be running the experiment and 80% will see no change in the standard UI. The percentage of the experiment is totally up to you. Variant B for Firebase Remote ConfigThen return variant_b for the parameter value:All variants in Firebase Remote Config
  7. Make sure to update and publish the changes. We have completed the server side setup of our experiment.

Setting up the A/B Test on Android Devices

Once you have configured the server, the hard part is done! The next section will cover how to get setup for Android but the code is really similar for iOS too – so don’t be deterred!

  1. In your android app, make sure you have the Firebase dependencies set up. Place the following in your top level build.gradle:
  2. Add the following in your app level build.gradle:
  3. Download google-services.json file from Firebase console and add it to your app/ folder.
  4. Create a remote config defaults file in the res/xml folder like the one below. This file will be used if the user has no internet connection when loading up the app for the first time.
  5. Initialise Firebase remote config. This is also where we set the defaults to the file we created in the previous step.
  6. Then fetch the remote config values. Firebase will automatically cache the values for the specified time that you provide. By default, it caches for 12 hours.
  7. In order to run our experiment, we should fetch the experiment_variant parameter from our remote config and use it in our app.

    In this example, we fetch the parameter and set a custom analytics user property with the variant that has been assigned to the current device. This will allow us to properly track how our A/B test is performing.  In the conditional, we are free to make the changes we want to the app. We can set the text to different things and change the colours depending on the variant.
  8. Now that we have the experiment running, we need to track when a user clicks on the “Checkout” button in order to determine which button variant a user is more likely to tap on. We then log the event to Firebase analytics which will include the user property that we set previously.

    It is worth noting that there are a bunch of default events and parameters that you can make use of with Firebase Analytics. [adwords_square]
  9. In order to determine the success of your A/B test, you need to analyse your data to decide which UI is the most successful. After logging into the console, we can see there were 24 counts of the “CheckoutClicked” event.firebase_analytics_ab_testingAfter applying the filter for the user property “Experiment = Variant A”, we can see that 19 of the 24 clicks were from variant A of the experiment (granted this was just me clicking around in my sample app – but you get the idea).

Variant A filters - Remote config

For larger data sets where the result of the experiment may not be obvious at first, you can export your events to BigQuery. This will allow you to make more complex deductions from your experiments.

For now at least, we know the clear winner (in this very large test 😬) is Variant A – the pink checkout button wins!

AB Testing variant A winner

There you have it, an easy way to experiment with your UI using Firebase Remote Config.

Sample code can be found here (It is not very complex as all the hard work is done by the Firebase APIs.) 😊

Happy AB Testing!


Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Google Developer Launchpad Build SSA – Nairobi and Cape Town Events

I was lucky enough to be invited to speak in Nairobi and Cape Town this past week for the Google Developer Launchpad Build Series events.

The theme this year was Firebase. The event was a huge success and I had the best time! I gave a talk about Firebase Remote Config and Test Lab. Here are the slides from my talk:

The events were filled with great speakers and topics:

Slides from all the talks can be found here: https://bit.ly/lpb-slides [adwords_square]

Instead of posting photos and stuff from the event, I made a travel vlog. Check out my vlog below of all the fun we had!

See you soon Cape Town and Nairobi!

Introduction to Automated Android Testing – Part 6

In the previous 5 blog posts, we covered different aspects of building an Android app from scratch. We focused on including tests in the process. Here are the links to the previous posts:

  • Post #1 – Why should we write tests?
  • Post #2 – Set up your app for testing
  • Post #3 – Creating API calls
  • Post #4 – Creating repositories
  • Post #5 – Following the MVP pattern

In this last post of the series, we will cover creating Espresso tests for the View we created in part 5. The Github repo for this post can be found here.

Testing that a view contains the exact information expected can be tricky if the data is dynamic. This data can change at any time and our tests should not fail because of it. In order for the tests to be reliable and repeatable, we should not call any production APIs.

Mocking out the responses of the API calls will enable us to write tests that depend on the mocked data. There are a couple of ways in which we can mock out our API calls:

  • Option 1 – Use WireMock and run a standalone server which serves up the same static JSON for specific network calls.
  • Option 2 – Use OkHttp’s MockWebServer which runs a webserver on your device and serves up any response you request.
  • Option 3 – Create a custom implementation of the Retrofit REST interface that returns dummy objects.

Obviously, the choice is entirely up to you as to how you would want to go about writing UI tests. In my case, WireMock is extra effort as I would need to ensure I have a standalone server running with a static IP address.

MockWebServer is a lot easier to use than WireMock as you don’t have to set up a standalone web server (The server runs on the device). MockWebServer is also flexible because you can give it different scenarios. Useful features like specifying the failure rate of a certain call or simulating a slow network are possible using MockWebServer. (Read more here).

I am going to use option 3 for the purpose of testing that the UI matches the mock response data. If I wanted to add tests for slow network conditions (or some kind of non-functional test), I would choose option 2. If you are unable to use OkHttp,  I would choose option 1 as Wiremock works with any HTTP client.

Mocking out data using Gradle flavors

By making use of Gradle flavors, we can easily mock out of API responses.  If you read post #2 on Gradle flavors, you should already have a “mock” and a “production” flavor set up.

  1. Make sure you are switched to the mockDebug flavor. Select mockDebug variant
  2. Create a mock folder in the src directory. Then create a package within the mock folder, that mimics the main package name. Make a class called MockGithubUserRestServiceImpl. Your resulting file structure should look like this: Folder structure for mock testing
  3. Create a prod directory. Move the Injection class defined previously into this folder. We will be creating another Injection class in the mock folder. This class will inject the mocked out Github service instead of the production API. Move Injection to prod and create one in mock folderIn the Injection class that is located in the mock folder, we simply return the MockGithubUserServiceImpl that we created. In the prod folder, we return the actual Retrofit Github service. [adwords_square]
    Mock Injection class:

    Prod Injection class:
  4. The data returned from the mock service is dependant on your specific requirements. Below is my implementation of the MockGithubUserRestServiceImpl:

    In this case, I am just returning some dummy data. Let’s run the mock version of the app and we should get the same results no matter what you search.

    Android Test app with mock data

    Cool. Now we have a working dummy app! We can now write Espresso UI tests. [adwords_square]

Basics of Writing an Espresso Test

When writing an Espresso test, the following formula is used for performing functions within your UI:

  • ViewMatchers – Used to find a view in an activity. There are a bunch of different kinds of matchers. For example: withId(R.id.menu_search), withText("Search"), withTag("custom_tag") .
  • ViewActions – Used to interact with a view. For example: click(), doubleClick(), swipeUp(), typeText().
  • ViewAssertations – Used to make assertions that certain views possess specific properties. For example:   doesNotExist(), isAbove(), isBelow().

There is a great cheat sheet for the different Espresso methods that can be found in pdf form here: android-espresso-testing.pdf. It is worth mentioning that ordinary hamcrest matchers can be used when writing Espresso tests. Methods such as not(), allOf()  and anyOf() are valid.

Writing Espresso UI Tests

If you can recall, in post #2 we covered what dependencies will need to be added in order to write espresso tests. Now we will cover writing an Espresso test.

  1. Create a folder androidTestMock. The tests in this folder will only run on the mock variant and not on the production variant. Then make a directory that matches the main package name. In that directory, add a new class called  UserSearchActivityTest. Your project should then look like this:androidTestMock folder
  2. We will start by writing a basic test that ensures that when the activity is started, the text “Start typing to search” is displayed:

    The @Rule  ActivityTestRule specifies which activity this test will run with. In this case this test will run with UserSearchActivity. This will automatically start up the UserSearchActivity. Passing extra parameters will indicate if you want the activity to auto start or not.

    The test searchActivity_onLaunch_HintTextDisplayed() is quite simple. It searches in the view for the text and asserts that the text is visible on the UI.

  3. The next test is slightly more complicated:

    After typing into the SearchView and pressing enter, we assert that the dummy results are displayed on the UI.

  4. We have now written tests for the positive scenarios, we should add a test for the negative case too. We will need to adjust the MockGithubUserRestServiceImpl in order to allow it to return custom error observables if required.

    In the code above, a method was created in order to set a dummy observable for the search results. That observable will be returned if it is not null when searchGithubUsers() is called.

  5. Now we can create a test that checks if the error is displayed on the UI.

    In this test, we first ensure that the service will return an exception. Then we assert that the error message is displayed on the UI.

  6. Let’s run the tests:

Passing_UI_Tests

They all pass!

Code Coverage in Android

In order to know how effective your tests are, it is great to get code coverage metrics.

  1. To enable code coverage on your UI tests, add testCoverageEnabled = true  to your build.gradle:
  2. Run the task createMockDebugCoverageReport. You will then find the HTML report located here: app/build/reports/coverage/mock/debug/index.html.

Code Coverage Report in Android

Yay – we have 82% coverage just with the mock UI test. Taking into consideration the coverage reports we saw in post #4 and this post, it gives us a good indication of the test coverage of our entire app. Now we can iteratively go back and try cover more areas of our code.

PS – Code coverage currently doesn’t work with the Jack compiler. I switched to use Retrolambda in order to get the code coverage report to work. If you are interested in learning more, check out this branch.

Conclusion

We have finished writing our feature. Whew! 6 blog posts later. There is obviously a lot more testing that can be completed in this app. Non-functional tests such as testing how your app behaves on devices with low memory or with poor network connectivity can also be added.
That concludes the series on an “Introduction to Automated Android Testing”. I hope you enjoyed reading this series. If you enjoyed it, be sure to subscribe to the blog to receive future updates and share the post with your friends.

Further Reading

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.


If you would like to see more posts from me, please consider buying me a cupcake to keep them coming.

[buy_cupcake]


Introduction to Automated Android Testing – Part 5

In this series of blog posts, we are working through a sample app called Github User Search. Parts 1 – 4 covered why we should test, getting set up with testing, creating API calls and creating a presenter. Take a look at the previous posts as part 5 is a continuation of the series.

In part 5, we will take a look at interacting with the Presenter  created in part 4 and we will create the UI to display the list of search results.

Creating the UI

For the User Interface we want a simple list that displays the avatar, name and other user information in the list.

In part 4, we defined a View contract which the Activity should implement. This is where the Android specific code will be located (things such as visibility changes or any UI changes will be located here). To refresh your memory, this is the View contract we created in this last post:

Let’s implement the View! [adwords_square]

  1. Create a class called  UserSearchActivity. This class will implement the UserSearchContract.View contract and extend AppCompatActivity. Define a variable called userSearchPresenter of type UserSearchContract.Presenter. This is the object that we will interact with in order to perform our network calls.

    In onCreate(), create the presenter object. Provide it with the User repo defined in the Injection class. Pass the io() scheduler and the AndroidSchedulers.mainThread() scheduler so that the RxJava subscriptions know which threads they should perform their work on.

    On the next line, you can see I call userSearchPresenter.attachView(this) . This attaches the view to the presenter, so that the presenter can notify the view of any changes. Because the presenter isn’t aware of the activity’s lifecycle, in onDestroy() we need to inform the presenter that the view is no longer in existence, so we should then call userSearchPresenter.detachView(). This will unregister any RxJava subscriptions and prevent memory leaks from occurring.

  2. Create activity_user_search.xml in the layout folder. This will contain a RecyclerView, a ProgressBar, an error TextView and a Toolbar. I am using ConstraintLayout to design my screen, so I won’t go into too much detail as it is mostly drag and drop. (If you want to read more about ConstraintLayout check out my blog post about it here)UserSearchActivity

  3. We also need to add a SearchView to the toolbar so we have somewhere to type. Add a menu_user_search.xml file to the menu resource folder. Inside it, you will need to add a SearchView:

  4. We need to create a layout that will be used for each item in the RecyclerView. Create a file named list_item_user.xml in the layout folder. I used ConstraintLayout with one ImageView for the avatar and two TextViews.List_item_user_designmode

  5. Now that we have all the layouts we need, let’s tie the XML to the Activity. First, in onCreate() we will get references to the views we need.

  6. We need to hook the SearchView up into our activity to make it trigger the presenters search() method. In the onCreateOptionsMenu(), add the following code:

    This will inflate the correct menu, find the search view and set a query text listener. In this case, only when someone presses submit on the keyboard, we will respond by calling the search presenter with the query. We could do it in onQueryTextChange too but due to rate limiting on the Github API I will stick to onQueryTextSubmit. By default, the item will be expanded.[adwords_square]

  7. Next, we will implement the callbacks that the presenter will call when the items are finished loading.

    We are basically just toggling visibility of views here and setting the usersAdapter  to the new items that the service returned.

  8. For completeness, here is the UserSearchAdapter class which is used for the RecyclerView on the activity:

     

    Injection class

  9. Now if you run the app, you should be able to search for a username on Github and see results.github_user_search

Yay! We have a working app. Code for this post can be found here. In the next part, we will look at writing UI tests for the app. Make sure you subscribe so you don’t miss a post!

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.


If you enjoyed this blog post, please consider buying me a cupcake to support future blog posts.

[buy_cupcake]