This article will help in analysing the performance of the app

By Sahil G — “Android Ninja”


Want to measure and improve the app loading time when the user launches the app the first time but doesn’t know where to start? or have you optimized the performance and wondered if there’s space for improvement

Then you can benchmark your app…

In this article, we will take a look at how to measure the app performance by using Jetpack Macrobenchmark Library and improve it by approx. 30% by using the Baseline profile Library.

Inspecting Performance

This is to understand what is happening in an app and ensure it meets your expectations.

We have several tools to inspect the app’s performance but when getting started it’s recommended by Google to focus on only one area at a time.

These areas can include:

Manual Inspections

For manual inspection, we can use multiple tools like system tracing which is one of them. In this blog, we will not be covering this topic. For more details visit this link.

Automated Inspections

In this, we write test cases that we can run to collect and aggregate performance data. This helps you understand what users are actually seeing and identify when regressions may occur. To set up automated test cases we need to use benchmarks. Let’s move to the next step using macrobenchmark.

What is Macrobenchmark

Macrobenchmark is a jetpack library that provides us with an automated way to inspect performance by running tests and covers cases like app startup time, UI manipulation(i.e. Scrolling Events), Cross-Activity navigation, and animations. The library provides the performance results in the Android studio itself and generates the reports in JSON files as well.

With Jetpack Macrobenchmark you can:

Instrumentation using this library doesn’t call your application code directly but instead navigates your application as a user would — touching, clicking, swiping, etc. The measurement happens on the device during these interactions. If you want to measure parts of your application code directly, see Jetpack Microbenchmark instead.

Writing a benchmark is like writing an instrumented test, except you don’t need to verify the state your app is in. The benchmarks use the JUnit syntax (@RunWith, @Rule, @Test, etc.), but the tests run in a separate process to allow restarting or pre-compiling of your app. This allows us to run your app without interfering with its internal states, just as a user would. We do this by using UiAutomator to interact with the target application.

Enough theory…Let’s get started on the coding part!!

Add benchmark to your project

Add a benchmark module to an existing project by following the steps otherwise, create a new project first.

The recommendation is to use the latest version of the android studio as these are new features in that version of the IDE that integrate with Macrobenchmark.

To create a new module, do the following:

  1. Right-click your project or module in the Project panel in Android Studio and click New > Module.
  2. Select Benchmark from the Templates pane.
  3. You can customize the target application (the app to be benchmarked), as well as the package and module name for the new macrobenchmark module.
  4. Click Finish.

Minimum SDK — at least Android 6 (API level 23) or newer is required

Changes made after Module Integration

  1. It adds the gradle module by name of macrobenchmark (or the name which you gave in the module name while creating the module) and it uses the plugin which tells app grade not to include it in your application and it is only for testing purposes.

2. Also in the app gradle file the new build type is added by name of the benchmark.

InitWith — This is to keep our macrobenchmark module as close as possible to release build type so it provides us the accurate performance result

Signing Config — We are using the debug signing config so we don’t need to add the release keystore.

Matching Fallbacks — This is to specify the app gradle should try to use if the same matching build type is not present in the module gradle(or dependent module). For more info visit this link

Debuggable — This flag should be false for macrobenchmark otherwise it adds a lot of performance overhead and makes the result inaccurate.

3. As the debuggable option is false it adds the profilable tag in the manifest which allows the benchmark to profile your app with release performance.
In this, we require a minimum API level of 29 and prior to profilable we only profile the debuggable app which leads to performance overhead. For more detail

Measure App Startup

AppStartup time or the time taken by the app to load and become usable for the users. In macrobenchmark module creates a test class by the name ExampleStartupBenchmark which provides us with the metric of app startup time taken by the app.

What do all of the parameters mean?

When writing a benchmark, your entry point is the measureRepeated function of the MacrobenchmarkRule. This function takes care of everything for your benchmarks, but you need to specify these parameters:

Write Benchmark Test Cases

Macrobenchmark will reinstall and restart your app. Macrobenchmark provides several useful functions and parameters to interact with your app.

StartActivityAndWait() — This is the main function that starts your default activity and waits until it renders the first frame before continuing with the instructions in your benchmark.

pressHome() — This allows you to reset the benchmark to a base condition in cases where you don’t kill your app in each iteration (for example. when you use StartupMode.HOT).

Run Benchmark Test Case

To run the test case first follow the below steps:
1. Go to the build variants (Generally at the bottom left).
2. Select the active variant of the app to benchmark
3. Wait for gradle sync

Now run the benchmark test by clicking on the run button as shown in the screenshot

Make sure you are testing it on the physical device because running it on an emulator means measuring your host machine’s performance if that is under heavy load, your benchmarks will perform slower and vice versa and it will fail at runtime with a warning that it will give incorrect results.

Once you run the benchmark, your app will be rebuilt and then it will run your benchmarks. The benchmarks will start, stop and even reinstall your app several times based on the iterations you defined.

Run on Emulator

If you have a physical device then you can skip this step otherwise to run it on the emulator we need to follow the below steps.

There are two ways to run it on the emulator

First Way

  1. Add the testInstrumentationRunnerArguments line in macrobenchmark gradle file.
  2. Sync gradle.
  3. Now select the emulator and run on it.

Second Way

  1. Select “Edit Configurations…” from the run menu:

2. From the opened window, select the “options” icon next to “Instrumentation arguments”

3. Add the Instrumentation extra param by clicking ➕ and typing the details

4. Confirm the choice by clicking OK. You should see the argument in the “Instrumentation arguments” row

5. Click Apply and now run that configuration.

Understand the startup results

Run the test case on Samsung Galaxy M30 and get the minimum value of 366.7ms, a median was 395.8ms and max was 641.6ms.

Note — Every time you will get different results even after running the same benchmark on the same device results depends upon the following factors:

Because of that, it’s important to compare results on the same device, ideally being in the same state, otherwise, you can see great differences. If you can’t guarantee the same state, you may want to increase the number of iterations to properly deal with the result outliers.

Optional — To check when the app is ready to use

In the previous test we got the result only for the initial time taken by the app to load but when you also want to identify the time taken to draw the first frame and become usable for the users(This is called time to full display tfd) then we need to add an extra line of code.

Add Activity.reportFullyDrawn() function to lifecycle methods of your activity or fragment when we are expecting that UI is fully visible to the user like in onCreate /onViewCreated() then run the benchmark test again.

In my case, there is no difference between TID and TFD because this is the dummy project created for this blog with only a single activity class and significantly fewer UI components. Still, for sure in a project having more UI components, we can notice the difference quickly.

The Macrobenchmark library automatically collects the timing taken by the first frame to be fully drawn.

For more info

Frame Timing Metric

Previously we used startupTimingMetric which we could use to detect the app’s initial loading time and now if we want to identify any drops in frames by the app or in simple terms how smooth the app is.. in that case, we can use FrameTimingMetric.

Example — Let’s say you only want to identify the frames drops while scrolling the recycler view before you don’t want to measure anything then we need to add that code in setupBlock and the code whose performance you want to measure put that in measured Block.

In this test case, we are checking that the recycler view having id rvList is inflated and after that, we are setting the gesture margin because the app could trigger the system navigation and leave your app instead of scrolling the content.

Now run the test case and now you will get the results as in Snapshot

Frame Duration CPU MS — It means how much time the frame took to be produced on the CPU — on both the UI Thread and RenderThread.

Frame Overrun Ms — How much time a given frame missed its deadline by. Positive numbers indicate a dropped frame and visible jank/stutter, negative numbers indicate how much faster than the deadline a frame was. Available only on Android 12 (API level 31) and higher.

The results are in percentile and which means that the median value(P50 the 50%of frames) loads in 6.3 ms which is 10.7 ms under the frame time limit and if the frame overrun value is positive then we need to check the systrace to identify where the problem is on clicking on iterations(0,1,2,3 etc) because there may some frames skipped which will be noticeable to the user.

We can use other metrics based on our requirements please go through the below link for more info

What’s Next

In the upcoming blog, we will learn about how to improve app performance by using baseline profiles.

In this blog, we learned about performance inspection in two ways manual and automation. For automation, we are using macrobenchmark and to make it automated for analyzing the performance we are writing test cases in which we can use multiple metrics based on requirements and gets the results accordingly.
Hope this blog will help you to learn the automated way to analyze the performance which we improve by using baseline profile which helps us to make the app smoother.
I hope you enjoyed this tutorial. If you have any questions or feedback, please feel free to share in the comments.


Meet The Team!!


Sahil G

Reviewd By

Praveen Apk


Seema Jain

We at CaratLane are solving some of the most intriguing challenges to make our mark in the relatively uncharted omnichannel jewellery industry. If you are interested in tackling such obstacles, feel free to drop your updated resume/CV to!