Multi-Theme Screenshot Tests in Jetpack Compose

I have been working a lot with a design system, and one of the recent challenges was to add support to screenshot testing while making it easy to scale for several different themes. In my vision, we would be able only to write one single test, but automatically generate the screenshots for all the themes in one single run.

Meanwhile, I reached a solution that can be seen in this sample project created for this purpose. In this project, we have a simple design system and we can see an example of how to generate screenshot tests that will verify the system’s Typography in both Light and Dark theme.

Light	Dark

P.s - if you want to look into the code, you can look at this specific PR where I add the multi-theme screenshot support.

Let’s go step by step of what was the thought process into getting into the final solution:

1. Integrating with Shot#

To be able to have screenshot tests, we are integrating Shot as our screenshot testing tool.

Any test can have the ability to take screenshot by:

Extending shot.ScreenshotTest
Calling compareScreenhot(rule: ComposeTestRule).

This way we could easily have a screenshot test in this form:

class TypographyTest {
   
   @get:Rule
   val composeTestRule = createComposeRule()

   @Test
   fun paragraph() {
       composeTestRule.setContent {
          Theme {
              Text(
              	 text = "This is a paragraph.", 
                 style = Theme.typography.paragraph
              )
           }
       }
    
       compareScreenshot(rule = composeTestRule)
   }
}

We then have a screenshot generated in our module under app/screenshots/com.fabiocarballo.designsystem.TypographyTest.label

com fabiocarballo designsystem TypographyTest_com fabiocarballo designsystem TypographyTest_paragraph_light

2. Improving the API#

At this point, we know how to make a screenshot test. However, as we grow the number of tests we would be having a lot of structural duplication:

Extending ScreenshotTest
Declaring the ComposeTestRule
Setting the composable content (and not forgetting to wrap it under our Theme)
Comparing the results of the screenshot.

We can then extract this structure into a DesignSystemScreenshotTest:

abstract class DesignSystemScreenshotTest: ScreenshotTest {

   @get:Rule
   val composeTestRule = createComposeRule()

   fun runScreenshotTest(content: @Composable () -> Unit) {
         composeTestRule.setContent {
             Theme(content = content)
          }
          
          compareScreenshot(composeTestRule)
   }
}

You can see that all the structure was then passed into this abstract class that every test class should extend. Another point to note is that all content should be run under the scope of this runScreenshotTest. Let’s see how our TypographyTest would look like now:

class TypographyTest: DesignSystemScreenshotTest() {
    
    @Test
    fun paragraph() = runScreenshotTest {
       Text(
       	  text = "This is a paragraph.", 
       	  style = Theme.typography.paragraph
       )
    }

    @Test
    fun label() = runScreenshotTest {
        Text(
           text = "This is a label.", 
           style = Theme.typography.label
        )
    }

    @Test
    fun display() = runScreenshotTest {
        Text(
            text = "This is a display.", 
            style = Theme.typography.display
        )
    }
}

The goal was to make a test being almost as easy as just declaring the composable that should be screenshotted.

By now, our screenshots would be:

Label	Paragraph	Display

3. Add support to multi-theme#

At this point, we now can quickly generate screenshots for our Light Theme. As a next step, we want to use the same codebase and automatically generate screenshots also to Dark Theme.

For that, we are going to enrich our DesignSystemScreenshotTest with the capability to run parameterized tests. Hence, we are using Test Parameter Injector to build the parameterized behavior.

What we are going to do first is to declare a private enum with the themes we want to parameterize with:

private enum class ThemeMode { LIGHT, DARK }

Then we are going to declare it as a test parameter so that each test will run with both LIGHT and DARK. This is done by adding a field annotated with @TestParameter and by adding integrating with the TestParameterInjector test runner.

@RunWith(TestParameterInjector::class)
abstract class DesignSystemScreenshotTest : ScreenshotTest {

    @get:Rule
    val composeTestRule = createComposeRule()

    @TestParameter
    private lateinit var themeMode: ThemeMode

    fun runScreenshotTest(
        content: @Composable () -> Unit
    ) {

        composeTestRule.setContent {
            Theme(
                isSystemInDarkMode = themeMode == ThemeMode.DARK,
                content = content
            )
        }

        compareScreenshot(
            rule = composeTestRule,
        )
    }

    private enum class ThemeMode { LIGHT, DARK }
}

However, we still have one small problem: Shot default behavior is to name the screenshot as “ClassName_MethodName”. This way, we record the screenshots for the test in both modes, but the last run always overrides the first one.

To fix that, we are going to generate the screenshot name by ourselves with the help of the TestName rule from junit (thanks Ahmed Abdelmeged for the tip!!):

@get:Rule
val testNameRule = TestName()

We can then use this information together with the ThemeMode that is being used for the test to generate a test name as:

TypographyTest_paragraph_light
TypographyTest_paragraph_dark

Below you have the final version of the DesignSystemScreenshotTest:

@RunWith(TestParameterInjector::class)
abstract class DesignSystemScreenshotTest : ScreenshotTest {

    @get:Rule
    val composeTestRule = createComposeRule()

    @get:Rule
    val testNameRule = TestName()

    @TestParameter
    private lateinit var themeMode: ThemeMode

    fun runScreenshotTest(
        content: @Composable () -> Unit
    ) {

        composeTestRule.setContent {
            Theme(
                isSystemInDarkMode = themeMode == ThemeMode.DARK,
                content = content
            )
        }

        val name = "${testNameRule.methodName}_${themeMode.name.lowercase()}"

        compareScreenshot(
            rule = composeTestRule,
            name = name
        )
    }

    private enum class ThemeMode { LIGHT, DARK }
}

And that is it! When you run your screenshot tests you will generate both Light and Dark mode screenshots in one go. In our example, the generated screenshots are the following:

Label	Paragraph	Display