Adetunji Dahunsi

Animating ContentScale during image shared element transitions

A Jetpack Compose twist on an old favorite

TJ Dahunsi

Mar 21 2024 · 12 min read

Categories:

android

jetpack-compose

kotlin

I'm obsessed with animations in apps. They provide a sense of place, grant visual continuity and greatly improve the user experience. For details as to why, read this evergreen blog post about building smarter animations with motional intelligence.

Shared element transitions today

This sense of place and continuity is brought to the fore for navigation animations in apps with shared elements, especially with media. To simplify things, this blog will restrict media to just images. Shared elements in the Android View system rely on a smoke and mirrors effect to create the illusion that an element is moving from one destination to the next. It goes something like this:

Screen A passes the size and position of the shared element to screen B.
Screen B reviews the information passed to it and matches its own views to the shared elements.
Screen B then places the matching views where they used to be on Screen A.
Screen B finally animates them to their final position.

This generally works fine, however there are few things that bother me about this:

There are 2 views created that render the same information with minor differences, if any at all.
If the image fetched is asynchronous, and the image has had a transformation applied (different aspect ratio for example); you may have to use Fragment.postponeEnterTransition()to postpone the transition and wait till the image on screen B is loaded. After loading is complete, Fragment.startPostponedEnterTransition() may be used to resume the transition. This interruption can cause jank.
If the image is being loaded in a different aspect ratio (ContentScale), there's still a mild visual discontinuity as the size and position of the shared element is animated, but the ContentScale is not.

Can Jetpack Compose improve on this?

Thinking differently with Jetpack Compose

I LOVE Jetpack Compose. It requires a new mental model, but with it comes new, and often better ways to approach and solve existing problems. Specifically, with the problem defined above, there are a few neat APIs we can take advantage of to have the closest literal implementation of the concept of a shared element transition on any platform.

Note: If you're looking to get up and running with shared elements in compsose ASAP, Please follow the offical guide.

To do this, we'll go a little deeper and build upon the foundational APIs for shared elements in Compose that power shared elements. The official guide linked above offers high level APIs for shared elements:

SharedTransitionLayout: The outermost layout required to implement shared element transitions. It provides a SharedTransitionScope. Composables need to be in a SharedTransitionScope to use the shared element modifiers.
Modifier.sharedElement: The modifier that flags to the SharedTransitionScope the composable that should be matched with another composable.
Modifier.sharedBounds: The modifier that flags to the SharedTransitionScope that this composable's bounds should be used as the container bounds for where the transition should take place. In contrast to sharedElement(), sharedBounds() is designed for visually different content.

These work well for 90% of use cases, but it has the some limitations. This post is focused on ContentScale for images not being animated by default; it snaps to the set end ContentScale of the image and this can break immersion.

To implement this, we'll need to look at the lower level APIs powering the shared element transitions to create a custom Modifier that maintains immersion and preserved continuity for images even when the ContentScale changes. The end result is the following:

Note: Modifier.sharedElementWithCallerManagedVisibility was not used because the animation trigger with it is a size and position change, and a 1:1 shared element match for an exiting item and an entering item. In my use case this is not always the case. My trigger is always a navigation change and sometimes there is a straight substitution; and the exiting item is replaced with an empty box for the illusion of space being vacated. This can be seen with the predictive back gesture before the shared element falls into place. I may be using the API worng, but I was not able to get Modifier.sharedElementWithCallerManagedVisibility to work.

API overview

The general Compose APIs used in the gif above are:

TargetBasedAnimation via the animate method.
Side effect APIs like LaunchedEffect and produceState to precisely time when animations should be run.
moveableContentOf: To move UI in composition from one place to another.

The lower level shared element APIs are:

LookaheadScope: To perform lookahead measurements to guide moving elements to their destination. Notably, SharedTransitionScope extends LookaheadScope.
ApproachLayoutModifierNode: Implementation Modifier.Node used to guide elements that have moved to their destination after lookahead.
GraphicsLayer.record: Used to record drawing commands into a GraphicsLayer and render them elsewhere.

When assembled, the step by step breakdown is:

A single shared LookaheadScope at the root of the navigation hierarchy implemented with a SharedTransitionLayout.
Screen A to use a moveableContentOf Composable with an ApproachLayoutModifierNode attached to render its shared element.
Screen A to signal navigation to Screen B.
The navigation state to be read on both screens.
Screen A to immediately stop composing the moveable shared element.
Screen B to immediately start composing the shared element.
The animation APIs to gradually animate the size, position AND the ContentScale of the shared element to the destination.

Interpolating ContentScale

Interpolating ContentScale in Compose is as simple as using the linear interpolation lerp function. To do so, a Composable can be written that remembers and updates the current ContentScale, and updates the previous ContentScale whenever the current one changes.

When a change occurs, an animation is launched to drive the interpolation fraction from 0f to 1f. Then an anonymous ContentScale that reads this fraction is finally returned to act as the real ContentScale in the composition.

1@Composable
2private fun ContentScale.interpolate(): ContentScale {
3    var interpolation by remember {
4        mutableFloatStateOf(1f)
5    }
6    var previousScale by remember {
7        mutableStateOf(this)
8    }
9    val currentScale by remember {
10        mutableStateOf(this)
11    }.apply {
12        if (value != this@interpolate) previousScale = when {
13            interpolation == 1f -> value
14            else -> CapturedContentScale(
15                capturedInterpolation = interpolation,
16                previousScale = previousScale,
17                currentScale = value
18            )
19        }.also { interpolation = 0f }
20        value = this@interpolate
21    }
22
23    LaunchedEffect(currentScale) {
24        animate(
25            initialValue = 0f,
26            targetValue = 1f,
27            animationSpec = spring(
28                stiffness = Spring.StiffnessMedium
29            ),
30            block = { progress, _ ->
31                interpolation = progress
32            },
33        )
34    }
35
36    return remember {
37        object : ContentScale {
38            override fun computeScaleFactor(
39                srcSize: Size,
40                dstSize: Size
41            ): ScaleFactor {
42                val start = previousScale.computeScaleFactor(
43                    srcSize = srcSize,
44                    dstSize = dstSize
45                )
46                val stop = currentScale.computeScaleFactor(
47                    srcSize = srcSize,
48                    dstSize = dstSize
49                )
50                return if (start == stop) stop
51                else lerp(
52                    start = start,
53                    stop = stop,
54                    fraction = interpolation
55                )
56            }
57        }
58    }
59}

If for any reason the interpolation animation is interrupted before it completes, an intermediate ContentScale that captures the interpolation progress is created and set as the current ContentScale and the process continues as normal:

1private class CapturedContentScale(
2    private val capturedInterpolation: Float,
3    private val previousScale: ContentScale,
4    private val currentScale: ContentScale,
5    ) : ContentScale {
6
7    override fun computeScaleFactor(
8        srcSize: Size,
9        dstSize: Size
10    ): ScaleFactor = lerp(
11        start = previousScale.computeScaleFactor(
12            srcSize = srcSize,
13            dstSize = dstSize
14        ),
15        stop = currentScale.computeScaleFactor(
16            srcSize = srcSize,
17            dstSize = dstSize
18        ),
19        fraction = capturedInterpolation
20    )
21}

This can be put together for the demo screen below:

Now that ContentScale can be interpolated, the next step is actually sharing a Composable across navigation destinations while preserving its existing state. More succintly, I'd like to perform a Composable transplant.

The definition that follows is the Composable I want to move:

1data class ImageArgs(
2    val url: String?,
3    val description: String? = null,
4    val contentScale: ContentScale
5)
6
7@Composable
8fun Image(
9    ImageArgs: ImageArgs,
10    modifier: Modifier
11) {
12    Box(modifier) {
13        AsyncImage(
14            modifier = modifier,
15            model = ImageArgs.url,
16            contentDescription = null,
17            contentScale = ImageArgs.contentScale.interpolate()
18        )
19    }
20}

Note: The Image Composable takes a single argument because of the arity of moveableContentOf. While variants of it supports multiple arguments, restricting the signature to one lets me move arbitrary Composables with one moveableContentOf signature.

Next I create a custom scope at the root of my navigation hierarchy to store my shared elements:

1@Stable
2interface AdaptiveContentState {
3    val navigationState: Adaptive.NavigationState
4
5    val overlays: Collection<SharedElementOverlay>
6}

Where a SharedElementOverlay is the shared element itself:

1interface SharedElementOverlay {
2    fun ContentDrawScope.drawInOverlay()
3}

The actual implementation of AdaptiveContentState treats navigation as state, and stores the shared elements in a MutableStateMap.

1@Stable
2class SavedStateAdaptiveContentState @AssistedInject constructor(
3    ...
4    @Assisted saveableStateHolder: SaveableStateHolder,
5) : AdaptiveContentState,
6    SaveableStateHolder by saveableStateHolder {
7
8    override val overlays: Collection<SharedElementOverlay>
9        get() = keysToSharedElements.values
10
11    private val keysToSharedElements = mutableStateMapOf<Any, SharedElementData<*>>()
12
13    fun <T> createOrUpdateSharedElement(
14        key: Any,
15        sharedElement: @Composable (T, Modifier) -> Unit,
16    ): @Composable (T, Modifier) -> Unit {
17        val sharedElementData = keysToSharedElements.getOrPut(key) {
18            SharedElementData(
19                sharedElement = sharedElement,
20                onRemoved = { keysToSharedElements.remove(key) }
21            )
22        }
23        // Can't really guarantee that the caller will use the same key for the right type
24        return sharedElementData.moveableSharedElement
25    }
26}

The SavedStateAdaptiveContentState has a method for creating or updating shared elements that may be moved in between Composables by retaining the Composable in a SharedElementData class. This class is what hosts the moveableContentOf and keeps track of it's state:

1internal class SharedElementData<T>(
2    sharedElement: @Composable (T, Modifier) -> Unit,
3    onRemoved: () -> Unit
4) : SharedElementOverlay {
5
6    val moveableSharedElement: @Composable (Any?, Modifier) -> Unit =
7        movableContentOf { state, modifier ->
8            @Suppress("UNCHECKED_CAST")
9            sharedElement(
10                // The shared element composable will be created by the first screen and reused by
11                // subsequent screens. This updates the state from other screens so changes are seen.
12                state as T,
13                Modifier
14                    .movableSharedElement(
15                        sharedElementData = this,
16                    ) then modifier,
17            )
18            ...
19        }
20}

Note: Much like Modifier.sharedElement() or Modifier.sharedbounds(), the custom shared element Modifier (Modifier.moveableSharedElement) needs to be called before the actual Modifier supplied to the Composable.

Creating a custom shared element modifier

This is the fun part. So far, the system desgin allows for moving a Composable between screen level Composables, but not for animating it. To animate it, the approachLayout Modifier is used. Inside the SharedElementData state are DeferredTargetAnimation instances for animating the size ind position of the Composable that are self managed. When these animations are running, the Composable can be said to be performing a shared element transition. However this is not the only condition to be met, otherwise the Composables will animate when scrolled and not track with the rest of the UI. To solve this, the Composable watches the navigation state, and allows for animating each time the navigation state changes up until the first time the animations complete:

1private fun SharedElementData<*>.isInProgress(
2    animationMapper: (SharedElementData<*>) -> DeferredTargetAnimation<*, *>
3): Boolean {
4    val animation = remember { animationMapper(this) }
5    val containerState = LocalAdaptiveContentScope.current
6        ?.containerState
7        ?.also(::updateContainerStateSeen)
8
9    val (laggingScopeKey, animationInProgressTillFirstIdle) = produceState(
10        initialValue = Pair(
11            containerState?.key,
12            containerState.canAnimateOnStartingFrames()
13        ),
14        key1 = containerState
15    ) {
16        value = Pair(
17            containerState?.key,
18            containerState.canAnimateOnStartingFrames()
19        )
20        value = snapshotFlow { animation.isIdle }
21            .filter(true::equals)
22            .first()
23            .let { value.first to false }
24    }.value
25
26    return when(laggingScopeKey == containerState?.key) {
27        true -> animationInProgressTillFirstIdle && hasBeenShared()
28        false -> containerState.canAnimateOnStartingFrames()
29    }
30}

With this, the custom shared element modifier for moveable shared elements can be defined:

1internal fun Modifier.movableSharedElement(
2    sharedElementData: SharedElementData<*>,
3): Modifier = composed {
4    val sharedTransitionScope = LocalSharedTransitionScope.current
5    val coroutineScope = rememberCoroutineScope()
6
7    val sizeAnimInProgress = sharedElementData.isInProgress(
8        SharedElementData<*>::sizeAnimation
9    )
10        .also { sharedElementData.sizeAnimInProgress = it }
11
12    val offsetAnimInProgress = sharedElementData.isInProgress(
13        SharedElementData<*>::offsetAnimation
14    )
15        .also { sharedElementData.offsetAnimInProgress = it }
16
17    val layer = rememberGraphicsLayer().also {
18        sharedElementData.layer = it
19    }
20    approachLayout(
21        isMeasurementApproachInProgress = {
22            sharedElementData.sizeAnimation.updateTarget(
23                target = it,
24                coroutineScope = coroutineScope,
25                animationSpec = sizeSpec
26            )
27            sizeAnimInProgress
28        },
29        isPlacementApproachInProgress = {
30            val target = with(sharedTransitionScope) {
31                lookaheadScopeCoordinates.localLookaheadPositionOf(it)
32            }
33            sharedElementData.offsetAnimation.updateTarget(
34                target = target.round(),
35                coroutineScope = coroutineScope,
36                animationSpec = offsetSpec,
37            )
38            offsetAnimInProgress
39        },
40        approachMeasure = { measurable, _ ->
41            val (width, height) = sharedElementData.sizeAnimation.updateTarget(
42                target = lookaheadSize,
43                coroutineScope = coroutineScope,
44                animationSpec = sizeSpec
45            )
46            val animatedConstraints = Constraints.fixed(width, height)
47            val placeable = measurable.measure(animatedConstraints)
48
49            layout(placeable.width, placeable.height) layout@{
50                val currentCoordinates = coordinates ?: return@layout placeable.place(
51                    x = 0,
52                    y = 0
53                )
54                val targetOffset = with(sharedTransitionScope) {
55                    lookaheadScopeCoordinates.localLookaheadPositionOf(
56                        currentCoordinates
57                    )
58                }
59
60                sharedElementData.targetOffset = sharedElementData.offsetAnimation.updateTarget(
61                    target = targetOffset.round(),
62                    coroutineScope = coroutineScope,
63                    animationSpec = offsetSpec
64                )
65
66                val currentOffset = with(sharedTransitionScope) {
67                    lookaheadScopeCoordinates.localPositionOf(
68                        sourceCoordinates = currentCoordinates,
69                        relativeToSource = Offset.Zero
70                    ).round()
71                }
72
73                val (x, y) = sharedElementData.targetOffset - currentOffset
74                placeable.place(
75                    x = x,
76                    y = y
77                )
78            }
79        }
80    )
81        .drawWithContent {
82            layer.record {
83                this@drawWithContent.drawContent()
84            }
85            if (!sharedElementData.canDrawInOverlay) {
86                drawLayer(layer)
87            }
88        }
89}

Rendering in overlays

When the shared element transition is in progress, the shared content has to be rendered above other content so they're not occuluded. To do this, Modifier.drawWithContent is used along with GraphicsLayer.record() to preserve drawing instructions into the GraphicsLayer. If the shared element is not transitioning, its simply rendered in place:

1drawWithContent {
2    layer.record {
3        this@drawWithContent.drawContent()
4    }
5    if (!sharedElementData.canDrawInOverlay) {
6        drawLayer(layer)
7    }
8}

If it is transitioning however, it should be drawn at the topmost level in the hierarchy, that is where the SharedTransitionLayout is defined:

1@Composable
2fun AdaptiveContentRoot(
3    adaptiveContentState: AdaptiveContentState,
4    content: @Composable () -> Unit
5) {
6    SharedTransitionLayout(
7        modifier = Modifier.drawWithContent {
8            drawContent()
9            adaptiveContentState.overlays.forEach { overlay ->
10                with(overlay) {
11                    drawInOverlay()
12                }
13            }
14        }
15    ) {
16        CompositionLocalProvider(LocalSharedTransitionScope provides this) {
17            content()
18        }
19    }
20}

As previously mentioned, these overlays are the SharedElementData from before:

1@Stable
2internal class SharedElementData<T>(
3    sharedElement: @Composable (T, Modifier) -> Unit,
4    onRemoved: () -> Unit
5) : SharedElementOverlay {
6
7    private var layer: GraphicsLayer? = null
8    private var sizeAnimInProgress by mutableStateOf(false)
9    private var offsetAnimInProgress by mutableStateOf(false)
10
11    private val canDrawInOverlay get() = sizeAnimInProgress || offsetAnimInProgress
12
13    val offsetAnimation = DeferredTargetAnimation(
14        vectorConverter = IntOffset.VectorConverter,
15    )
16    val sizeAnimation = DeferredTargetAnimation(
17        vectorConverter = IntSize.VectorConverter,
18    )
19
20    val moveableSharedElement: @Composable (Any?, Modifier) -> Unit = ...
21
22    override fun ContentDrawScope.drawInOverlay() {
23        if (!canDrawInOverlay) return
24        val overlayLayer = layer ?: return
25        val (x, y) = targetOffset.toOffset()
26        translate(x, y) {
27            drawLayer(overlayLayer)
28        }
29    }
30}

Wrapping up

In the above, the shared element state is preserved across both screens. i.e; it is a literal shared element. No dual views, no smoke and mirrors, no postponing and starting transitions. Just well designed and aptly named Jetpack Compose APIs doing exactly what they say on the tin. I still find it mind blowing that this is even possible. This is one of those things that I'm not even sure where I'd start to do this with Views.

Thinking outside the Box() {}

So that was cool. Let's push on a bit.

Predictive Back

Predictive back in Android is a cool feature that lets users see where they're going when they perform the back gesture without committing to it. Since we're pushing boundaries, let's assume that apps can have 1 - N navigation destinations active at any one time.

Using the setup described above, a skeuomorphic predictive back navigation transition of quite literally popping a navigation destination off the top of a stack can be created, while still having the shared element fall to the lower destination that will end up at the top.

Large screens

You're not sated? Perhaps you're even unimpressed. Maybe 2 simultaneous navigation destinations isn't impressive. Fair shout. May I interest you in 3? With the shared element transition still working?

Declarative APIs for Declarative UIs

The above is part of a series of experiments I have under the working title "Declarative APIs for Declarative UIs".

The basic argument is, Jetpack Compose is amazing and powerful, but it is a significant paradigm shift. Should we also start to rethink the APIs that assemble the state for Compose in the UI layer to get the best from it?

I think the answer is yes, and phrased even stronger: Jetpack Compose requires first frame ready developer readable and writable state production all across the UI layer to truly shine as the next generation UI toolkit that it is.

In the examples above, I needed the following APIs to be first frame ready in the UI layer:

Pagination as state: Paginated data is an immutable List subclass that can be written to and read from at any time. For shared elements with navigation, the pagination pipeline can be seeded from navigation arguments and available instantaneously at the point the shared element transition starts. My experiment for this is Tiler.
Navigation as state: Navigation data is immutable data hoisted outside the composition and transparently readable and writable anywhere in the application. Changes to navigation are read downstream, and combined with other UI signals like display size, can be dynamically adapted to the visible UI. My experiment for this is treeNav.
A robust UDF state management solution: I personally prefer MVI, and my experiment for this is Mutator.

Each of these will have dedicated blog posts deconstructing how they contributed to the examples shown above, so stay tuned. Also, if you're still on the fence on Compose, you really ought to come on down. It's quite nice here.

The app hosting all these experiments can be found below.

Tags:

animation

transition

153