New APIs in the Android Gradle Plugin

Published in

Android Developers

11 min readJul 29, 2020

The Android Gradle Plugin is the supported build system for Android applications and includes support for compiling many different types of sources and linking them together into an application that you can run on a physical Android device or an emulator.

The Android Gradle Plugin is decoupled from Android Studio — Android Studio can open projects that use the corresponding Android Gradle Plugin or any previous stable version, currently down all the way to Android Gradle Plugin 1.1.

Gradle and the Android Gradle Plugin provide the ability to configure and extend the build in different ways depending on the use case.

For simple projects where the configuration is mostly declarative, the Android Gradle Plugin can be configured in build.gradle files which are written with Gradle Build Language, or Domain Specific Language (DSL) which is built on top of Groovy script, or build.gradle.kts files that are written using Gradle Kotlin DSL, which was introduced more recently.

The Android Gradle Plugin DSL was written before the Gradle Kotlin DSL was introduced, and needed a refresh to work well with the Gradle Kotlin DSL. Android Gradle Plugin 4.1 now fully supports the new Kotlin DSL.

The DSL is not the only way to extend the build. For reuse within a project, scripts can be written in Groovy or Kotlin and applied in multiple build files, but often a better place for custom imperative logic is buildSrc, which can use Java, Groovy and Kotlin, and for reuse by separate projects or the build can be customized by binary plugins written in Java, Groovy or Kotlin. The Android Gradle Plugin is a binary plugin itself.

Many build authors want to extend the Android Gradle Plugin, such as by configuring specific variants in a fine grained way, generating additional artifacts that contribute to the final artifact or transforming intermediate artifacts. The Android Gradle Plugin already had APIs for all of these use cases, but they needed a refresh to be compatible with Gradle’s lazy configuration and to be less coupled to internal implementation details of the Android Gradle Plugin to provide a better foundation for more flexibility in the future.

New in Android Gradle Plugin: DSL interfaces and documentation

Android Gradle Plugin 4.1 includes new DSL interfaces extracted from the existing implementation.
From those interfaces, we have published new documentation for the DSL and API of the Android Gradle Plugin.
The API and DSL Interfaces are now in Kotlin for better Kotlin script support, which resulted in us making some breaking changes for existing Kotlin script users.

As the ecosystem around the Android Gradle Plugin had grown, our existing DSL definitions hadn’t kept up. There wasn’t a clear definition of which APIs were stable, which were experimental, and worse, some internal implementation details leaked into the apparent API of DSL and plugin implementation classes. Also, our documentation generator was an old fork from Gradle and was missing support for Java 8 language features and Kotlin.

Cleaning all this up was made more challenging by the different compatibility requirements of the various ways build authors interact with the APIs of the Android Gradle Project. Simple projects might only use the Groovy DSL, some projects have already adopted the Kotlin DSL, some projects use buildSrc, with both Java and Kotlin, and many projects bring in binary plugins. In cleaning this up we ideally want to avoid breaking compatibility with any of those use cases, especially Groovy DSL compatibility and binary compatibility for plugins.

We had to preserve the existing DSL implementation classes in order to keep binary compatibility. We decided to create interfaces in Kotlin in the gradle-api artifact that define all the public functions of the DSL and are implemented by the existing implementation classes with the goal of this being the sole source of truth for what the APIs of the Android Gradle Plugin are. The documentation for Android Gradle Plugin is now directly generated from those Kotlin interfaces, giving a single set of documentation that should be helpful for everyone using or extending the Android Gradle Plugin, whether writing Groovy or Kotlin script, buildSrc or binary plugins that add functionality.

In defining the interfaces in Kotlin, we forced several decisions about nullability and mutability to be made explicit. This is very helpful long term, but has the unfortunate side effect of potentially being a source-level breaking change for existing projects where the build authors had already embraced Gradle’s Kotlin script support, and while most breakages result in source compatibility breakages for certain errors that would have manifest as run-time errors in previous releases, there are some cases where we had to decide to make a breaking change for something that would have worked before. We hope by landing these changes now we can avoid making these sorts of breaking changes going forward.

The most challenging choice we were forced to make was around how to express collection types that are designed to be mutated in the DSL. These types are now uniformly defined as val collection: MutableCollectionType.

This means that it is no longer possible to assign collections in Kotlin scripts for some collections that previously supported it, collection = collectionTypeOf(...)

However, mutating the collection is supported uniformly so collection += …and collection.add(...) should now work everywhere.

While Gradle Kotlin DSL and Gradle Groovy DSL can look very similar, there are some constructs which were ergonomic in Groovy that don’t work well in Kotlin Script. A common example of this is what use of the = operator does. Most Android projects use an integer Android compileSdkVersion, and in Groovy script writing compileSdkVersion = 30 results in a call to setCompileSdkVersion(30). However, for some use cases with preview and add-ons compile sdk is a string, for example compileSdkVersion = "android-R". While in Groovy that simply maps to a different setCompileSdkVersion(...) method, in Kotlin we are forced to assign a type to the property.

In Android Gradle Plugin 4.1 we’ve introduced some new properties to make this easier to write in Kotlin script. We expect most builds to simply use var compileSdk: Int, and for the more specialized cases we have introduced var compileSdkPreview: String and the helper method compileSdkAddon(vendor: String, name: String, version: Int): Unit. We aim for these new properties to eventually replace compileSdkVersion, but until the community has migrated they will continue to work side-by-side.

New variant APIs

Update: This section has been updated to reflect changes to the naming in the API since initial publication.

The Android Gradle Plugin already has a variant API, but it has some limitations from its design choices. Rather than continue building up technical debt by adding new extensibility to the existing API, we decided to introduce a new API that addresses those issues, providing a clearer extension model that is decoupled from the internals of the Android Gradle Plugin. The existing API will be retained for now, and we will work to help migrate plugins and authors to the new system. We have also published a library of samples of use of the new variant and artifact APIs which we will continue to expand.

The new variant API runs much earlier during configuration than the previous API. This allows the variants to be modified in ways that affect the build flow, unlike the previous API where those decisions had already been made by the time the API ran. These changes can be both explicit, allowing setting properties that affect the build flow, but also implicit, if there is an optimization that is incompatible with something exposed in the variant API, it can simply be done conditionally on the use of that API.

The new API further split into different callbacks, beforeVariants and onVariants, and separately for the test components beforeAndroidTest and onAndroidTest, and beforeUnitTest and onUnitTest.

The beforeVariants runs the given action for each that will be created. Build authors and plugin authors can use it to customize properties that affect the overall build flow, such as whether a particular variant or test component is enabled. After that, the onVariants callbacks are called, allowing build authors and plugin authors to create custom logic and register tasks that consume or modify intermediates of the build.

Here’s an overview of the stages that an Android project build goes through:

Build scripts are run, allowing the build and plugins to configure the Android Gradle Plugin DSL objects as well as registering variant callbacks.
The Android Gradle Plugin combines build types and product flavors to create variants and test components.
The beforeVariants API is invoked for each variant, allowing the customization of variant settings, and the before[Android|Unit]Test API is invoked for each test component.
The onVariants API is invoked for each variant that was enabled and the on[Android|Unit]Test API is invoked for each enabled test component allowing the registration of tasks that consume or modify intermediates of the build.
The Android Gradle Plugin registers tasks for the variants after both the before[Component] and the on[Component] callbacks have been called.
The previous variant API is called for each variant, using the registered tasks.
Gradle calculates the task graph, and the build can begin executing.

The onVariants API makes use of Gradle Properties and Providers. Properties allow lazy computation of the value and track dependency information in the same place, allowing the use of the API without having to worry about when the value of any particular property will be known. The beforeVariants API doesn’t use properties since the values are used at configuration time, and so does not benefit from laziness. It does benefit from running much earlier during configuration, which is why it can offer much more flexibility than the previous API, which was constrained by running after the tasks were created.

Artifact APIs

The previous variant API of the Android Gradle Plugin exposed tasks of the build in order to allow consumption or extension of the build. For example, in Android Gradle Plugin 4.0 to apply the AAR output of a library to a custom task a build author or plugin author might have written something along the lines of:

While that might look simple, especially in this small example where the output was already available as a provider with dependency information attached, this is strongly coupled to the implementation of the task that produces the final AAR. If that code were compiled into a binary plugin, it would prevent the Android Gradle Plugin from ever removing a public method from a task, changing the type of the task or splitting it up in order to improve the build flow. For example, we considered switching from using the inbuilt Gradle Zip task to a custom one for building AARs, but we haven’t yet as it breaks commonly used binary plugins.

The above example only gets the artifact for consumption by a custom task. Using the previous API for replacing or transforming an artifact was generally very brittle, if possible at all, as it required coordinating multiple tasks and updating every consumer of an artifact.

With the new artifact API in Android Gradle Plugin 4.2 you can now write:

The difference here, the reference to the task is gone: the API provides more flexibility while also being decoupled from the internal implementation in the Android Gradle Plugin.

The Android Gradle Plugin manages a registry of artifacts attached to each variant for all the input, output and intermediate files, expressed as providers with dependency information attached. This registry is used throughout the Android Gradle Plugin to replace the manual wiring of tasks, and manages the locations of the artifacts to avoid accidental clashes and inconsistencies.

A subset of those artifacts are exposed through the new variant properties API.

The keys to this registry are instances of Artifact. Whether the artifact is a file or a directory is represented as part of its type. When writing custom tasks it is important to use the right corresponding Property, i.e. DirectoryProperty or RegularFileProperty. The cardinality of the artifact is also encoded in its type and specific APIs must be used to interact with ListProperty when dealing with a MultipleArtifact type.

Other interfaces that decorate artifacts indicate the type of operations that can be performed on the artifact. Such operations allow custom tasks to participate in the Android build process by either replacing, transforming or appending an artifact, all that without specific knowledge about how the items are either produced or consumed by the other tasks participating in the build process.

Operations that are now possible on artifacts include:

Get

This operation gets the final version of an artifact. Irrespective of how the artifact is produced and sometimes transformed by Tasks, calling get (or getAll for MultipleArtifacts) will always ensure the final version of the artifact is provided. It is a read-only value: modifying the artifact obtained by a get is not allowed. For that reason, the return type of the get method is a Provider and not a Property. This Provider can only be used as a task input.

For instance, you might be interested in having access to the merged manifest, not to modify it but maybe to ensure that the permissions have not changed:

Now you can wire your task :

Transformation

This operation modifies the artifact in some way. The original artifact is provided as an input to the given task along with a new location to output the transformed artifact.

Let’s look at a Task that transforms the merged manifest by setting the version to git head value.

First, let’s define the Task that will calculate the git head:

Now, create the task that will transform the manifest file using the git version calculated by the previous task.

The wiring clearly states the intent :

Append

Append is only relevant for artifact types that are decorated with MultipleArtifact. Because such types are represented as a List of either Directory or RegularFile, a task can declare an output that will be appended to the list.

The wiring is similar to other operations:

Note that at this point, we have not released a public MultipleArtifact artifact so this API will only become useful in the future.

Creation

This operation replaces the current provider of an Artifact with a new one, discarding all previous producers. It is an ‘out’ operation where a task declares itself to be the sole provider of the artifact. If there is more than one task declaring itself as the artifact provider, the last one wins.

For instance, a custom task may not use the built-in manifest merger but instead relies on a manually merged manifest checked into the source code management tool.

Wired with:

When a custom task is creating an artifact, all original producers/transformers of that artifact will not be part of the build process unless needed to produce another artifact. We don’t expect many builds to use this as the majority of the use cases we’ve considered will be served better by Transformation. If you find yourself working around a limitation in the Android Gradle Plugin using creation, please file an issue or feature request.

By focusing on artifacts rather than tasks, custom plugins or build scripts can safely extend the Android Gradle Plugin without being at the mercy of build flow changes or private details like task implementations, and we hope that these changes mean build authors can upgrade their Gradle Plugin more easily in the future.

We plan to allow more configuration of the variants and make more artifacts available as public types in upcoming releases, and we will continue to add more examples.