Bazel Remote Caching ( A remote cache is used by a team of developers…
Bazel Remote Caching
- A remote cache is used by a team of developers and/or a continuous integration (CI) system to share build outputs.
- If your build is reproducible, the outputs from one machine can be safely reused on another machine, which can make builds significantly faster
- Bazel breaks a build into separate steps, which are called actions.
- Each action has inputs, output names, a command line, and environment variables.
- Required inputs and expected outputs are declared explicitly for each action
- You can set up a server to be a remote cache for build outputs, which are these action outputs.
- These outputs consist of a list of output file names and the hashes of their contents.
- With a remote cache, you can reuse build outputs from another user’s build rather than building each new output locally
To use remote caching:
The remote cache stores two types of data:
- Set up a server as the cache’s backend
- Configure the Bazel build to use the remote cache
- Use Bazel version 0.10.0 or later
- The action cache, which is a map of action hashes to action result metadata.
- A content-addressable store (CAS) of output files
- CAS is a way to store information so it can be retrieved based on its content, not its location
How a build uses remote caching
- When you run a Bazel build that can read and write to the remote cache, the build follows these steps:
- Bazel creates the graph of targets that need to be built, and then creates a list of required actions. Each of these actions has declared inputs and output filenames
- Bazel checks your local machine for existing build outputs and reuses any that it finds
- Bazel checks the cache for existing build outputs. If the output is found, Bazel retrieves the output. This is a cache hit
- For required actions where the outputs were not found, Bazel executes the actions locally and creates the required build outputs
- New build outputs are uploaded to the remote cache
- As of version 0.11.0 support for HTTP Basic Authentication was added to Bazel.
- You can pass a username and password to Bazel via the remote cache URL.
- The syntax is https://username:email@example.com:port/path.
- Please note that HTTP Basic Authentication transmits username and password in plaintext over the network and it’s thus critical to always use it with HTTPS.
HTTP Caching Protocol
- Bazel supports remote caching via HTTP/1.1.
- The protocol is conceptually simple: Binary data (BLOB) is uploaded via PUT requests and downloaded via GET requests.
- Action result metadata is stored under the path /ac/ and output files are stored under the path /cas/
- Bazel can use a directory on the file system as a remote cache.
- This is useful for sharing build artifacts when switching branches or working on multiple workspaces of the same project, such as multiple checkouts.
- Since Bazel does not garbage-collect the directory, you might want to automate a periodic cleanup of this directory.
- Enable the disk cache as follows: