How to Speed Up Docker Builds with Build Cache

How to store and share Docker build cache across teams using Docker registry.

Introduction

Nowadays, Docker is a go-to tool for building, shipping, and running containerized applications. One of the challenges a developer may face is build times, especially for large and complex codebases.

Docker build cache can offer a powerful solution to this problem by allowing you to reuse previously built layers.

In this article we will explore how to create and store build cache for different stages, such as the builder stage, and how to share this cache with your team using Docker Registry.

Understanding Docker Build Cache

Docker build cache is a mechanism that allows Docker to reuse layers from previous builds. Each instruction in a Dockerfile creates a new layer, and Docker caches these layers to avoid redundant work. When you rebuild an image, Docker checks if it can reuse any of the cached layers, which can drastically reduce build times.

Multi-Stage Builds and Caching

Multi-stage builds are a powerful feature in Docker that allows you to use multiple FROM statements in your Dockerfile. This enables you to create intermediate stages, such as a builder stage, which can be cached and reused independently. By caching these stages, you can further optimise your build process.

Setting Up Docker Build Cache

Step 1: Create a Multi-Stage Dockerfile

First, create a Dockerfile with multiple stages. Here's an example:

# Stage 1: Builder
FROM node:14 AS builder
WORKDIR /app
COPY package.json ./
RUN npm install
COPY . .
RUN npm run build

# Stage 2: Production
FROM nginx:alpine
COPY --from=builder /app/build /usr/share/nginx/html

Step 2: A bit of setup

Locally, you might have to use containerd to be able to use the cache feature.

In the settings of your Docker Desktop, in General tab, you have to enable “Use containerd for pulling and storing images.”

I also recommend you, to add this environment variable: DOCKER_BUILDKIT=1

Step 3: Build first stage and Push the Cache

To build the image and push the cache to your Docker Registry, use the following commands:

docker build --target builder \
  --cache-to=type=registry,ref=myregistry.com/myapp:cache-builder \
  -t myapp:builder-latest .

So, you built but didn’t push, still something was written to our registry. The cache layers were written to your registry.

We can tweak a little bit more our cache options to compress more those layers so it takes less space to store with: compression=zstd,mode=max

docker build --target builder \
  --cache-to=type=registry,ref=myregistry.com/myapp:cache-builder,compression=zstd,mode=max \
  -t myapp:builder-latest .

Step 4: Build the second stage while using the builder’s cache

docker build --cache-from=type=registry,ref=myregistry.com/myapp:cache-builder -t myapp:latest .

# Push the final image
docker push myregistry.com/myapp:latest

You can also use —cache-from and —cache-to with Github Actions: build-push-action

Step 5: Update your docker-compose

For your team to benefit from this cache if they use docker-compose, you simply have to add in the build:

version: '3.5'

services:
    myapp:
        build:
            context: .
            args:
                - DOCKER_BUILDKIT=1
            cache_from:
                - type=registry,ref=myregistry.com/myapp:cache-builder
...

Sharing Build Cache Across Teams

By storing the build cache in a Docker Registry, you can easily share it with your team. This ensures that everyone benefits from the cached layers, reducing build times and improving productivity.

Best Practices for Using Docker Build Cache

1. Use Multi-Stage Builds: Break down your Dockerfile into multiple stages to optimize caching.

2. Minimize Layer Changes: Group frequently changing instructions together to minimize cache invalidation.

3. Use .dockerignore: Exclude unnecessary files from the build context to speed up the build process.

4. Regularly Update Cache: Periodically rebuild and push the cache to ensure it stays up-to-date with the latest changes.

Conclusion

Leveraging Docker build cache can significantly improve your development workflow by reducing build times and making the process more efficient. By storing and sharing the cache in a Docker Registry, you can ensure that all team members benefit from these optimizations. Implementing these practices will lead to faster builds, increased productivity, and a smoother development experience.

Further Reading

• Docker Documentation: Build Cache

• Multi-Stage Builds with Docker

• Setting Up a Docker Registry