How to Speed Up Docker Builds with Build Cache
How to store and share Docker build cache across teams using Docker registry.
Introduction
Nowadays, Docker is a go-to tool for building, shipping, and running containerized applications. One of the challenges a developer may face is build times, especially for large and complex codebases.
Docker build cache can offer a powerful solution to this problem by allowing you to reuse previously built layers.
In this article we will explore how to create and store build cache for different stages, such as the builder stage, and how to share this cache with your team using Docker Registry.
Understanding Docker Build Cache
Docker build cache is a mechanism that allows Docker to reuse layers from previous builds. Each instruction in a Dockerfile creates a new layer, and Docker caches these layers to avoid redundant work. When you rebuild an image, Docker checks if it can reuse any of the cached layers, which can drastically reduce build times.
Multi-Stage Builds and Caching
Multi-stage builds are a powerful feature in Docker that allows you to use multiple FROM statements in your Dockerfile. This enables you to create intermediate stages, such as a builder stage, which can be cached and reused independently. By caching these stages, you can further optimise your build process.
Setting Up Docker Build Cache
Step 1: Create a Multi-Stage Dockerfile
First, create a Dockerfile with multiple stages. Here's an example:
# Stage 1: Builder
FROM node:14 AS builder
WORKDIR /app
COPY package.json ./
RUN npm install
COPY . .
RUN npm run build
# Stage 2: Production
FROM nginx:alpine
COPY --from=builder /app/build /usr/share/nginx/html
Step 2: A bit of setup
Locally, you might have to use containerd to be able to use the cache feature.
In the settings of your Docker Desktop, in General tab, you have to enable “Use containerd for pulling and storing images.”
I also recommend you, to add this environment variable: DOCKER_BUILDKIT=1
Step 3: Build first stage and Push the Cache
To build the image and push the cache to your Docker Registry, use the following commands:
docker build --target builder \
--cache-to=type=registry,ref=myregistry.com/myapp:cache-builder \
-t myapp:builder-latest .
So, you built but didn’t push, still something was written to our registry. The cache layers were written to your registry.
We can tweak a little bit more our cache options to compress more those layers so it takes less space to store with: compression=zstd,mode=max
docker build --target builder \
--cache-to=type=registry,ref=myregistry.com/myapp:cache-builder,compression=zstd,mode=max \
-t myapp:builder-latest .
Step 4: Build the second stage while using the builder’s cache
docker build --cache-from=type=registry,ref=myregistry.com/myapp:cache-builder -t myapp:latest .
# Push the final image
docker push myregistry.com/myapp:latest
You can also use —cache-from and —cache-to with Github Actions: build-push-action
Step 5: Update your docker-compose
For your team to benefit from this cache if they use docker-compose, you simply have to add in the build:
version: '3.5'
services:
myapp:
build:
context: .
args:
- DOCKER_BUILDKIT=1
cache_from:
- type=registry,ref=myregistry.com/myapp:cache-builder
...
Sharing Build Cache Across Teams
By storing the build cache in a Docker Registry, you can easily share it with your team. This ensures that everyone benefits from the cached layers, reducing build times and improving productivity.
Best Practices for Using Docker Build Cache
Use Multi-Stage Builds: Break down your Dockerfile into multiple stages to optimize caching.
Minimize Layer Changes: Group frequently changing instructions together to minimize cache invalidation.
Use .dockerignore: Exclude unnecessary files from the build context to speed up the build process.
Regularly Update Cache: Periodically rebuild and push the cache to ensure it stays up-to-date with the latest changes.
Conclusion
Leveraging Docker build cache can significantly improve your development workflow by reducing build times and making the process more efficient. By storing and sharing the cache in a Docker Registry, you can ensure that all team members benefit from these optimizations. Implementing these practices will lead to faster builds, increased productivity, and a smoother development experience.
Further Reading
Sponsor
Stream is a DevTools company focused on making real-time chat, video, and activity feeds integrations as simple as possible. We take care of the hard part, allowing developers to focus on the parts of their app that help them stand out by providing them with an intuitive API and a rich set of UI components across all popular UI frameworks.
We are looking for Go engineers to join our backend team to help build and scale our systems to a global audience. Our stack is made up of GO, RocksDB, Redis, WebRTC and more. Check out one of our many backend opening to find your fit.