feat: Add custom Docker build for package registry#1554
feat: Add custom Docker build for package registry#1554fred-maussion wants to merge 11 commits intoelastic:mainfrom
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
820aadc to
0515ec1
Compare
0515ec1 to
c89ecfd
Compare
Bumps [github.com/cloudflare/circl](https://github.com/cloudflare/circl) from 1.6.0 to 1.6.1. - [Release notes](https://github.com/cloudflare/circl/releases) - [Commits](cloudflare/circl@v1.6.0...v1.6.1) --- updated-dependencies: - dependency-name: github.com/cloudflare/circl dependency-version: 1.6.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
) Bumps [github.com/google/go-querystring](https://github.com/google/go-querystring) from 1.1.0 to 1.2.0. - [Release notes](https://github.com/google/go-querystring/releases) - [Commits](google/go-querystring@v1.1.0...v1.2.0) --- updated-dependencies: - dependency-name: github.com/google/go-querystring dependency-version: 1.2.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Update Go runtime to 1.25.7 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add PR link to changelog entry for Go 1.25.7 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update CHANGELOG.md Co-authored-by: Mario Rodriguez Molins <marrodmo@gmail.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Mario Rodriguez Molins <marrodmo@gmail.com>
It is not available in Go 1.24, the version used in go.mod files.
9eb5ba2 to
5571c49
Compare
README.md
Outdated
|
|
||
| To build your own custom Elastic Package Registry container image, follow these steps: | ||
|
|
||
| 1. **Define Your Package Set**: Create or modify the `cmd/distribution/examples/custom-packages.yml` file to specify the desired package versions. You can filter by `kibana.version`, `spec.max`, and other criteria. |
There was a problem hiding this comment.
It would be helpful to add here the expected file path that users should use for their configuration files.
There was a problem hiding this comment.
Sorry, I was referring to add in this paragraph text the expected file (e.g. cmd/distribution/custom-packages.yml) to be created. And that file can be created based on the example one.
I think it is ok to keep using cmd/distribution/custom-packages.yml, so the example file is kept without changes in the working directory. But indicating in the documentation which file should be created.
This would also mean to revert the changes in the multi-stage dockerfile.
WDYT ?
There was a problem hiding this comment.
I see this change was related to this comment too
https://github.com/elastic/package-registry/pull/1554/changes/BASE..944d898d06acfe3cc7a973f1d1f01041e151bb26#r2820751820
Do you think it would be ok to just update the example file or create a new one ?
💚 Build Succeeded
History
|
| # STAGE 3: Build the final 'package-registry' binary | ||
| FROM --platform=${BUILDPLATFORM:-linux} ${BUILDER_IMAGE}:${GO_VERSION} AS registry-builder | ||
| WORKDIR /src | ||
| COPY . . | ||
| ARG TARGETPLATFORM | ||
| # Build the main package-registry binary from the root directory | ||
| RUN make release-${TARGETPLATFORM:-linux/amd64} | ||
|
|
||
|
|
||
| # STAGE 4: Final image assembly | ||
| FROM ${RUNNER_IMAGE} | ||
|
|
||
| # Get dependencies (same as original Dockerfile) | ||
| # Mailcap is installed to get mime types information. | ||
| RUN if grep -q "Red Hat" /etc/os-release ; then \ | ||
| microdnf install -y mailcap zip rsync && \ | ||
| microdnf clean all ; \ | ||
| else \ | ||
| apk update && \ | ||
| apk add mailcap zip rsync curl && \ | ||
| rm -rf /var/cache/apk/* ; \ | ||
| fi | ||
|
|
||
| WORKDIR /package-registry | ||
|
|
||
| # Copy the main binary from the 'registry-builder' stage | ||
| COPY --from=registry-builder /src/package-registry . | ||
|
|
||
| # --- THIS IS THE KEY STEP --- | ||
| # Copy the downloaded packages from the 'packager' stage into the final image | ||
| # The registry is configured to look for packages in '/packages/package-registry/' | ||
| COPY --from=packager /src/build/distribution/. /packages/package-registry/ | ||
|
|
||
| # Get in config which expects packages in /packages | ||
| COPY config.docker.yml ./config.yml | ||
|
|
||
| # Run as non-root user | ||
| USER 1000 | ||
|
|
||
| # Start registry when container is run an expose it on port 8080 | ||
| EXPOSE 8080 | ||
| ENTRYPOINT ["./package-registry"] | ||
|
|
||
| # Make sure it's accessible from outside the container | ||
| ENV EPR_ADDRESS=0.0.0.0:8080 | ||
|
|
||
| HEALTHCHECK --interval=1s --retries=30 CMD curl --silent --fail localhost:8080/health || exit 1 | ||
|
|
There was a problem hiding this comment.
I was wondering if it could be possible to avoid duplicating the Dockerfile that builds the package-registry (stages 3 and 4 of this Dockerfile).
What about using in this Dockerfile the latest docker image of the Package Registry released or a known docker image? It could be set via a build argument so it can be set in the CLI while building the container too.
Given that, maybe this multi-stage Dockerfile could be updated to just 3 stages:
ARG GO_VERSION
ARG BUILDER_IMAGE=golang
# Example: docker.elastic.co/package-registry/package-registry:v1.36.0
ARG EPR_RUNNER_IMAGE
# STAGE 1: Build the 'distribution' tool
FROM --platform=${BUILDPLATFORM:-linux} ${BUILDER_IMAGE}:${GO_VERSION} AS distribution-builder
WORKDIR /src
COPY . .
ARG TARGETPLATFORM
# Build the distribution binary inside its own directory
RUN make -C cmd/distribution release-${TARGETPLATFORM:-linux/amd64}
# STAGE 2: Run the 'distribution' tool to download packages
FROM distribution-builder AS packager
# Copy the custom configuration for the packages we want
COPY ./cmd/distribution/examples/custom-packages.yaml .
# Run the tool. It will create the packages in 'build/distribution/'
RUN ./cmd/distribution/distribution ./cmd/distribution/examples/custom-packages.yaml
# STAGE 3: Final image assembly
FROM ${EPR_RUNNER_IMAGE}
COPY --from=packager /src/build/distribution/. /packages/package-registry/And in the README, the instructions could be updated to be like:
# Ensure to pick the latest EPR docker image (tag) released as EPR_RUNNER_IMAGE build argument
# Or, it could be used another EPR docker image if needed
# Example using tags:
git fetch --tags
latest_version=$(git tag --sort=version:refname | tail -n 1)
epr_image="docker.elastic.co/package-registry/package-registry:$(latest_version)"
# Or, it could be used another EPR docker image if needed
# Example
epr_image="docker.elastic.co/package-registry/package-registry:v1.36.0"
docker build \
--build-arg GO_VERSION="$(cat .go-version)" \
--build-arg EPR_RUNNER_IMAGE="${epr_image}" \
-f Dockerfile.custom-registry \
-t custom-package-registry .
This is an example based on tags, but the user building this custom image could be setting directly a specific docker EPR image tag.
WDYT @fred-maussion ?
There was a problem hiding this comment.
It could be interesting to keep GO_VERSION and EPR_RUNNER_IMAGE to be without value to force setting those values as build arguments in the docker build command:
ARG GO_VERSION
ARG BUILDER_IMAGE=golang
# Example: docker.elastic.co/package-registry/package-registry:v1.36.0
ARG EPR_RUNNER_IMAGE
README.md
Outdated
|
|
||
| To build your own custom Elastic Package Registry container image, follow these steps: | ||
|
|
||
| 1. **Define Your Package Set**: Create or modify the `cmd/distribution/examples/custom-packages.yml` file to specify the desired package versions. You can filter by `kibana.version`, `spec.max`, and other criteria. |
There was a problem hiding this comment.
Sorry, I was referring to add in this paragraph text the expected file (e.g. cmd/distribution/custom-packages.yml) to be created. And that file can be created based on the example one.
I think it is ok to keep using cmd/distribution/custom-packages.yml, so the example file is kept without changes in the working directory. But indicating in the documentation which file should be created.
This would also mean to revert the changes in the multi-stage dockerfile.
WDYT ?
README.md
Outdated
|
|
||
| To build your own custom Elastic Package Registry container image, follow these steps: | ||
|
|
||
| 1. **Define Your Package Set**: Create or modify the `cmd/distribution/examples/custom-packages.yml` file to specify the desired package versions. You can filter by `kibana.version`, `spec.max`, and other criteria. |
There was a problem hiding this comment.
I see this change was related to this comment too
https://github.com/elastic/package-registry/pull/1554/changes/BASE..944d898d06acfe3cc7a973f1d1f01041e151bb26#r2820751820
Do you think it would be ok to just update the example file or create a new one ?
This PR introduces the capability for users to build a custom, lightweight Docker image for the Elastic Package Registry.
The official image includes all packages, which can be too heavy for environments that only require a subset. This feature allows users to create a smaller image containing only the packages relevant to their specific Kibana version, reducing the image footprint significantly based on the cmd/distrubution of @jsoriano
Changes:
Added Dockerfile.custom-registry : A new multi-stage Dockerfile that builds a custom image. It uses the cmd/distribution tool to download a specific set of packages before assembling the final image.
Added cmd/distribution/custom-packages.yml: A default configuration file for the custom build. Users can modify this file to define the exact set of packages they need.
Updated README.md : Added a new section on how to use the new Dockerfile and configuration to build a custom image, including an example of the resulting smaller image size.
This solves #1283 by providing a way for users to create optimized Package Registry images.