Skip to content

Develop macos arm docker #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
t0mdavid-m merged 7 commits into from
Mar 20, 2026
+45 −19
Merged

Develop macos arm docker #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ae46665
Allow publish workflow to build from branch without a release tag
t0mdavid-m Mar 19, 2026
323b3fe
Temporarily disable amd64 build and manifest for arm64-only test
t0mdavid-m Mar 19, 2026
18617af
Fix multi-arch manifest creation and Redis ARM64 startup
t0mdavid-m Mar 19, 2026
5451218
Temporarily disable amd64 and manifest for arm64-only test build
t0mdavid-m Mar 19, 2026
b1c6a43
Re-enable amd64 build and manifest creation
t0mdavid-m Mar 19, 2026
84cf134
Fix Docker token leak, add CI disk cleanup, update PATH and actions
t0mdavid-m Mar 20, 2026
15b03a4
Sanitize VERSION for valid Docker tags in publish workflow
t0mdavid-m Mar 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions .github/workflows/build-docker-images.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@ jobs:
contents: read

steps:
- uses: actions/checkout@v3
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache
sudo apt-get clean
df -h
- uses: actions/checkout@v4
- name: Build the full Docker image
run: docker build --file Dockerfile --tag streamlitapp:latest --build-arg GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} --output type=tar,dest=/dev/null .

Expand All @@ -26,6 +31,11 @@ jobs:
contents: read

steps:
- uses: actions/checkout@v3
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache
sudo apt-get clean
df -h
- uses: actions/checkout@v4
- name: Build the ARM64 Docker image
run: docker build --file Dockerfile.arm --tag streamlitapp:latest-arm64 --build-arg GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} --output type=tar,dest=/dev/null .
18 changes: 14 additions & 4 deletions .github/workflows/publish-docker-images.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
inputs:
tag:
description: 'Release tag to build (e.g., v0.9.15)'
required: true
required: false

jobs:

Expand All @@ -31,15 +31,23 @@ jobs:
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
TAG="${{ github.event.inputs.tag }}"
SHA=$(gh api repos/${{ github.repository }}/git/ref/tags/${TAG} --jq '.object.sha')
if [ -n "$TAG" ]; then
SHA=$(gh api repos/${{ github.repository }}/git/ref/tags/${TAG} --jq '.object.sha')
VERSION="${TAG#v}"
Comment on lines +35 to +36
Copy link
Contributor

@coderabbitai coderabbitai bot Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Checking whether origin has annotated tags (which use tag objects)..."
ANNOTATED_TAG=$(
  git ls-remote --tags origin \
  | awk '/\^\{\}$/ {tag=$2; sub("refs/tags/","",tag); sub(/\^\{\}$/,"",tag); print tag; exit}'
)

if [ -z "${ANNOTATED_TAG:-}" ]; then
  echo "No annotated tags found right now; risk still exists for future annotated tags."
  exit 0
fi

echo "Annotated tag detected: ${ANNOTATED_TAG}"
ENCODED_TAG=$(python - <<'PY'
import os, urllib.parse
print(urllib.parse.quote(os.environ["ANNOTATED_TAG"], safe=""))
PY
)

echo "GitHub /git/ref/tags/{tag} response object fields:"
curl -s "https://api.github.com/repos/OpenMS/FLASHApp/git/ref/tags/${ENCODED_TAG}" \
| python - <<'PY'
import json,sys
d=json.load(sys.stdin)
print("object.type =", d.get("object",{}).get("type"))
print("object.sha  =", d.get("object",{}).get("sha"))
PY

Repository: OpenMS/FLASHApp

Length of output: 208

🌐 Web query:

GitHub REST API git/ref/tags annotated tags object.type object.sha

💡 Result:

In the GitHub REST API, git/refs/tags refer to references under refs/tags/, accessed via endpoints like GET /repos/{owner}/{repo}/git/refs/tags/{tag} or matching refs. For annotated tags (the only type supported by the Git tags API), the ref's object has type "tag" and sha pointing to the annotated tag object SHA. To create an annotated tag: 1. POST /repos/{owner}/{repo}/git/tags to create the tag object, providing "object" (SHA of the target commit/tree/blob) and "type" ("commit", "tree", or "blob"). The response includes the new tag object's sha. 2. POST /repos/{owner}/{repo}/git/refs with "ref": "refs/tags/{tagname}" and "sha": {tag_object_sha}. Example creation request: { "tag": "v0.0.1", "message": "initial version", "object": "c3d0be41ecbe669545ee3e94d31ed9a4bc91ee3c", "type": "commit", ... } The tag object response includes its own "sha" (tag SHA), "object.sha" (target SHA), and "object.type". For a refs/tags/{tag} response: {"object": {"type": "tag", "sha": "{tag_sha}"}}. To get the tagged object's SHA/type, GET the tag object at /repos/{owner}/{repo}/git/tags/{tag_sha}, which has "object": {"type": "...", "sha": "..."}.

Citations:

Resolve annotated tags to commit SHA before exporting sha.

On Line 35, .object.sha from git/ref/tags/{tag} returns a tag object SHA for annotated tags, not a commit SHA. While the repository currently has no annotated tags, using a tag object SHA in the checkout at Line 74/117 via ref: needs.resolve-tag.outputs.sha will fail if annotated tags are created. The code should handle both annotated and lightweight tags.

🔧 Proposed fix 
           if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
             TAG="${{ github.event.inputs.tag }}"
             if [ -n "$TAG" ]; then
-              SHA=$(gh api repos/${{ github.repository }}/git/ref/tags/${TAG} --jq '.object.sha')
+              REF_TYPE=$(gh api repos/${{ github.repository }}/git/ref/tags/${TAG} --jq '.object.type')
+              REF_SHA=$(gh api repos/${{ github.repository }}/git/ref/tags/${TAG} --jq '.object.sha')
+              if [ "$REF_TYPE" = "tag" ]; then
+                SHA=$(gh api repos/${{ github.repository }}/git/tags/${REF_SHA} --jq '.object.sha')
+              else
+                SHA="$REF_SHA"
+              fi
               VERSION="${TAG#v}"
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/publish-docker-images.yml around lines 35 - 36, The
current SHA assignment uses the ref for ${TAG} which returns a tag object SHA
for annotated tags; update the logic that sets SHA so it resolves to the
underlying commit for annotated tags: call gh api repos/${{ github.repository
}}/git/ref/tags/${TAG} and inspect .object.type and .object.sha (store as
TAG_SHA); if .object.type == "tag" then call gh api repos/${{ github.repository
}}/git/tags/${TAG_SHA} and use its .object.sha as the final SHA, otherwise use
the original .object.sha; ensure the final value is assigned to the SHA variable
used by downstream steps (e.g., needs.resolve-tag.outputs.sha).

else
# No tag — build from branch HEAD
SHA="${{ github.sha }}"
VERSION="${{ github.ref_name }}"
fi
else
# workflow_run: get the tag from the head branch (release events set head_branch to the tag)
TAG="${{ github.event.workflow_run.head_branch }}"
SHA="${{ github.event.workflow_run.head_sha }}"
VERSION="${TAG#v}"
fi

# Strip leading 'v' for version
VERSION="${TAG#v}"
# Sanitize VERSION for valid Docker tags: replace / with -, strip invalid chars
VERSION=$(echo "$VERSION" | tr '/' '-' | tr -cd 'A-Za-z0-9_.-')
Comment on lines +49 to +50
Copy link
Contributor

@coderabbitai coderabbitai bot Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fail fast when sanitized VERSION is invalid.

After Line 50, VERSION can become empty (or exceed Docker’s tag limit) and fail later during push/manifest steps with opaque errors. Validate immediately.

🔧 Proposed fix 
           # Sanitize VERSION for valid Docker tags: replace / with -, strip invalid chars
           VERSION=$(echo "$VERSION" | tr '/' '-' | tr -cd 'A-Za-z0-9_.-')
+          if [ -z "$VERSION" ]; then
+            echo "::error::Resolved VERSION is empty after sanitization"
+            exit 1
+          fi
+          if [ ${`#VERSION`} -gt 128 ]; then
+            echo "::error::Resolved VERSION exceeds 128 characters (${`#VERSION`})"
+            exit 1
+          fi

Also applies to: 52-54

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/publish-docker-images.yml around lines 49 - 50, After
sanitizing VERSION (the line using VERSION=$(echo "$VERSION" | tr '/' '-' | tr
-cd 'A-Za-z0-9_.-')), validate it immediately: if VERSION is empty or longer
than Docker's tag limit (128 characters) print a clear error message that
includes the sanitized value and exit non‑zero so the workflow fails fast
instead of producing opaque errors later during push/manifest steps; implement
this check right after the tr/... line and use explicit checks for -z "$VERSION"
and string length > 128, returning a helpful error and exit 1.


echo "Resolved tag=${TAG} version=${VERSION} sha=${SHA}"
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
Expand Down Expand Up @@ -82,6 +90,7 @@ jobs:
file: Dockerfile
push: true
tags: ${{ env.IMAGE }}:${{ needs.resolve-tag.outputs.version }}-amd64
provenance: false
build-args: |
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}
cache-from: type=registry,ref=${{ env.IMAGE }}:buildcache-amd64
Expand Down Expand Up @@ -124,6 +133,7 @@ jobs:
file: Dockerfile.arm
push: true
tags: ${{ env.IMAGE }}:${{ needs.resolve-tag.outputs.version }}-arm64
provenance: false
build-args: |
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}
cache-from: type=registry,ref=${{ env.IMAGE }}:buildcache-arm64
Expand Down
17 changes: 10 additions & 7 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ ARG OPENMS_BRANCH=FVdeploy
ARG PORT=8501
# GitHub token to download latest OpenMS executable for Windows from Github action artifact.
ARG GITHUB_TOKEN
ENV GH_TOKEN=${GITHUB_TOKEN}
# Streamlit app Gihub user name (to download artifact from).
ARG GITHUB_USER=OpenMS
# Streamlit app Gihub repository name (to download artifact from).
Expand Down Expand Up @@ -90,7 +89,7 @@ RUN mkdir /thirdparty && \
cp -r THIRDPARTY/All/* /thirdparty && \
cp -r THIRDPARTY/Linux/x86_64/* /thirdparty && \
chmod -R +x /thirdparty
ENV PATH="/thirdparty/LuciPHOr2:/thirdparty/MSGFPlus:/thirdparty/Sirius:/thirdparty/ThermoRawFileParser:/thirdparty/Comet:/thirdparty/Fido:/thirdparty/MaRaCluster:/thirdparty/MyriMatch:/thirdparty/OMSSA:/thirdparty/Percolator:/thirdparty/SpectraST:/thirdparty/XTandem:/thirdparty/crux:${PATH}"
ENV PATH="/thirdparty/LuciPHOr2:/thirdparty/MSGFPlus:/thirdparty/ThermoRawFileParser:/thirdparty/Comet:/thirdparty/Percolator:/thirdparty/Sage:${PATH}"

# Build OpenMS and pyOpenMS.
FROM setup-build-system AS compile-openms
Expand Down Expand Up @@ -190,7 +189,7 @@ service cron start\n\
\n\
# Start Redis server in background\n\
echo "Starting Redis server..."\n\
redis-server --daemonize yes --dir /var/lib/redis --appendonly no\n\
redis-server --daemonize yes --dir /var/lib/redis --appendonly no --ignore-warnings ARM64-COW-BUG\n\
\n\
# Wait for Redis to be ready\n\
until redis-cli ping > /dev/null 2>&1; do\n\
Expand Down Expand Up @@ -250,11 +249,15 @@ RUN mamba run -n streamlit-env python hooks/hook-analytics.py
RUN jq '.online_deployment = true' settings.json > tmp.json && mv tmp.json settings.json

# Download latest OpenMS App executable as a ZIP file
RUN if [ -n "$GH_TOKEN" ]; then \
echo "GH_TOKEN is set, proceeding to download the release asset..."; \
gh release download -R ${GITHUB_USER}/${GITHUB_REPO} -p "OpenMS-App.zip" -D /app; \
# Re-declare ARGs needed in this stage (ARGs don't persist across FROM)
ARG GITHUB_TOKEN
ARG GITHUB_USER=OpenMS
ARG GITHUB_REPO=FLASHApp
RUN if [ -n "$GITHUB_TOKEN" ]; then \
echo "Downloading release asset..."; \
GH_TOKEN="$GITHUB_TOKEN" gh release download -R ${GITHUB_USER}/${GITHUB_REPO} -p "OpenMS-App.zip" -D /app; \
else \
echo "GH_TOKEN is not set, skipping the release asset download."; \
echo "No token, skipping download."; \
fi


Expand Down
15 changes: 9 additions & 6 deletions Dockerfile.arm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ ARG OPENMS_BRANCH=FVdeploy
ARG PORT=8501
# GitHub token to download latest OpenMS executable for Windows from Github action artifact.
ARG GITHUB_TOKEN
ENV GH_TOKEN=${GITHUB_TOKEN}
# Streamlit app Gihub user name (to download artifact from).
ARG GITHUB_USER=OpenMS
# Streamlit app Gihub repository name (to download artifact from).
Expand Down Expand Up @@ -192,7 +191,7 @@ service cron start\n\
\n\
# Start Redis server in background\n\
echo "Starting Redis server..."\n\
redis-server --daemonize yes --dir /var/lib/redis --appendonly no\n\
redis-server --daemonize yes --dir /var/lib/redis --appendonly no --ignore-warnings ARM64-COW-BUG\n\
\n\
# Wait for Redis to be ready\n\
until redis-cli ping > /dev/null 2>&1; do\n\
Expand Down Expand Up @@ -252,11 +251,15 @@ RUN mamba run -n streamlit-env python hooks/hook-analytics.py
RUN jq '.online_deployment = true' settings.json > tmp.json && mv tmp.json settings.json

# Download latest OpenMS App executable as a ZIP file
RUN if [ -n "$GH_TOKEN" ]; then \
echo "GH_TOKEN is set, proceeding to download the release asset..."; \
gh release download -R ${GITHUB_USER}/${GITHUB_REPO} -p "OpenMS-App.zip" -D /app; \
# Re-declare ARGs needed in this stage (ARGs don't persist across FROM)
ARG GITHUB_TOKEN
ARG GITHUB_USER=OpenMS
ARG GITHUB_REPO=FLASHApp
RUN if [ -n "$GITHUB_TOKEN" ]; then \
echo "Downloading release asset..."; \
GH_TOKEN="$GITHUB_TOKEN" gh release download -R ${GITHUB_USER}/${GITHUB_REPO} -p "OpenMS-App.zip" -D /app; \
else \
echo "GH_TOKEN is not set, skipping the release asset download."; \
echo "No token, skipping download."; \
fi


Expand Down
Loading