Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions .github/workflows/container-images.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
name: container-images

on:
push:
branches:
- main
tags:
- '**'
pull_request:
branches:
- main
Comment on lines +9 to +11
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this will work since you are trying to use the GITHUB_TOKEN secret in an environment which is controlled by the pull request author, isn't it? This would need to be pull_request_target.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GITHUB_TOKEN is only used in steps with the condition github.event_name == 'push'. The idea of including pull_request is to check if the container image still builds with the PR.


jobs:
build-and-push:
runs-on: ubuntu-latest

env:
DOCKER_BUILDX_PLATFORM: linux/amd64

steps:
- uses: actions/checkout@v6

- name: set up Docker buildx
uses: docker/setup-buildx-action@v4
with:
platforms: ${{ env.DOCKER_BUILDX_PLATFORM }}

- uses: docker/login-action@v4
if: ${{ github.event_name == 'push' }}
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: info
run: |
docker version
docker info

echo '${{ github.ref_name }}' | sed -e 's/[^a-zA-Z0-9._-]/_/g' > VERSION_TAG
echo "version_tag=$(cat VERSION_TAG)"

- name: build
run: |
docker buildx build \
--load \
--platform "${DOCKER_BUILDX_PLATFORM}" \
\
--build-arg "VERSION=${{ github.ref_name }}" \
--build-arg "FFMPEG_VERSION=release" \
--build-arg "BUILD_DATE=$(date -u +"%Y-%m-%dT%TZ")" \
--build-arg "GIT_COMMIT=${{ github.sha }}" \
\
-t "ghcr.io/opencast/pyca:latest" \
-t "ghcr.io/opencast/pyca:main" \
Comment on lines +54 to +55
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will create latest and main tagged container images from pull request branches. They aren't pushed, but this might still be confusing.

I think that's what these docker Actions take care of:
https://github.com/opencast/opencast-admin-interface/blob/aec24429505cdd9d12f4587b027ed916a7090c11/.github/workflows/deploy-container-image.yaml#L32-L44

But to be fair, I just copied them from a college who ensured me that this is what I wanted :D

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you said, this only tags images within the build environment. In my CI pipelines, I usually tag images with any potential tag and push if necessary. For this reason, I don't use docker/build-push-action directly, as I want to control if and what tags are pushed.

-t "ghcr.io/opencast/pyca:${{ github.sha }}" \
-t "ghcr.io/opencast/pyca:$(cat VERSION_TAG)" \
.

- name: push release
if: ${{ github.event_name == 'push' && github.ref_type == 'tag' }}
run: |
docker push "ghcr.io/opencast/pyca:$(cat VERSION_TAG)"
# assumption: last tag is always latest version
docker push "ghcr.io/opencast/pyca:latest"

- name: push dev version
if: ${{ github.event_name == 'push' && github.ref_name == 'main' }}
run: |
docker push "ghcr.io/opencast/pyca:main"

- name: delete untagged container images
uses: snok/container-retention-policy@v3.0.1
if: ${{ github.event_name == 'push' }}
with:
account: opencast
token: ${{ secrets.GITHUB_TOKEN }}
tag-selection: untagged
image-names: pyca
cut-off: 1y
Comment on lines +15 to +80

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
103 changes: 64 additions & 39 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
FROM alpine:3.13 AS build
FROM docker.io/library/alpine:3.23 AS alpine

RUN apk --no-cache add \
FROM alpine AS base
RUN apk --no-cache --virtual .run-deps add \
libcurl \
postgresql-libs \
py3-pip \
python3
RUN apk --no-cache --virtual .build-deps add \
curl-dev \
g++ \
gcc \
Expand All @@ -9,63 +15,82 @@ RUN apk --no-cache add \
musl-dev \
nodejs \
npm \
py3-pip \
python3 \
python3-dev \
util-linux \
&& ln -s /usr/bin/python3 /usr/bin/python
postgresql-dev \
python3-dev
RUN pip install --break-system-packages \
gunicorn \
psycopg2

WORKDIR /usr/local/src

FROM base as build-pyca
WORKDIR /usr/local/src
COPY requirements.txt package.json package-lock.json ./
RUN pip install -r requirements.txt \
&& npm i

RUN pip install --break-system-packages -r requirements.txt
RUN npm ci
COPY . .
RUN make pypi

FROM alpine:3.13
LABEL maintainer="pyCA team"

COPY --from=build /usr/local/src/dist/pyca-*.tar.gz /tmp/pyca.tar.gz
FROM base as build

RUN apk --no-cache --virtual .run-deps add \
libcurl \
postgresql-libs \
py3-pip \
python3 \
&& apk --no-cache --virtual .build-deps add \
COPY --from=build-pyca /usr/local/src/dist/pyca-*.tar.gz /tmp/pyca.tar.gz
RUN pip install --break-system-packages \
/tmp/pyca.tar.gz
RUN apk del .build-deps \
&& rm /tmp/pyca.tar.gz


FROM alpine AS build-ffmpeg
ARG TARGETARCH
ARG FFMPEG_VERSION=release
RUN apk add --no-cache \
curl \
curl-dev \
g++ \
gcc \
linux-headers \
make \
musl-dev \
postgresql-dev \
python3-dev \
tar \
xz \
&& ln -s /usr/bin/python3 /usr/bin/python \
&& pip install \
/tmp/pyca.tar.gz \
gunicorn \
psycopg2 \
&& cd /usr/local/bin \
&& curl -sSL "https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz" \
&& mkdir -p /tmp/ffmpeg \
&& cd /tmp/ffmpeg \
&& curl -sSL "https://s3.opencast.org/opencast-ffmpeg-static/ffmpeg-${FFMPEG_VERSION}-${TARGETARCH}-static.tar.xz" \
| tar xJf - --strip-components 1 --wildcards '*/ffmpeg' '*/ffprobe' \
&& apk del .build-deps \
&& rm -rf /tmp/pyca.tar.gz
&& chown root:root ff* \
&& mv ff* /usr/local/bin


FROM scratch AS assembly
COPY --from=build / /
COPY --from=build-ffmpeg /usr/local/bin/ff* /usr/local/bin/

COPY etc/pyca.conf etc/gunicorn.conf.py /etc/pyca/
RUN echo 'bind = "0.0.0.0:8000"' >> /etc/pyca/gunicorn.conf.py

RUN addgroup -S -g 800 pyca \
&& adduser -S -D -h /var/lib/pyca -G pyca -u 800 pyca \
&& addgroup pyca audio \
&& addgroup pyca video

COPY etc/pyca.conf etc/gunicorn.conf.py /etc/pyca/
RUN echo 'bind = "0.0.0.0:8000"' >> /etc/pyca/gunicorn.conf.py

FROM scratch AS squash
LABEL org.opencontainers.image.base.name="docker.io/library/alpine:3.23"

COPY --from=assembly / /
WORKDIR /var/lib/pyca

ARG VERSION=main
ARG BUILD_DATE=unknown
ARG GIT_COMMIT=unknown

LABEL maintainer="pyCA team" \
org.opencontainers.image.title="pyCA" \
org.opencontainers.image.description="Python Capture Agent for Opencast" \
org.opencontainers.image.version="${VERSION}" \
org.opencontainers.image.vendor="Opencast" \
org.opencontainers.image.authors="pyCA team" \
org.opencontainers.image.licenses="LGPL-3.0-only" \
org.opencontainers.image.url="https://github.com/opencast/pyCA/blob/${VERSION}/README.rst" \
org.opencontainers.image.documentation="https://github.com/opencast/pyCA/blob/${VERSION}/README.rst" \
org.opencontainers.image.source="https://github.com/opencast/pyCA" \
org.opencontainers.image.created="${BUILD_DATE}" \
org.opencontainers.image.revision="${GIT_COMMIT}"

USER pyca
VOLUME [ "/var/lib/pyca" ]
EXPOSE 8000
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ pypi: clean build
@printf "\nUpload to PyPI with \"twine upload dist/$$(python setup.py --fullname).tar.gz\"\n"

docker:
@docker build -t quay.io/opencast/pyca .
@docker build -t ghcr.io/opencast/pyca .

clean:
@python setup.py clean --all
Expand Down
4 changes: 2 additions & 2 deletions docs/install/container.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Install PyCA via Container

PyCA containers will automatically be built for each release and commit.
This can be used to easily deploy pyCA e.g. for capturing network streams.
The containers can be found at `quay.io/repository/opencast/pyca <https://quay.io/repository/opencast/pyca>`_.
The containers can be found at `ghcr.io/opencast/pyca <https://github.com/opencast/pyca/pkgs/container/pyca>`_.


Compose Files
Expand All @@ -14,6 +14,6 @@ For a simple example, run::

cp etc/pyca.conf init/container/pyca.conf
sed -i "s|#name .*|name = pyca-container|g" init/container/pyca.conf
docker-compose -f init/container/docker-compose.sqlite.yml up
docker compose -f init/container/docker-compose.sqlite.yml up

More details can be found in the `container readme file <../../init/container/README.rst>`_.
4 changes: 2 additions & 2 deletions init/container/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ PyCA + SQLite

cp etc/pyca.conf init/container/pyca.conf
sed -i "s|#name .*|name = pyca-container|g" init/container/pyca.conf
docker-compose -f init/container/docker-compose.sqlite.yml up
docker compose -f init/container/docker-compose.sqlite.yml up

PyCA + PostgreSQL
-------------
Expand All @@ -20,4 +20,4 @@ PyCA + PostgreSQL
cp etc/pyca.conf init/container/pyca.conf
sed -i "s|#name .*|name = pyca-container|g" init/container/pyca.conf
sed -i "s|#database .*|database = postgresql://pyca:pyca@database/pyca|g" init/container/pyca.conf
docker-compose -f init/container/docker-compose.postgres.yml up
docker compose -f init/container/docker-compose.postgres.yml up
12 changes: 6 additions & 6 deletions init/container/docker-compose.postgres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,39 +7,39 @@ volumes:
services:
pyca-schedule:
command: schedule
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
- pyca:/var/lib/pyca

pyca-ingest:
command: ingest
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
- pyca:/var/lib/pyca

pyca-capture:
command: capture
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
- pyca:/var/lib/pyca

pyca-agentstate:
command: agentstate
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
- pyca:/var/lib/pyca

pyca-ui:
entrypoint: ["gunicorn", "--config=/etc/pyca/gunicorn.conf.py", "pyca.ui:app"]
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
Expand All @@ -48,7 +48,7 @@ services:
- "8000:8000"

database:
image: postgres:12.3
image: docker.io/library/postgres:latest
restart: always
environment:
- POSTGRES_PASSWORD=pyca
Expand Down
10 changes: 5 additions & 5 deletions init/container/docker-compose.sqlite.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,39 +6,39 @@ volumes:
services:
pyca-schedule:
command: schedule
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
- pyca:/var/lib/pyca

pyca-ingest:
command: ingest
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
- pyca:/var/lib/pyca

pyca-capture:
command: capture
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
- pyca:/var/lib/pyca

pyca-agentstate:
command: agentstate
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
- pyca:/var/lib/pyca

pyca-ui:
entrypoint: ["gunicorn", "--config=/etc/pyca/gunicorn.conf.py", "pyca.ui:app"]
image: quay.io/opencast/pyca
image: ghcr.io/opencast/pyca
restart: always
volumes:
- ./pyca.conf:/etc/pyca/pyca.conf:ro
Expand Down
Loading