-
Notifications
You must be signed in to change notification settings - Fork 2
Container #12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Container #12
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,12 @@ | ||
| target/ | ||
| !target/release/shield-ci | ||
| .git/ | ||
| tests/ | ||
| tests/repo/ | ||
| tests/node_modules/ | ||
| tests/scan_output.log | ||
| tests/shield_results.json | ||
| tests/SHIELD_REPORT.md | ||
| tests/package-lock.json | ||
| *.md | ||
| SHIELD_REPORT.md | ||
| !README.md | ||
| temp_prompt.txt |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,11 +2,16 @@ name: ShieldCI Security Scan | |
|
|
||
| on: | ||
| push: | ||
| branches: [main, master] | ||
| branches: [main, master, fixed] | ||
| pull_request: | ||
| branches: [main, master] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
| issues: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| shieldci-scan: | ||
| runs-on: self-hosted | ||
|
|
@@ -31,67 +36,74 @@ jobs: | |
| fi | ||
| echo "commit_msg=$(git log -1 --pretty=%s 2>/dev/null || echo 'scan')" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Build ShieldCI engine from checked-out source | ||
| - name: Build ShieldCI engine | ||
| run: | | ||
| cd "$GITHUB_WORKSPACE" | ||
| cd "$HOME/Desktop/ShieldCI" | ||
| cargo build --release | ||
|
|
||
| - name: Check ShieldCI engine is available | ||
| run: | | ||
| if [ ! -f "$GITHUB_WORKSPACE/target/release/shield-ci" ]; then | ||
| echo "ERROR: ShieldCI engine not found after build" | ||
| if [ ! -f "$HOME/Desktop/ShieldCI/target/release/shield-ci" ]; then | ||
|
||
| echo "ERROR: ShieldCI engine not found" | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Build Kali Docker image | ||
| - name: Copy shieldci.yml config | ||
| run: | | ||
| cd "$GITHUB_WORKSPACE" | ||
| docker build -t shieldci-kali-image . | ||
| if [ -f "shieldci.yml" ]; then | ||
| cp shieldci.yml "$HOME/Desktop/ShieldCI/tests/shieldci.yml" | ||
| fi | ||
|
|
||
| - name: Install test app dependencies | ||
| - name: Copy target repo to engine | ||
| run: | | ||
| cd "$GITHUB_WORKSPACE/tests" | ||
| npm install | ||
| rm -rf "$HOME/Desktop/ShieldCI/tests/repo" | ||
| cp -r "$GITHUB_WORKSPACE" "$HOME/Desktop/ShieldCI/tests/repo" | ||
|
|
||
| - name: Run ShieldCI engine | ||
| id: scan | ||
| run: | | ||
| START_TIME=$(date +%s) | ||
| cd "$GITHUB_WORKSPACE/tests" | ||
| "$GITHUB_WORKSPACE/target/release/shield-ci" 2>&1 | tee scan_output.log || true | ||
| cd "$HOME/Desktop/ShieldCI/tests" | ||
| "$HOME/Desktop/ShieldCI/target/release/shield-ci" 2>&1 | tee scan_output.log || true | ||
|
||
| END_TIME=$(date +%s) | ||
| echo "duration=$((END_TIME - START_TIME))s" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Push results to ShieldCI dashboard | ||
| if: always() | ||
| env: | ||
| SHIELDCI_API_URL: ${{ secrets.SHIELDCI_API_URL }} | ||
| SHIELDCI_API_KEY: ${{ secrets.SHIELDCI_API_KEY }} | ||
| SHIELDCI_API_URL: http://localhost:3000 | ||
| SHIELDCI_API_KEY: fc09420a3737855a3094ff7831a6219565cee6777a0fbeec | ||
|
Comment on lines
+74
to
+75
|
||
| SHIELDCI_REPO: ${{ steps.meta.outputs.repo }} | ||
| SHIELDCI_BRANCH: ${{ steps.meta.outputs.branch }} | ||
| SHIELDCI_COMMIT: ${{ steps.meta.outputs.commit }} | ||
| SHIELDCI_COMMIT_MSG: ${{ steps.meta.outputs.commit_msg }} | ||
| SHIELDCI_DURATION: ${{ steps.scan.outputs.duration }} | ||
| SHIELDCI_TRIGGERED_BY: ${{ steps.meta.outputs.trigger }} | ||
| SHIELDCI_RESULTS_FILE: ${{ github.workspace }}/tests/shield_results.json | ||
| SHIELDCI_RESULTS_FILE: ${{ runner.temp }}/../../../Desktop/ShieldCI/tests/shield_results.json | ||
| run: | | ||
| python3 "$GITHUB_WORKSPACE/push_results.py" | ||
| export SHIELDCI_RESULTS_FILE="$HOME/Desktop/ShieldCI/tests/shield_results.json" | ||
| python3 "$HOME/Desktop/ShieldCI/push_results.py" | ||
|
Comment on lines
+84
to
+85
|
||
|
|
||
| - name: Post scan summary as PR comment | ||
| if: github.event_name == 'pull_request' | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| const fs = require('fs'); | ||
| const reportPath = process.env.GITHUB_WORKSPACE + '/tests/SHIELD_REPORT.md'; | ||
| const reportPath = process.env.HOME + '/Desktop/ShieldCI/tests/SHIELD_REPORT.md'; | ||
|
|
||
| let report = 'Scan completed but no report was generated.'; | ||
| try { | ||
| report = fs.readFileSync(reportPath, 'utf8'); | ||
| if (report.length > 60000) report = report.substring(0, 60000) + '\n\n... (truncated)'; | ||
| } catch (e) { report = 'Could not read scan report.'; } | ||
| if (report.length > 60000) | ||
| report = report.substring(0, 60000) + '\n\n... (truncated)'; | ||
| } catch (e) { | ||
| report = 'Could not read scan report.'; | ||
| } | ||
|
|
||
| await github.rest.issues.createComment({ | ||
| issue_number: context.issue.number, | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| body: '## 🛡️ ShieldCI Security Scan Results\n\n' + report | ||
| body: 'ShieldCI Security Scan Results\n\n' + report | ||
| }); | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # ── ShieldCI Engine — Kubernetes variant ── | ||
| # Runs inside gVisor-sandboxed pods as non-root user 10001. | ||
| # No Docker socket — tools are installed directly in the image. | ||
|
|
||
| # ── Stage 1: Build the Rust binary ── | ||
| FROM rust:1.82-bookworm AS builder | ||
|
|
||
| WORKDIR /build | ||
| COPY Cargo.toml Cargo.lock ./ | ||
| COPY src/ src/ | ||
|
|
||
| RUN cargo build --release | ||
|
|
||
| # ── Stage 2: Kali tool layer (cached separately) ── | ||
| FROM kalilinux/kali-rolling AS tools | ||
|
|
||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| nmap \ | ||
| nikto \ | ||
| gobuster \ | ||
| sqlmap \ | ||
| curl \ | ||
| python3 \ | ||
| python3-pip \ | ||
| git \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Install nuclei | ||
| RUN curl -sSL https://github.com/projectdiscovery/nuclei/releases/latest/download/nuclei_$(curl -s https://api.github.com/repos/projectdiscovery/nuclei/releases/latest | grep tag_name | cut -d '"' -f 4 | tr -d 'v')_linux_amd64.zip -o /tmp/nuclei.zip \ | ||
| && cd /tmp && unzip -o nuclei.zip && mv nuclei /usr/local/bin/ && rm nuclei.zip || true | ||
|
Comment on lines
+17
to
+30
|
||
|
|
||
| # Install trivy | ||
| RUN curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin || true | ||
|
|
||
| # Install semgrep | ||
| RUN pip3 install --break-system-packages semgrep 2>/dev/null || pip3 install semgrep || true | ||
|
||
|
|
||
| # ── Stage 3: Minimal runtime ── | ||
| FROM debian:bookworm-slim | ||
|
|
||
| # Install only runtime dependencies (no docker.io) | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| ca-certificates \ | ||
| python3 \ | ||
| python3-pip \ | ||
| curl \ | ||
| git \ | ||
| nmap \ | ||
| libpcap0.8 \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Create non-root user matching gVisor pod spec | ||
| RUN groupadd -g 10001 shieldci && \ | ||
| useradd -u 10001 -g shieldci -d /app -s /bin/sh shieldci | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| # Copy Rust binary | ||
| COPY --from=builder /build/target/release/shield-ci /app/shield-ci | ||
|
|
||
| # Copy tools from Kali layer | ||
| COPY --from=tools /usr/bin/nikto /usr/bin/nikto | ||
| COPY --from=tools /usr/bin/gobuster /usr/bin/gobuster | ||
| COPY --from=tools /usr/share/sqlmap /usr/share/sqlmap | ||
| COPY --from=tools /usr/bin/sqlmap /usr/bin/sqlmap | ||
| COPY --from=tools /usr/local/bin/nuclei /usr/local/bin/nuclei | ||
| COPY --from=tools /usr/local/bin/trivy /usr/local/bin/trivy | ||
|
|
||
| # Copy support files | ||
| COPY push_results.py /app/push_results.py | ||
| COPY kali_mcp.py /app/kali_mcp.py | ||
| COPY run.sh /app/run.sh | ||
| COPY detector.sh /app/detector.sh | ||
| COPY tool_call.gbnf /app/tool_call.gbnf | ||
|
|
||
| # Create writable directories for read-only root filesystem | ||
| RUN mkdir -p /app/tests /results /tmp/.shieldci && \ | ||
| chown -R 10001:10001 /app /results /tmp/.shieldci | ||
|
|
||
| ENV OLLAMA_HOST=http://ollama.shieldci-control-plane.svc.cluster.local:11434 | ||
| ENV SHIELDCI_LOCAL_TOOLS=1 | ||
| ENV SHIELDCI_RESULTS_FILE=/results/shield_results.json | ||
| ENV HOME=/tmp/.shieldci | ||
|
|
||
| USER 10001 | ||
|
|
||
| ENTRYPOINT ["/app/shield-ci"] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -27,4 +27,4 @@ echo "🚀 Starting scan..." | |
| echo "" | ||
|
|
||
| # Run the orchestrator | ||
| exec /app/shield-ci "$@" | ||
| exec /app/shield-ci "$@" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # GitHub App | ||
| GITHUB_APP_ID= | ||
| GITHUB_PRIVATE_KEY_PATH=./private-key.pem | ||
| GITHUB_WEBHOOK_SECRET= | ||
|
|
||
| # ShieldCI Backend (for push_results.py if using central API) | ||
| SHIELDCI_API_URL= | ||
| SHIELDCI_API_KEY= | ||
|
|
||
| # Ollama LLM endpoint | ||
| OLLAMA_HOST=http://localhost:11434 | ||
|
|
||
| # Server | ||
| PORT=3001 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # ShieldCI GitHub App | ||
|
|
||
| ## Required Environment Variables | ||
|
|
||
| ``` | ||
| GITHUB_APP_ID=<your app id> | ||
| GITHUB_PRIVATE_KEY_PATH=<path to .pem file> | ||
| GITHUB_WEBHOOK_SECRET=<webhook secret> | ||
| SHIELDCI_API_URL=<backend API URL> | ||
| PORT=3001 | ||
| ``` | ||
|
|
||
| ## Setup | ||
|
|
||
| 1. Register a GitHub App at https://github.com/settings/apps/new | ||
| - **Webhook URL**: `https://your-domain.com/webhook` | ||
| - **Permissions**: `checks: write`, `pull_requests: write`, `contents: read` | ||
| - **Events**: `push`, `pull_request` | ||
| 2. Download the private key `.pem` file | ||
| 3. Set environment variables in `.env` | ||
| 4. `npm install && npm start` | ||
|
|
||
| ## Architecture | ||
|
|
||
| ``` | ||
| webhook (push/PR) → Express server → authenticate as GitHub App | ||
| → queue scan job | ||
| → create "in_progress" check run | ||
| → run ShieldCI scan (Docker or local) | ||
| → post results as check run + PR comment | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| { | ||
| "name": "shieldci-github-app", | ||
| "version": "0.1.0", | ||
| "description": "ShieldCI GitHub App — automated security scanning on PRs and pushes", | ||
| "main": "src/index.js", | ||
| "scripts": { | ||
| "start": "node src/index.js", | ||
| "dev": "node --watch src/index.js" | ||
| }, | ||
| "dependencies": { | ||
| "octokit": "^3.1.0", | ||
| "@octokit/webhooks": "^12.0.0", | ||
| "express": "^4.18.0", | ||
| "jsonwebtoken": "^9.0.0", | ||
| "dotenv": "^16.3.0" | ||
| }, | ||
| "engines": { | ||
| "node": ">=18" | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The workflow depends on a developer-machine-specific path (
$HOME/Desktop/ShieldCI) rather than the checked-out repo ($GITHUB_WORKSPACE). This will break on most self-hosted runners and makes the workflow non-portable; run builds/scans from$GITHUB_WORKSPACEand write artifacts to$RUNNER_TEMPor$GITHUB_WORKSPACE.