feat: add the ability to deploy plugin integrations

[PatAKnight]

PR Change Summary

This document summarizes all changes made in this PR, grouped by concern. Each section links to the relevant files and explains the motivation behind the change.

---

Overview

This PR introduces a plugin-aware deployment system for RHDH test instances. The core goals are:

1. Plugin-aware deployment — Keycloak and Lighthouse are now optional, opt-in plugins rather than always-on components
2. Centralize secrets management — rhdh-secrets is seeded early and patched incrementally by each plugin as it runs
3. Enable idempotent re-runs — re-running deploy with a different plugin set works without a prior teardown
4. Add proper teardown — new teardown.sh with plugin-aware cleanup at the right granularity
5. Add Guest auth fallback — RHDH works out of the box without Keycloak; OIDC is layered on when the Keycloak plugin is active

---

1. Centralized URL Resolution

Files: deploy.sh, helm/deploy.sh, operator/deploy.sh

Previously, CLUSTER_ROUTER_BASE, RHDH_PROTOCOL, and RHDH_BASE_URL were each calculated independently in both helm/deploy.sh and operator/deploy.sh, with slightly different logic. This caused inconsistencies and meant plugin scripts (which run before the method-specific deploy scripts) never had access to RHDH_BASE_URL.

The calculation was moved into deploy.sh before setup_resources runs, and the result is exported. Both sub-scripts now rely on the exported value.

deploy.sh
 └─ calculates CLUSTER_ROUTER_BASE, RHDH_PROTOCOL, RHDH_BASE_URL
 └─ exports RHDH_BASE_URL
     ├─ scripts/setup-resources.sh  ← uses RHDH_BASE_URL to seed rhdh-secrets
     ├─ scripts/config-plugins.sh   ← plugins use RHDH_BASE_URL directly
     ├─ helm/deploy.sh              ← no longer recalculates
     └─ operator/deploy.sh          ← no longer recalculates

---

2. Secrets Management

Files: scripts/setup-resources.sh, scripts/plugins/config-keycloak-plugin.sh, scripts/plugins/config-lighthouse-plugin.sh, config/rhdh-secrets.yaml

rhdh-secrets is created in two phases so that each plugin only needs to know about its own keys:

Phase 1 — setup-resources.sh creates the Secret idempotently using oc create --dry-run=client | oc apply. At this point only RHDH_BASE_URL is known; all Keycloak and Lighthouse keys are seeded as empty strings.

Phase 2 — Plugin scripts patch only their own keys into the existing Secret using oc patch secret rhdh-secrets --type=merge. This avoids one plugin clobbering keys written by another:

• config-keycloak-plugin.sh patches KEYCLOAK_BASE_URL, KEYCLOAK_METADATA_URL, KEYCLOAK_CLIENT_ID, KEYCLOAK_CLIENT_SECRET, KEYCLOAK_REALM, KEYCLOAK_LOGIN_REALM
• config-lighthouse-plugin.sh patches LIGHTHOUSE_URL (set to the RHDH backend proxy path) and LIGHTHOUSE_SVC_URL (set to the in-cluster service URL used as the proxy target)

config/rhdh-secrets.yaml serves as a reference template showing all expected keys — it is not applied directly.

---

3. Keycloak Credential Surfacing

File: scripts/plugins/config-keycloak-plugin.sh

Test users are provisioned with a single randomly generated password at runtime. Previously this password was only printed to stdout during standalone execution and was otherwise lost. Now it is stored in a dedicated cluster Secret:

oc get secret keycloak-test-credentials -n rhdh -o jsonpath='{.data}' | \
  jq 'to_entries[] | "\(.key): \(.value | @base64d)"' -r

Secret keys:

Key	Value
KEYCLOAK_URL	Keycloak base URL
KEYCLOAK_USERNAMES	Comma-separated list of all test usernames (read from users.json at runtime — not hardcoded)
KEYCLOAK_USER_PASSWORD	Shared password for all test users (generated at runtime)

Admin credentials are stored by the Bitnami Helm chart in keycloak-keycloak (a standard Bitnami-managed Secret).

The keycloak-test-credentials Secret is deleted during teardown.

---

4. Idempotent Re-Runs

File: helm/deploy.sh

helm install was replaced with helm upgrade --install. This allows re-running the deploy script with a different plugin configuration (or after a failed run) without needing to tear everything down first.

All oc operations that create resources already use --dry-run=client -o yaml | oc apply -f - or oc apply -f -, making them safe to re-run.

---

5. Teardown

Files: teardown.sh (new), Makefile

A dedicated teardown.sh script was created to orchestrate full or partial cleanup. It mirrors deploy.sh's argument structure and runs cleanup in the correct order:

1. Plugin infrastructure teardown (Keycloak, Lighthouse) — while the namespace still exists
2. RHDH deployment removal (helm uninstall or oc delete backstage)
3. RHDH-owned ConfigMaps deletion (app-config-rhdh, dynamic-plugins)
4. Shared resource teardown via scripts/setup-resources.sh teardown_resources (catalog ConfigMaps, RBAC, image streams, rhdh-secrets)

A --clean flag was added to teardown.sh to support full namespace deletion after the standard teardown steps complete. This is the recommended approach when you are done with a test instance entirely, as pre-deleting resources (Helm releases, StatefulSets, PVCs) before namespace deletion significantly reduces the time OpenShift spends in the Terminating state.

./teardown.sh helm --plugins keycloak,lighthouse        # standard teardown, preserves namespace + image streams
./teardown.sh helm --plugins keycloak,lighthouse --clean # full wipe: standard teardown then deletes namespace

PVCs created by StatefulSets are intentionally not deleted by Helm uninstall (Kubernetes default behavior). The teardown scripts now delete them explicitly to prevent conflicts on re-deploy:

• data-redhat-developer-hub-postgresql-0 — deleted in teardown.sh after RHDH Helm uninstall
• data-keycloak-postgresql-0 — deleted in uninstall_keycloak() after Keycloak Helm uninstall

The Makefile was updated with new targets:

Target	What it does
make undeploy-helm	Tears down Helm-deployed RHDH and shared resources
make undeploy-operator	Tears down Operator-deployed RHDH and shared resources
make undeploy-helm PLUGINS=keycloak,lighthouse	Same, plus plugin infrastructure
make undeploy-plugins PLUGINS=keycloak	Tear down plugin infra only, leave RHDH running
make clean	Delete the entire namespace

---

6. Default Guest Authentication

Files: config/app-config-rhdh.yaml, resources/catalog-entities/users.yaml, resources/keycloak/dynamic-plugins.yaml

Previously RHDH was configured with OIDC (Keycloak) as the only sign-in provider, meaning Keycloak had to be running for any login to work.

The base configuration now uses the Backstage guest provider via dangerouslyAllowOutsideDevelopment: true, allowing immediate sign-in without any plugin dependencies. A user:default/guest entity is pre-registered in the catalog via the new users-config-map.

When --plugins keycloak is used, the pluginConfig block inside resources/keycloak/dynamic-plugins.yaml overrides auth.providers and signInPage to switch the portal to OIDC — without requiring a separate app-config-keycloak ConfigMap.

---

7. Shared Catalog Entity ConfigMaps

Files: resources/catalog-entities/, scripts/setup-resources.sh, helm/value_file.yaml, operator/subscription.yaml

Five catalog entity ConfigMaps are now applied unconditionally by setup-resources.sh:

ConfigMap	Contents
users-config-map	user:default/guest entity
operators-config-map	OpenShift Operator catalog entries
plugins-config-map	RHDH plugin catalog entries
components-config-map	Example component catalog entries
resources-config-map	Example resource catalog entries

Both helm/value_file.yaml and operator/subscription.yaml were updated to mount all five ConfigMaps as a projected volume at /opt/app-root/src/catalog-entities, which maps to the catalog location paths in app-config-rhdh.yaml.

---

8. Restructured Plugin Resources

Files: resources/keycloak/, resources/lighthouse/, resources/image-stream-imports/, resources/rhdh-script-examples/

Plugin-specific resources were moved from utils/keycloak/ into a structured resources/ directory:

Old path	New path
utils/keycloak/keycloak-values.yaml	resources/keycloak/keycloak-values.yaml
utils/keycloak/rhdh-client.json	resources/keycloak/rhdh-client.json
utils/keycloak/users.json	resources/keycloak/users.json (expanded to 50 users)
utils/keycloak/groups.json	resources/keycloak/groups.json
(new)	resources/keycloak/dynamic-plugins.yaml
(new)	resources/lighthouse/dynamic-plugins.yaml
(new)	resources/lighthouse/lighthouse-deployment.yaml
(new)	resources/lighthouse/lighthouse-scan-job.yaml
(new)	resources/image-stream-imports/lighthouse-import.yaml
(new)	resources/rhdh-script-examples/backstage-test.yaml

utils/keycloak/keycloak-deploy.sh, utils/keycloak/users.json, and utils/keycloak/groups.json are deleted — this logic is now handled by scripts/plugins/config-keycloak-plugin.sh.

---

9. New Scripts

File	Purpose
scripts/setup-resources.sh	Applies/tears down shared cluster resources (catalog ConfigMaps, RBAC, image streams, rhdh-secrets, demo workloads)
scripts/config-plugins.sh	Orchestrates plugin setup and teardown in round-robin order; sources individual plugin scripts
scripts/plugins/config-keycloak-plugin.sh	Full Keycloak lifecycle: deploy, realm/client config, user/group provisioning, secret patching, label application, teardown
scripts/plugins/config-lighthouse-plugin.sh	Lighthouse lifecycle: deploy, URL injection, initial scan, teardown
teardown.sh	Main teardown entry point; mirrors deploy.sh argument structure

---

10. Lighthouse Backend Proxy

Files: config/app-config-rhdh.yaml, resources/lighthouse/dynamic-plugins.yaml, scripts/plugins/config-lighthouse-plugin.sh

The Lighthouse frontend plugin makes direct browser requests to the Lighthouse audit service API. Pointing it at an OpenShift Route with a self-signed cluster cert causes ERR_CERT_AUTHORITY_INVALID in the browser.

To avoid this, Lighthouse traffic is routed through RHDH's built-in backend proxy. The browser only ever communicates with RHDH (which has a valid cert), and RHDH forwards requests to the Lighthouse service over plain HTTP on the cluster-internal network.

Two URL values are now managed separately:

Secret key	Value	Purpose
LIGHTHOUSE_SVC_URL	http://lighthouse.<namespace>.svc.cluster.local:3003	Proxy target — used by RHDH backend to reach Lighthouse internally
LIGHTHOUSE_URL	${RHDH_BASE_URL}/api/proxy/lighthouse	Plugin baseUrl — what the frontend plugin calls; routes through RHDH

The proxy endpoint is configured in app-config-rhdh.yaml:

backend:
  proxy:
    endpoints:
      /lighthouse:
        target: '${LIGHTHOUSE_SVC_URL}'
        changeOrigin: true

---

Files Changed at a Glance

Status	File	Summary
Modified	deploy.sh	Centralized URL resolution; removed deleted source; added --plugins arg
Modified	helm/deploy.sh	helm upgrade --install; removed duplicate URL calc and envsubst
Modified	operator/deploy.sh	Removed duplicate URL calc and envsubst
Modified	Makefile	Added undeploy-plugins, PLUGINS var; updated all cleanup targets
Modified	README.md	Full documentation refresh
Modified	config/app-config-rhdh.yaml	Switched default auth to Guest; added Lighthouse backend proxy config
Modified	config/rhdh-secrets.yaml	Reference template only (no longer applied directly)
Modified	config/dynamic-plugins.yaml	Trimmed to base config only
Modified	helm/value_file.yaml	Added all 5 catalog ConfigMaps as projected volume sources
Modified	operator/subscription.yaml	Added extraFiles for all catalog ConfigMaps and RBAC policy
Added	config/rbac-policies.yaml	RBAC policy ConfigMap
Added	teardown.sh	Main teardown entry point
Added	scripts/setup-resources.sh	Shared resource provisioning and teardown
Added	scripts/config-plugins.sh	Plugin orchestration
Added	scripts/plugins/config-keycloak-plugin.sh	Keycloak plugin lifecycle
Added	scripts/plugins/config-lighthouse-plugin.sh	Lighthouse plugin lifecycle
Added	resources/catalog-entities/users.yaml	Guest user entity ConfigMap
Added	resources/catalog-entities/components.yaml	Component catalog entries
Added	resources/catalog-entities/operators.yaml	Operator catalog entries
Added	resources/catalog-entities/plugins.yaml	Plugin catalog entries
Added	resources/catalog-entities/resources.yaml	Resource catalog entries
Added	resources/keycloak/dynamic-plugins.yaml	Keycloak plugin config + OIDC auth overlay
Added	resources/keycloak/users.json	50 test users (Marvel characters)
Added	resources/keycloak/groups.json	Test groups
Renamed	utils/keycloak/keycloak-values.yaml → resources/keycloak/keycloak-values.yaml	Helm values for Keycloak
Renamed	utils/keycloak/rhdh-client.json → resources/keycloak/rhdh-client.json	Keycloak OIDC client definition
Added	resources/lighthouse/dynamic-plugins.yaml	Lighthouse plugin config
Added	resources/lighthouse/lighthouse-deployment.yaml	Lighthouse Deployment and Service
Added	resources/lighthouse/lighthouse-scan-job.yaml	Lighthouse initial scan Job
Added	resources/image-stream-imports/lighthouse-import.yaml	ImageStreamImport to pre-pull Lighthouse image
Added	resources/rhdh-script-examples/backstage-test.yaml	Demo workload for Topology/Kubernetes views
Deleted	utils/keycloak/keycloak-deploy.sh	Replaced by scripts/plugins/config-keycloak-plugin.sh
Deleted	utils/keycloak/users.json	Moved and expanded under resources/keycloak/
Deleted	utils/keycloak/groups.json	Moved under resources/keycloak/