Fix parallel catchup v2 false alarm on empty queues with running workers

[jayz22]

Summary

Fixes the false alarm where the mission aborts with "No jobs left but some workers are still running" even when all jobs completed successfully (#243)

• Workers now register ownership in a Redis hash (job_owners) when claiming a job, cleared atomically in
  the completion transaction
• The job monitor only pings workers that own in-progress jobs (not all N workers), and confirms worker
  death with 3 retries (30s gaps) before retrying stuck jobs
• The F# orchestrator exits based solely on Redis queue state — worker liveness no longer gates mission
  completion

[Copilot]

Pull request overview

This PR fixes a false-alarm failure mode in parallel catchup v2 where the orchestrator aborted when Redis queues were empty but some workers were still running/reachable intermittently. It shifts completion criteria to Redis queue state and improves “stuck job” detection by tracking job ownership in Redis.

Changes:

• Add a Redis hash (job_owners) and wire it through Helm config to workers and the job monitor.
• Workers record job ownership on claim and delete ownership in the completion transaction.
• Job monitor pings only workers that own in-progress jobs and adds retry confirmation before retrying stuck jobs; orchestrator no longer gates completion on worker liveness.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/MissionParallelCatchup/parallel_catchup_helm/values.yaml	Adds job_owners Redis key to chart values.
src/MissionParallelCatchup/parallel_catchup_helm/templates/job_monitor.yaml	Passes JOB_OWNERS env var to job monitor.
src/MissionParallelCatchup/parallel_catchup_helm/templates/catchup_workers.yaml	Passes JOB_OWNERS env var to workers.
src/MissionParallelCatchup/parallel_catchup_helm/files/worker.sh	Writes/clears job ownership in Redis for each claimed job.
src/MissionParallelCatchup/job_monitor.py	Uses ownership map to limit pings and adds multi-retry confirmation before retrying in-progress jobs.
src/FSLibrary/MissionHistoryPubnetParallelCatchupV2.fs	Removes “all workers must be down” completion assertion; completes based on queue state.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.