feat(worker): Add TaskWorker mode for TaskBroker RPC by NicoHinderling · Pull Request #582 · getsentry/launchpad

NicoHinderling · 2026-03-13T04:20:03Z

Summary

Adds a new TaskWorker mode (launchpad worker) where Launchpad receives tasks via TaskBroker RPC instead of Kafka. This is a lighter-weight execution mode with no HTTP server — the TaskBroker handles task distribution and dispatches work to the worker via gRPC.

Depends on #581 (decouples ArtifactProcessor from Kafka message schema).

What's included

src/launchpad/worker/: New module — TaskbrokerApp config, WorkerConfig with multi-host support, process_artifact task registration, in-memory dedup store
src/launchpad/cli.py: Adds worker CLI command
requirements.txt: Adds taskbroker-client dependency
.envrc: Adds LAUNCHPAD_WORKER_RPC_HOST and LAUNCHPAD_WORKER_CONCURRENCY env vars
Makefile: Adds make worker target
Docs: Updates CLAUDE.md and README.md to document both operational modes

Production readiness fixes

Kafka producer reads KAFKA_BOOTSTRAP_SERVERS env var (was hardcoded to 127.0.0.1:9092)
broker_hosts supports comma-separated hosts for HA
TODO added for health_check_file_path (K8s file-based liveness probes)

Kafka-side skip for taskworker-only projects

src/launchpad/kafka.py: Adds _should_skip_kafka_processing() — checks PROJECT_IDS_TO_ONLY_TRY_TASKWORKER_PROCESSING env var and skips Kafka processing for listed project IDs so they are only handled via the TaskWorker path
tests/unit/test_kafka.py: New unit tests covering skip/no-skip/empty/unset/integer-project-id cases

Test plan

make check passes
make test-unit passes (including new test_worker_config.py and test_kafka.py)
make worker starts without errors locally

NicoHinderling · 2026-03-13T04:20:15Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

sentry · 2026-03-13T04:20:47Z

Sentry Build Distribution

App Name	App ID	Version	Configuration	Install Page
Hacker News	com.emergetools.hackernews	1.0.2 (13)	Release	Install Build

requirements.txt

src/launchpad/kafka.py

src/launchpad/worker/app.py

src/launchpad/worker/tasks.py

I missed fixing some logger names when porting code from sentry. Refs getsentry/launchpad#582

src/launchpad/utils/logging.py

src/launchpad/worker/app.py

markstory · 2026-03-13T15:54:01Z

src/launchpad/worker/app.py

+app = TaskbrokerApp(
+    name="launchpad",
+    producer_factory=producer_factory,
+    router_class=CustomRouter(),


We'll want worker runtime metrics too. You'll need a taskbroker_client.metrics.MetricsBackend implementation for that.

alright did a take to implement 🙏

That looks ok to me. Does launchpad not have an existing metrics API that you could wrap?

yes this is our current metrics implementation https://github.com/getsentry/launchpad/blob/main/src/launchpad/utils/statsd.py

went ahead and updated this branch to make sure we didn't duplicate logic

markstory · 2026-03-13T15:55:29Z

src/launchpad/worker/config.py

+
+    logger.info(f"Starting TaskWorker (rpc_hosts={config.rpc_hosts}, concurrency={config.concurrency})")
+
+    # TODO: Should we explore setting health_check_file_path for K8s file-based liveness probes (TaskWorker has no HTTP server)


Yes, the healthcheck file is important as sometimes multiprocessing queues deadlock, and without the healthcheck file the worker will freeze.

markstory · 2026-03-13T15:56:24Z

src/launchpad/worker/config.py

+        result_queue_maxsize=config.concurrency * 2,
+        rebalance_after=32,
+        processing_pool_name="launchpad",
+        process_type="forkserver",


I've had problems with forkserver + sentry as the forking would timeout. Hopefully it works well for launchpad though as it provides measurable memory savings.

(we synced offline and can consider spawn as an alternative (what is used for sentry) too. regarding the issues that were encountered w forkserver: "the child processes wouldn't start, they would timeout on a socket inside multiprocessing.")

src/launchpad/worker/config.py

requirements.txt

src/launchpad/worker/app.py

src/launchpad/config.py

src/launchpad/worker/app.py

src/launchpad/worker/tasks.py

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

src/launchpad/worker/tasks.py

…llowlisted

Ensures project_id is converted to string before checking the taskworker-only allowlist, matching the Kafka consumer behavior. Prevents type mismatch if TaskBroker RPC deserializes project_id as an integer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

_TASKWORKER_ONLY_PROJECT_IDS is computed at module import time, so patching os.environ after import has no effect. Patch the set directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Disambiguate from the identically-named class in utils/arroyo_metrics.py which implements the Arroyo Metrics protocol. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

src/launchpad/worker/app.py

* fix(client) Align logger names with package name I missed fixing some logger names when porting code from sentry. Refs getsentry/launchpad#582 * Align logging scopes more * Align on __name__ as that is the convention we use most often

NicoHinderling mentioned this pull request Mar 13, 2026

ref(processor): Decouple ArtifactProcessor from Kafka message schema #581

Merged

3 tasks

NicoHinderling force-pushed the 03-12-ref_processor_decouple_artifactprocessor_from_kafka_message_schema branch from fcce11c to fd1fdde Compare March 13, 2026 04:29

NicoHinderling force-pushed the 03-12-feat_worker_add_taskworker_mode_for_taskbroker_rpc branch 2 times, most recently from 0217fd3 to dddb238 Compare March 13, 2026 05:01

NicoHinderling changed the base branch from 03-12-ref_processor_decouple_artifactprocessor_from_kafka_message_schema to graphite-base/582 March 13, 2026 05:07

NicoHinderling marked this pull request as ready for review March 13, 2026 05:29

cursor bot reviewed Mar 13, 2026

View reviewed changes

requirements.txt Outdated Show resolved Hide resolved

src/launchpad/kafka.py Outdated Show resolved Hide resolved

NicoHinderling force-pushed the 03-12-feat_worker_add_taskworker_mode_for_taskbroker_rpc branch from dddb238 to 62b8424 Compare March 13, 2026 05:39

NicoHinderling changed the base branch from graphite-base/582 to 03-12-ref_processor_decouple_artifactprocessor_from_kafka_message_schema March 13, 2026 05:39

sentry bot reviewed Mar 13, 2026

View reviewed changes

src/launchpad/worker/app.py Show resolved Hide resolved

cursor bot reviewed Mar 13, 2026

View reviewed changes

src/launchpad/worker/tasks.py Show resolved Hide resolved

markstory added a commit to getsentry/taskbroker that referenced this pull request Mar 13, 2026

fix(client) Align logger names with package name

c7f2b49

I missed fixing some logger names when porting code from sentry. Refs getsentry/launchpad#582

markstory mentioned this pull request Mar 13, 2026

fix(client) Align logger names with package name getsentry/taskbroker#572

Merged

markstory reviewed Mar 13, 2026

View reviewed changes

cursor bot reviewed Mar 13, 2026

View reviewed changes

src/launchpad/worker/app.py Outdated Show resolved Hide resolved

src/launchpad/config.py Outdated Show resolved Hide resolved

sentry bot reviewed Mar 13, 2026

View reviewed changes

src/launchpad/config.py Show resolved Hide resolved

cursor bot reviewed Mar 13, 2026

View reviewed changes

src/launchpad/worker/app.py Show resolved Hide resolved

NicoHinderling force-pushed the 03-12-feat_worker_add_taskworker_mode_for_taskbroker_rpc branch from 0beb9b8 to c172651 Compare March 13, 2026 19:07

NicoHinderling force-pushed the 03-12-ref_processor_decouple_artifactprocessor_from_kafka_message_schema branch from 8f447fb to 77d269c Compare March 13, 2026 19:07

NicoHinderling mentioned this pull request Mar 13, 2026

feat(taskworker): Attempt to add healthcheck logic #583

Open

sentry bot reviewed Mar 13, 2026

View reviewed changes

src/launchpad/worker/tasks.py Outdated Show resolved Hide resolved

cursor bot reviewed Mar 13, 2026

View reviewed changes

src/launchpad/worker/tasks.py Outdated Show resolved Hide resolved

Base automatically changed from 03-12-ref_processor_decouple_artifactprocessor_from_kafka_message_schema to main March 13, 2026 19:15

NicoHinderling and others added 5 commits March 13, 2026 12:16

feat(worker): Add TaskWorker mode for TaskBroker RPC

d4591fd

make it so that actual processing only happens if the project id is a…

b9a31d1

…llowlisted

do 16 not 32

defc790

improvements

514bfa5

NicoHinderling and others added 4 commits March 13, 2026 12:16

bot comments

282142f

fix(worker): Patch config set directly instead of os.environ in tests

ccf83f8

_TASKWORKER_ONLY_PROJECT_IDS is computed at module import time, so patching os.environ after import has no effect. Patch the set directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

ref(worker): Rename DatadogMetricsBackend to TaskworkerMetricsBackend

d91cd45

Disambiguate from the identically-named class in utils/arroyo_metrics.py which implements the Arroyo Metrics protocol. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

no more requested_features

e7fae19

NicoHinderling force-pushed the 03-12-feat_worker_add_taskworker_mode_for_taskbroker_rpc branch from 5b4d191 to e7fae19 Compare March 13, 2026 19:16

sentry bot reviewed Mar 13, 2026

View reviewed changes

src/launchpad/worker/app.py Show resolved Hide resolved

consolidate metrics implementation

31e60c6


		logger.info(f"Starting TaskWorker (rpc_hosts={config.rpc_hosts}, concurrency={config.concurrency})")

		# TODO: Should we explore setting health_check_file_path for K8s file-based liveness probes (TaskWorker has no HTTP server)

Uh oh!

Conversation

NicoHinderling commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's included

Production readiness fixes

Kafka-side skip for taskworker-only projects

Test plan

Uh oh!

NicoHinderling commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sentry bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sentry Build Distribution

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markstory Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

NicoHinderling Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

markstory Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

NicoHinderling Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

markstory Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

NicoHinderling Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

markstory Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

NicoHinderling Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

NicoHinderling commented Mar 13, 2026 •

edited

Loading

NicoHinderling commented Mar 13, 2026 •

edited

Loading

sentry bot commented Mar 13, 2026 •

edited

Loading

NicoHinderling Mar 13, 2026 •

edited

Loading