Use sharded pub/sub by callmedenchick · Pull Request #163 · ton-connect/bridge

callmedenchick · 2025-11-17T09:45:57Z

Setup Cluster Valkey, bridge. In 2 terminals:

Terminal 1: sending messages:

/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000001&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000002&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000003&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000001&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000002&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000003&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000004&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000005&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000006&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000007&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000008&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000009&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}
/ # curl -X POST "http://172.20.0.30:8081/bridge/message?client_id=sender&to=0000000000000000
000000000000000000000000000000000000000000000010&ttl=300&topic=test" -d "message $(date +%s)"
{"message":"OK","statusCode":200}

Terminal 2: receiving messeges:

...
vent: heartbeat

event: message
id: 115572233409015125
data: {"from":"sender","message":"message 1763492331","trace_id":"019a9855-7e3e-7942-8f22-80cb46655597","connect_source":{"ip":""}}

event: message
id: 115572233839717718
data: {"from":"sender","message":"message 1763492337","trace_id":"019a9855-97ea-7eb6-b45d-f843fd639e68","connect_source":{"ip":""}}

event: heartbeat

event: message
id: 115572234261835095
data: {"from":"sender","message":"message 1763492344","trace_id":"019a9855-b113-7655-8cfa-54cfee8c33c1","connect_source":{"ip":""}}

event: heartbeat

event: message
id: 115572234664815960
data: {"from":"sender","message":"message 1763492350","trace_id":"019a9855-c918-70f6-a3a1-b45c9b4ff881","connect_source":{"ip":""}}

event: message
id: 115572235029654873
data: {"from":"sender","message":"message 1763492355","trace_id":"019a9855-ded7-7d5a-9c71-b67cc2901c44","connect_source":{"ip":""}}

event: message
id: 115572235271286106
data: {"from":"sender","message":"message 1763492359","trace_id":"019a9855-ed3e-776e-917d-7db10cb8f7b1","connect_source":{"ip":""}}

event: heartbeat

event: message
id: 115572235544112475
data: {"from":"sender","message":"message 1763492363","trace_id":"019a9855-fd81-73f6-a3bd-997c47ffa618","connect_source":{"ip":""}}

event: message
id: 115572235915570524
data: {"from":"sender","message":"message 1763492369","trace_id":"019a9856-13a5-78ff-9b67-9167d3584fa6","connect_source":{"ip":""}}

event: heartbeat

event: message
id: 115572236225949021
data: {"from":"sender","message":"message 1763492374","trace_id":"019a9856-2625-7b37-90e2-7adc021d959c","connect_source":{"ip":""}}

event: heartbeat

event: message
id: 115572236655734110
data: {"from":"sender","message":"message 1763492380","trace_id":"019a9856-3fc3-7da7-86d5-dc791361ca4c","connect_source":{"ip":""}}

event: heartbeat

All messages received.

github-actions · 2025-11-17T09:48:02Z

📊 Performance Metrics

Performance Metrics (memory storage)

CPU: 0.66s (4 cores) • Goroutines: 7 • Threads: 7
Memory: 15.6MB heap • 53.0MB RAM • 27.2MB total • 232350 allocs
GC: 5 cycles (0.69ms avg)
FDs: 8/65536 (0.0%)

Performance Metrics (postgres storage)

CPU: 0.77s (4 cores) • Goroutines: 9 • Threads: 9
Memory: 15.8MB heap • 41.7MB RAM • 25.1MB total • 248298 allocs
GC: 5 cycles (0.75ms avg)
FDs: 34/65536 (0.1%)

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (dnsmasq storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

github-actions · 2025-11-17T13:34:53Z

📊 Performance Metrics

Performance Metrics (memory storage)

CPU: 0.70s (4 cores) • Goroutines: 7 • Threads: 7
Memory: 12.7MB heap • 45.1MB RAM • 25.4MB total • 231603 allocs
GC: 6 cycles (0.55ms avg)
FDs: 8/65536 (0.0%)

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (nginx storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (dnsmasq storage)

❌ Bridge service not accessible after 30 seconds

github-actions · 2025-11-17T13:34:54Z

📊 Performance Metrics

Performance Metrics (postgres storage)

CPU: 0.78s (4 cores) • Goroutines: 9 • Threads: 9
Memory: 15.2MB heap • 43.9MB RAM • 26.4MB total • 250053 allocs
GC: 6 cycles (1.57ms avg)
FDs: 34/65536 (0.1%)

Copilot

Pull Request Overview

This PR migrates the Valkey storage implementation from regular Redis pub/sub to sharded pub/sub (SPUBLISH/SSUBSCRIBE). The change uses Redis hash tags {client} in channel names to ensure all client channels are routed to the same shard in a Redis cluster, which is a requirement for sharded pub/sub to work correctly.

Updated channel naming from client: to {client}: format with hash tags
Migrated pub/sub methods to use sharded variants (SPublish, SSubscribe, SUnsubscribe)
Updated channel parsing logic to handle the new format

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2025-11-18T19:02:45Z

📊 Performance Metrics

Performance Metrics (memory storage)

CPU: 0.67s (4 cores) • Goroutines: 7 • Threads: 7
Memory: 19.7MB heap • 48.9MB RAM • 27.9MB total • 233715 allocs
GC: 5 cycles (0.25ms avg)
FDs: 8/65536 (0.0%)

Performance Metrics (postgres storage)

CPU: 0.79s (4 cores) • Goroutines: 9 • Threads: 9
Memory: 21.8MB heap • 47.2MB RAM • 27.2MB total • 251095 allocs
GC: 5 cycles (0.38ms avg)
FDs: 34/65536 (0.1%)

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (dnsmasq storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (nginx storage)

❌ Bridge service not accessible after 30 seconds

github-actions · 2025-11-19T23:08:06Z

📊 Performance Metrics

Performance Metrics (memory storage)

CPU: 0.68s (4 cores) • Goroutines: 7 • Threads: 8
Memory: 7.2MB heap • 31.5MB RAM • 24.2MB total • 233393 allocs
GC: 7 cycles (0.11ms avg)
FDs: 8/65536 (0.0%)

Performance Metrics (postgres storage)

CPU: 0.81s (4 cores) • Goroutines: 9 • Threads: 8
Memory: 12.0MB heap • 48.2MB RAM • 28.7MB total • 251756 allocs
GC: 6 cycles (1.53ms avg)
FDs: 34/65536 (0.1%)

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (dnsmasq storage)

❌ Bridge service not accessible after 30 seconds

github-actions · 2025-11-19T23:28:08Z

📊 Performance Metrics

Performance Metrics (memory storage)

CPU: 0.70s (4 cores) • Goroutines: 7 • Threads: 7
Memory: 20.0MB heap • 49.1MB RAM • 26.8MB total • 232262 allocs
GC: 5 cycles (0.74ms avg)
FDs: 8/65536 (0.0%)

Performance Metrics (postgres storage)

CPU: 0.80s (4 cores) • Goroutines: 9 • Threads: 11
Memory: 13.4MB heap • 38.0MB RAM • 24.4MB total • 248892 allocs
GC: 6 cycles (0.68ms avg)
FDs: 34/65536 (0.1%)

📊 Performance Metrics (dnsmasq storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (cluster-valkey storage)

❌ Bridge service not accessible after 30 seconds

📊 Performance Metrics (nginx storage)

❌ Bridge service not accessible after 30 seconds

g3co · 2026-01-13T12:14:35Z

1st run with std redis lib
2nd run with valkey lib (lower traffic, smooth and equal traffic distribution to each node)

github-actions · 2026-01-19T10:46:55Z

📊 Performance Metrics

Performance Metrics (memory storage)

CPU: 0.77s (4 cores) • Goroutines: 8 • Threads: 7
Memory: 14.5MB heap • 47.1MB RAM • 38.4MB total • 245257 allocs
GC: 10 cycles (0.16ms avg)
FDs: 8/65536 (0.0%)

github-actions · 2026-01-19T10:47:00Z

📊 Performance Metrics

Performance Metrics (postgres storage)

CPU: 0.86s (4 cores) • Goroutines: 10 • Threads: 8
Memory: 13.5MB heap • 41.8MB RAM • 38.1MB total • 270633 allocs
GC: 10 cycles (0.22ms avg)
FDs: 34/65536 (0.1%)

github-actions · 2026-01-19T10:47:56Z

📊 Performance Metrics

Performance Metrics (cluster-valkey storage)

CPU: 0.76s (4 cores) • Goroutines: 15 • Threads: 10
Memory: 22.9MB heap • 56.6MB RAM • 41.3MB total • 308286 allocs
GC: 7 cycles (0.78ms avg)
FDs: 55/65536 (0.1%)

github-actions · 2026-01-19T10:48:10Z

📊 Performance Metrics

Performance Metrics (dnsmasq storage)

CPU: 0.48s (4 cores) • Goroutines: 14 • Threads: 10
Memory: 17.8MB heap • 40.5MB RAM • 25.3MB total • 161332 allocs
GC: 5 cycles (3.05ms avg)
FDs: 73/65536 (0.1%)

github-actions · 2026-01-19T10:48:13Z

📊 Performance Metrics

Performance Metrics (nginx storage)

CPU: 0.13s (4 cores) • Goroutines: 6 • Threads: 6
Memory: 3.9MB heap • 19.6MB RAM • 4.8MB total • 19334 allocs
GC: 2 cycles (0.08ms avg)
FDs: 15/65536 (0.0%)

use Sharded Pub/Sub

1ea534e

minor

f67acad

callmedenchick requested a review from Copilot November 17, 2025 21:51

Copilot started reviewing on behalf of callmedenchick November 17, 2025 21:51 View session

Copilot finished reviewing on behalf of callmedenchick November 17, 2025 21:53

Copilot AI reviewed Nov 17, 2025

View reviewed changes

callmedenchick added 2 commits November 18, 2025 18:44

add tests

94fc322

minor

415b723

implement PubSubManager

5fe4af2

fix lint

1753896

callmedenchick requested a review from TrueCarry November 19, 2025 23:31

callmedenchick changed the title ~~use sharded pub/sub~~ Use sharded pub/sub Nov 19, 2025

g3co added 3 commits January 16, 2026 17:57

- valkey with metrics

3961283

Merge branch 'master' into always-use-sharded-pub-sub

5617a56

- update lib source

348090e

g3co mentioned this pull request Jan 20, 2026

- valkey pubsub rework #191

Open

Conversation

callmedenchick commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Performance Metrics

Uh oh!

github-actions bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Performance Metrics

Uh oh!

github-actions bot commented Nov 17, 2025

📊 Performance Metrics

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

github-actions bot commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Performance Metrics

Uh oh!

github-actions bot commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Performance Metrics

Uh oh!

github-actions bot commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Performance Metrics

Uh oh!

g3co commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 19, 2026

📊 Performance Metrics

Uh oh!

github-actions bot commented Jan 19, 2026

📊 Performance Metrics

Uh oh!

github-actions bot commented Jan 19, 2026

📊 Performance Metrics

Uh oh!

github-actions bot commented Jan 19, 2026

📊 Performance Metrics

Uh oh!

github-actions bot commented Jan 19, 2026

📊 Performance Metrics

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

callmedenchick commented Nov 17, 2025 •

edited

Loading

github-actions bot commented Nov 17, 2025 •

edited

Loading

github-actions bot commented Nov 17, 2025 •

edited

Loading

github-actions bot commented Nov 18, 2025 •

edited

Loading

github-actions bot commented Nov 19, 2025 •

edited

Loading

github-actions bot commented Nov 19, 2025 •

edited

Loading

g3co commented Jan 13, 2026 •

edited

Loading