feat(relay): Operation-based rate limiting + Resolve most recent by afterburn · Pull Request #185 · pubky/pkarr

afterburn · 2025-09-18T10:35:51Z

Summary
This PR replaces current rate limiting with an operation-based configuration that maps directly to the relay's three core operations: publish, resolve, and resolve_most_recent.

Configuration Example

[relay]
[[relay.rate_limits]]
operation = "resolve"
quota = "10r/s"
burst = 20

[[relay.rate_limits]]
operation = "resolve_most_recent"
quota = "2r/s"
burst = 5

[[relay.rate_limits]]
operation = "publish"
quota = "2r/s"
burst = 10
whitelist = ["127.0.0.1", "192.168.1.0/24"]  # optional, both ips and subnets supported

Operations in this toml file automatically map to:

publish → PUT requests (publishing records)
resolve → GET requests without most_recent=true (cached lookups)
resolve_most_recent → GET requests with most_recent=true (DHT queries)

If no config.toml file is present, system will use defaults, so this is not a breaking change.

SeverinAlexB

Need to do a second round later

relay/config.example.toml

relay/src/rate_limiter/extract_ip.rs

relay/src/rate_limiter/layer.rs

afterburn · 2025-09-24T10:42:20Z

@SeverinAlexB resolved your feedback:

Removed kb, mb, gb quota units, now always uses either r/s or r/m as requested
Changed default rate limits to something saner
Added behind_proxy flag in config toml, which allows users to respect/ignore proxy headers
Cleaned up IP extraction, only happens once per request now
Added operation 'Index' so users can specify rate limiting for root path

SeverinAlexB

Two comments, otherwise looks good.

In the Pubky Core team, we have a convention on how to format PR title. We stick to the conventional commit format.

So for this PR, the title could be: feat(relay): Operation-based rate limiting + Resolve most recent

relay/src/config.rs

relay/src/handlers.rs

afterburn · 2025-09-30T12:29:56Z

@SeverinAlexB

If no rate limits are defined in config.toml, rate limiting will be disabled completely
Added Index to Operation enum so calling GET http://0.0.0.0:6881/ doesn't trigger the incorrect rate limit
Added resolve_most_recent_enabled flag so users have the ability to completely ignore the most_recent=true query parameter (defaults to false for backward compatibility)

SeverinAlexB

One NIT, otherwise good to go

relay/src/config.rs

SeverinAlexB · 2025-10-06T13:50:12Z

Actually, should this add the most_recent option to the pkarr client too? You are just adding it to the relay in this PR?

pkarr/src/client/builder.rs

relay/src/config.rs

…ional

afterburn · 2025-11-04T11:01:13Z

@SeverinAlexB @SHAcollision Restored DHT rate limiting that was previously removed. Since the DHT can be accessed directly, keeping rate limiting in place is necessary. The implementation is now fully backward compatible. Existing [rate_limiter] configurations continue to work as before. New [dht_rate_limit] and [http_rate_limit] options are opt-in. By default, it falls back to the legacy IP-based limiter unless one of the new configs is defined. Also added a comprehensive test to cover the new behavior and ensure backward compat.

* Consolidate simple-dns dependency in workspace root * Bump simple-dns to latest 0.11.2 * Adapt calls to simple-dns methods * pkarr: fix port param lookup * extra: update Endpoint.params type Params type was updated to better match what simple-dns exposes. * Throw error if expected SVCParam type doesn't match * Simplify ipv4 parsing Co-authored-by: DZ <dzdidi@users.noreply.github.com> * Use static lifetine for SVCParam in Endpoint.params * fixed clippy * A few code cleanups * fmt --------- Co-authored-by: ok300 <106775972+ok300@users.noreply.github.com> Co-authored-by: DZ <dzdidi@users.noreply.github.com>

chore: release v6.0.0-rc.0

…most_recent sends ?most_recent=true to relays

86667

Few comments 👍

Also we could have quite a few more unit tests. All behaviour may be tested somewhere but unit tests inline with the code they test are pretty much always worth it these days with how fast AI can write them IMO

86667 · 2026-03-10T10:48:49Z

relay/src/config.rs

+    /// Whether to respect the most_recent query parameter in resolve requests
+    ///
+    /// If true, the most_recent query parameter is respected and will bypass cache.
+    /// If false, the most_recent query parameter is ignored and will always load from cache.


This feels like difficult behaviour to debug. Did we consider returning an error if most_recent is requested but the relay is configured with resolve_most_recent: false?

86667 · 2026-03-10T10:50:15Z

relay/src/lib.rs

+    /// Set the operation-based rate limiter configuration.
+    ///
+    /// Defaults to None (no operation-based rate limiting).
+    pub fn rate_limiter_config(&mut self, limits: Option<Vec<OperationLimit>>) -> &mut Self {


This may technically be a breaking change. im not sure if we have any other consumers of the module as a lib

86667 · 2026-03-10T10:52:47Z

pkarr/Cargo.toml

 [package]
 name = "pkarr"
-version = "5.0.3"
+version = "6.0.0-rc.0"


Should we be merging into v6 branch

if so, maybe we can remove the config backwards compatibility logic

especially concerning is behind_proxy being in two possible places - which takes precedence? If we need to keep backwards compatibility then this should be documented. Quick AI check says that [relay].behind_proxy will infact override [rate_limiter].behind_proxy

86667 · 2026-03-10T10:57:15Z

relay/src/rate_limiter/layer.rs

+        };
+
+        // Go through all the limits and check if we need to throttle or reject the request.
+        for limit in limits.clone() {


nit: no need to clone

86667 · 2026-03-10T11:02:03Z

relay/src/rate_limiter/layer.rs

+            Operation::Resolve => {
+                req.method() == Method::GET
+                    && Self::is_operation(req.uri().path())
+                    && !(resolve_most_recent && Self::has_query_param(req, "most_recent", "true"))


Why does the rate limiter need to consider the resolve_most_recent config var here? it cares for rate limiting based on the requests, the process's config is irrelevant

86667 · 2026-03-10T11:03:26Z

relay/src/rate_limiter/layer.rs

+                tracing::debug!("Rate limit of {} exceeded for {:?}", limit.limit.quota, key);
+                return Box::pin(async move {
+                    Ok(
+                        HttpError::new(StatusCode::TOO_MANY_REQUESTS, Some("Rate limit exceeded"))


Retry-After header always nice for 429s

afterburn requested review from SHAcollision and SeverinAlexB and removed request for SHAcollision September 18, 2025 12:56

afterburn force-pushed the feat/rate-limiting branch from dbfd368 to ed7d98b Compare September 22, 2025 08:27

afterburn marked this pull request as ready for review September 22, 2025 09:17

SeverinAlexB requested changes Sep 23, 2025

View reviewed changes

afterburn requested a review from SeverinAlexB September 24, 2025 10:42

SeverinAlexB reviewed Sep 25, 2025

View reviewed changes

relay/src/config.rs Outdated Show resolved Hide resolved

relay/src/handlers.rs Outdated Show resolved Hide resolved

afterburn changed the title ~~Add operation-based rate limiting~~ feat(relay): Operation-based rate limiting + Resolve most recent Sep 25, 2025

afterburn added 10 commits September 30, 2025 14:05

feat: add new relay modes

56d39c5

feat: operation based rate limiting

2d604fc

fix: remove kb, mb and gb unit support in rate limit quotas

d7feb39

fix: use saner defaults in rate limiting config example

3a4be32

feat: behind_proxy flag for ignoring proxy headers

08d3ba1

fix: add configurable rate limit for index path

753e2c8

fix: formatting

ca94a62

fix: improve comments & formatting

ff90b91

fix: only apply rate limits if specified in toml, no defaults

d3eb87a

fix: add resolve_most_recent_enabled flag

a1dbad2

afterburn force-pushed the feat/rate-limiting branch from fb7d396 to a1dbad2 Compare September 30, 2025 12:07

afterburn requested a review from SeverinAlexB September 30, 2025 12:30

SeverinAlexB reviewed Oct 6, 2025

View reviewed changes

relay/src/config.rs Outdated Show resolved Hide resolved

afterburn added 3 commits October 6, 2025 15:21

chore: resolve pr feedback

4b303be

Merge branch 'main' into feat/rate-limiting

25ca5e7

fix: failing build due to mainline version mismatch

febb535

afterburn added 2 commits October 7, 2025 15:14

fix: add most recent option to client

6198b4e

fix: formatting

7922397

dzdidi reviewed Oct 8, 2025

View reviewed changes

pkarr/src/client/builder.rs Outdated Show resolved Hide resolved

pkarr/src/client/builder.rs Outdated Show resolved Hide resolved

relay/src/config.rs Outdated Show resolved Hide resolved

relay/src/config.rs Outdated Show resolved Hide resolved

afterburn added 6 commits October 8, 2025 15:05

chore: resolve pr feedback

45975fe

chore: update authors

7666d40

Merge branch 'main' into feat/rate-limiting

8b4fc0a

chore: clean up

26bca59

chore: clean up

11a978d

refactor: restore ip-based dht limits, make relay specific limits opt…

14f5e23

…ional

afterburn requested review from SHAcollision and SeverinAlexB November 4, 2025 11:01

afterburn and others added 8 commits November 4, 2025 12:21

chore: update rate limiting test

2508940

Merge branch 'main' into feat/rate-limiting

95a55b2

fix: use burst

44d81cf

fix: clippy

f8a207b

chore: release v6.0.0-rc.0

16ba189

Merge pull request #218 from pubky/chore/v6.0.0-rc.0

df87c06

chore: release v6.0.0-rc.0

Merge branch 'main' into feat/rate-limiting

6de05fe

SeverinAlexB force-pushed the main branch from df87c06 to fb2890d Compare February 17, 2026 15:02

fix: pass most_recent flag per-call instead of per-client so resolve_…

0d89091

…most_recent sends ?most_recent=true to relays

afterburn force-pushed the feat/rate-limiting branch 3 times, most recently from 2bc52f7 to 0d89091 Compare February 27, 2026 14:33

afterburn mentioned this pull request Feb 27, 2026

fix(homeserver): Enable relay publishing for user keys pubky/pubky-core#330

Open

afterburn requested review from 86667 and removed request for SHAcollision March 6, 2026 16:27

afterburn added 3 commits March 9, 2026 21:38

fix: return success when either DHT or relay publish succeeds

37fbef9

style: format futures.rs tests

bea90e6

Merge branch 'main' into feat/rate-limiting

4b26be2

86667 reviewed Mar 10, 2026

View reviewed changes

Conversation

afterburn commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SeverinAlexB left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

afterburn commented Sep 24, 2025

Uh oh!

SeverinAlexB left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

afterburn commented Sep 30, 2025

Uh oh!

SeverinAlexB left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SeverinAlexB commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

afterburn commented Nov 4, 2025

Uh oh!

86667 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

afterburn commented Sep 18, 2025 •

edited

Loading

SeverinAlexB commented Oct 6, 2025 •

edited

Loading