NIP-91: AND operator for filters

[hzrd149]

Duplicate PR

This PR is a clone of PR #1365 in order to reopen that PR and continue the discussion and tracking of implementations.

---

Filter operand that has optimization benefits for relays, users and developers. Likely contentious.

This NIP is the result of convergent efforts. I wrote this NIP because it seemed obvious but didn't submit because I didn't have an implementation, @v0l wrote an implementation(*) around the same time. @lez connected the dots.

Rendered NIP

Discourse

• issue

Implementations

• nostr-rs-relay docker image
• satellite-node
• netstr
• worker-relay
• applesauce
• CodyTseng/nostr-relay
• nostr-relay-nestjs
• nostr-relay-tray
• nostr-relay-sqlite
• amethyst
• nostr.land
• nostr-idb

pending

• *nostr-rs-relay (draft PR)
• strfry PR #1
• strfry PR #2
• nostr-tools

Rationale

• Reduce bandwidth for all, meme AND cat objectively consumes less bandwidth than meme OR cat
• Reduce clock-time for relays, indexing with AND is fast for all common index formats, and faster compared to OR for some index formats. (See section below)
• Reduce client-side caching requirements
• Reduce centralization vectors by reducing or even eliminating the need for centralized REST, GraphQL APIs or specialized relay "feed" endpoints.
• Give relays the option to be more useful at the protocol level while improving efficiency for all parties.

Considerations

• New field for NIP-11.limitations: max_tags_per_and and max_tags_and
• Benchmarking should be conducted to validate that bandwidth and protocol usability as benefits supercede implementation and clock-time cost.

Index Efficiency

Index Type	AND Operation Efficiency	OR Operation Efficiency	Notes
B-Tree	High	Moderate	B-Tree indexes are very efficient for AND operations, especially with compound indexes. For OR operations, they are less efficient than for AND, as the database engine might need to traverse multiple paths.
Bitmap	High	High	Bitmap indexes excel in both AND and OR operations, particularly for columns with low cardinality. They utilize fast bitwise operations, making them ideal for read-heavy environments.
Hash	Not Applicable	Not Applicable	Hash indexes are designed for equality checks and do not directly support range-based queries or optimize for AND/OR operations efficiently.
Full-Text	High	High	Optimized for text search, full-text indexes efficiently handle both AND and OR conditions, making them suitable for complex text queries.

[staab]

should I merge this again 😂

[vitorpamplona]

should I merge this again 😂

Wait!!! We need to mindlessly debate until everybody gets tired and implement the original concept anyway.

[hzrd149]

I don't have anything to debate, but I felt it was necessary to open another PR because the old one was "merged" and the changes were reverted

So this is a second attempt to merge this NIP or at least continue whatever conversation was happening and tracking implementations