Skip to content

Replace linkedlist for takenbuffer with fixed array#336

Open
var77 wants to merge 3 commits intomainfrom
varik/scan-optimizations
Open

Replace linkedlist for takenbuffer with fixed array#336
var77 wants to merge 3 commits intomainfrom
varik/scan-optimizations

Conversation

@var77
Copy link
Collaborator

@var77 var77 commented Oct 7, 2024

  • Replace takenbuffers linked list with fixed size (m*2) array and release unused buffers when pinning new ones after the array is filled.
  • Remove unused htab and fa_cache instances in retriever context
  • Only lock buffers when getting nodes if LANTERNDB_COPYNODES is enabled

TODO

  • fix LANTERNDB_COPYNODES path, currently tests are failing with incorrect scan results

@var77 var77 requested a review from Ngalstyan4 October 7, 2024 10:33
@github-actions
Copy link

github-actions bot commented Oct 7, 2024

Benchmarks

metric old new pct change
recall (after create) 0.951 0.943 -0.84%
recall (after insert) 0.000 0.000 -
select tps 22287.227 25784.272 +15.69%
select bulk(100) tps 34.705 39.108 +12.69%
select latency (ms) 0.792 ± 1.649𝜎 0.818 ± 1.760𝜎 +3.28%
select bulk(100) latency (ms) 909.948 ± 118.369𝜎 789.290 ± 113.334𝜎 -13.26%
create latency (ms) 401926.841 401222.500 -0.18%
insert tps 438.335 541.854 +23.62%
insert bulk(100) tps 4.551 5.537 +21.67%
insert latency (ms) 72.304 ± 15.802𝜎 58.199 ± 13.234𝜎 -19.51%
insert bulk(100) latency (ms) 6844.376 ± 193.644𝜎 5654.672 ± 112.456𝜎 -17.38%
disk usage (bytes) 8192008192.000 8192008192.000 -

@codecov
Copy link

codecov bot commented Oct 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant