DiskANN v0.47.0

DiskANN v0.47.0

Summary

• This version contains a major breaking change to the search interface of DiskANNIndex. Please read the upgrade instructions below.
• An Aarch64 Neon has been added to diskann-wide.
• Various bug-fixes and code-quality improvements.

Changes to Search

The search interface has been unified around a single index.search() entry point using the Search trait.
The old per-search-type methods on DiskANNIndex (search, search_recorded, range_search, multihop_search) have been removed and replaced by typed parameter structs that carry their own search logic.

What Changed

Removed	Replacement
SearchParams struct	diskann::graph::search::Knn
RangeSearchParams struct	diskann::graph::search::Range
SearchParamsError	diskann::graph::KnnSearchError
RangeSearchParamsError	diskann::graph::RangeSearchError
index.search(&strategy, &ctx, &query, &params, &mut out)	index.search(knn, &strategy, &ctx, &query, &mut out)
index.search_recorded(..., &mut recorder)	index.search(RecordedKnn::new(knn, &mut recorder), ...)
index.range_search(&strategy, &ctx, &query, &params)	index.search(range, &strategy, &ctx, &query, &mut ())
index.multihop_search(..., &label_eval)	index.search(MultihopSearch::new(knn, &label_eval), ...)
index.diverse_search(...)	index.search(Diverse::new(knn, diverse_params), ...)

flat_search remains an inherent method on DiskANNIndex
Its search_params argument changed from &SearchParams to &Knn.

Upgrade Instructions

1. k-NN Search (search)

Before:

use diskann::graph::SearchParams;

let params = SearchParams::new(10, 100, None)?;
let stats = index.search(&strategy, &ctx, &query, &params, &mut output).await?;

After:

use diskann::graph::{Search, search::Knn};

let params = Knn::new(10, 100, None)?;
// Note: params is now the FIRST argument (moved before strategy)
let stats = index.search(params, &strategy, &ctx, &query, &mut output).await?;

Key differences:

• SearchParams -> Knn (import from diskann::graph::search::Knn)
• SearchParamsError -> KnnSearchError (import from diskann::graph::KnnSearchError)
• Search params moved to the first argument of index.search()
• k_value, l_value fields are now private; use .k_value(), .l_value() accessors (return NonZeroUsize)

2. Recorded/Debug Search (search_recorded)

Before:

use diskann::graph::SearchParams;

let params = SearchParams::new(10, 100, None)?;
let stats = index
   .search_recorded(&strategy, &ctx, &query, &params, &mut output, &mut recorder)
   .await?;

After:

use diskann::graph::{Search, search::{Knn, RecordedKnn}};

let params = Knn::new(10, 100, None)?;
let recorded = RecordedKnn::new(params, &mut recorder);
let stats = index.search(recorded, &strategy, &ctx, &query, &mut output).await?;

3. Range Search (range_search)

Before:

use diskann::graph::RangeSearchParams;

let params = RangeSearchParams::new(None, 100, None, 0.5, None, 1.0, 1.0)?;
let (stats, ids, distances) = index
   .range_search(&strategy, &ctx, &query, &params)
   .await?;

After:

use diskann::graph::{
    Search,
    search::Range,
    RangeSearchOutput,
};

// Simple form:
let params = Range::new(100, 0.5)?;
// Or full options form:
let params = Range::with_options(None, 100, None, 0.5, None, 1.0, 1.0)?;

// Note: output buffer is `&mut ()` — results come back in the return type
let result: RangeSearchOutput<_> = index
   .search(params, &strategy, &ctx, &query, &mut ())
   .await?;

// Access results:
let stats = result.stats;
let ids = result.ids;           // Vec<O>
let distances = result.distances; // Vec<f32>

Key differences:

• RangeSearchParams -> Range (import from diskann::graph::search::Range)
• RangeSearchParamsError -> RangeSearchError (import from diskann::graph::RangeSearchError)
• Return type changed from (SearchStats, Vec<O>, Vec<f32>) to RangeSearchOutput<O> (a struct with .stats, .ids, .distances fields)
• Pass &mut () as the output buffer
• Field starting_l_value -> constructor arg starting_l (accessor: .starting_l())
• Field initial_search_slack -> constructor arg initial_slack (accessor: .initial_slack())
• Field range_search_slack -> constructor arg range_slack (accessor: .range_slack())

4. Multihop / Label-Filtered Search (multihop_search)

Before:

use diskann::graph::SearchParams;

let params = SearchParams::new(10, 100, None)?;
let stats = index
   .multihop_search(&strategy, &ctx, &query, &params, &mut output, &label_eval)
   .await?;

After:

use diskann::graph::{Search, search::{Knn, MultihopSearch}};

let knn = Knn::new(10, 100, None)?;
let params = MultihopSearch::new(knn, &label_eval);
let stats = index.search(params, &strategy, &ctx, &query, &mut output).await?;

Key differences:

• MultihopSearch wraps a Knn -> label evaluator into a single params object
• The label evaluator is part of the params, not a separate argument

5. Flat Search (unchanged method, new param type)

Before:

use diskann::graph::SearchParams;

let params = SearchParams::new(10, 100, None)?;
index.flat_search(&strategy, &ctx, &query, &filter, &params, &mut output).await?;

After:

use diskann::graph::search::Knn;

let params = Knn::new(10, 100, None)?;
index.flat_search(&strategy, &ctx, &query, &filter, &params, &mut output).await?;

Only the parameter type changed (SearchParams -> Knn).

Import Path Changes

Old	New
diskann::graph::SearchParams	diskann::graph::search::Knn
diskann::graph::RangeSearchParams	diskann::graph::search::Range
diskann::graph::SearchParamsError	diskann::graph::KnnSearchError
diskann::graph::RangeSearchParamsError	diskann::graph::RangeSearchError
—	diskann::graph::search::MultihopSearch (new)
—	diskann::graph::search::RecordedKnn (new)
—	diskann::graph::search::Diverse (new, feature-gated)
—	diskann::graph::Search (trait, re-exported)
—	diskann::graph::RangeSearchOutput (re-exported)

Change List

• copy bftrees from the snapshot location to the save location by @backurs in #783
• (RFC) Refactor search interface with unified SearchDispatch trait by @narendatha in #773
• Make queue.closest_notvisited() safe and update call sites by @arrayka in #787
• git ignore: Ignore local settings for claude code AI agent by @arrayka in #789
• Enabling flag support in codecov by @arrayka in #790
• Increase unit test coverage for diskann-tools crate by @Copilot in #763
• Neon MVP by @hildebrandmw in #777
• Adding GraphParams to be able to save graph parameters of index to SavedParams by @backurs in #786

New Contributors

• @narendatha made their first contribution in #773

Full Changelog: 0.46.0...v0.47.0

DiskANN v0.46.0

What's Changed

API Breaking Changes

• Remove the experimental_avx512 feature. by @hildebrandmw in #732
• Use VirtualStorageProvider::new_overlay(test_data_root()) in tests by @Copilot in #726
• save and load max_record_size and leaf_page_size for bftrees by @backurs in #724
• [multi-vector] Verify Standard won't overflow in its constructor. by @hildebrandmw in #757
• VirtualStorageProvider: Make new() private, add new_physical by @Copilot in #764
• [minmax] Refactor full query by @arkrishn94 in #770
• Bump diskann-quantization to edition 2024. by @hildebrandmw in #772

Additions

• [multi-vector] Enable cloning of Mat and friends. by @hildebrandmw in #759
• adding bftreepaths in mod.rs by @backurs in #775
• [quantization] Add as_raw_ptr. by @hildebrandmw in #774

Bug Fixes

• Fix diskann compilation without default-features and add CI tests. by @hildebrandmw in #722

Docs and Comments

• Updating the benchmark README to use diskann-benchmark by @bryantower in #709
• Fix doc comment: Windows line endings are \r\n not \n\r by @Copilot in #717
• Fix spelling errors in streaming API documentation by @Copilot in #715
• Add performance diagnostic to diskann-benchmark by @hildebrandmw in #744
• Add agents.md onboarding guide for coding agents by @Copilot in #765
• [doc] Fix lots of little typos in diskann-wide by @hildebrandmw in #771

Performance

• [diskann-wide] Optimize load_simd_first for 8-bit and 16-bit element types. by @hildebrandmw in #747

Dependencies

• Bump bytes from 1.11.0 to 1.11.1 by @dependabot[bot] in #723
• [diskann] Add note on the selection of PruneKind in graph::config::Builder. by @hildebrandmw in #734
• [diskann-providers] Remove the LRU dependency and make vfs and serde_json optional. by @hildebrandmw in #733

Infrastructure

• Add initial QEMU tests for diskann-wide. by @hildebrandmw in #719
• [CI] Skip coverage for Dependabot. by @hildebrandmw in #725
• Add miri test coverage to CI workflow by @Copilot in #729
• [CI] Add minimal ARM checks by @hildebrandmw in #745
• Enable CodeQL security analysis by @Copilot in #754

New Contributors

• @backurs made their first contribution in #724
• @arkrishn94 made their first contribution in #770

Full Changelog:
0.45.0...0.46.0

Rust Open Source Transition

With this release, development is planned to be moved to the open-source repository. Stay tuned for more updates!

0.7.0

Version bump 0.7.0rc2->0.7.0 (#510)

* Version bump 0.7.0rc2->0.7.0

Preparing diskannpy for 0.7.0 release (filter support, static memory indices only)

* Update pyproject.toml

the GPG key from (presumably) 2019 is no longer valid

* Update pyproject.toml

* Update python-release.yml

By default, GITHUB_TOKEN no longer has write permissions - you have to explicitly ask for it in the specific job that needs it.

We use write permissions to update the Github release action that updates the published build artifacts with the results of the release flow.

Python 0.7.0rc2

Fixing a bug in StaticMemoryIndex regarding relative paths.

Python 0.7.0.rc1

This release adds a new set of features to the diskannpy.StaticMemoryIndex to allow for categorical filtering.

The new function and method signatures are still subject to change until 0.7.0 goes out.

0.5.0.rc3 Python version republish (0.5.0.rc3.post1)

Properly releases 0.5.0.rc3 with a corrected, valid version string.

Python 0.6.1 Release

What's Changed

• Added PDoc workflow to publish github pages documentation by @jonmclean in #412
• Update README.md by @harsha-simhadri in #416
• moved ssd index defaults to defaults.h by @harsha-simhadri in #415
• Add Performance Tests by @daxpryce in #421
• allow multi-sector layout for large vectors by @harsha-simhadri in #417
• Consolidate Index Constructors by @yashpatel007 in #418
• Add convenience functions for parsing the PQ index by @PhilipBAdams in #349
• fix OLS build by @PhilipBAdams in #428
• In Memory Graph Store by @yashpatel007 in #395
• Undo mistake, let frontier read in PQ flash index be asynchronous by @harsha-simhadri in #434
• Reduce CI tests for multi-sector disk layout from 10K to 5K points so… by @harsha-simhadri in #439
• hot fix definite memory leaks by @yashpatel007 in #440
• add num_Threads to indexwriteparams in sharded build by @harsha-simhadri in #438
• Added clarity to the universal label documentation by @jonmclean in #442
• Remove IndexWriteParams from build method. by @yashpatel007 in #441
• Python type hints and returns actually align this time. by @daxpryce in #444
• working draft PR for cleaning up disk based filter search by @rakri in #414
• Fixes #432, bug in using openmp with gcc and omp_get_num_threads() by @daxpryce in #445
• Preparing for 0.6.1 release by @daxpryce in #447

Full Changelog: 0.6.0...0.6.1

Python 0.6.0 Release

0.6.0 marks our first release that is most broadly usable by the average python developer.

The API has several significant changes that prioritize a consistent and clear API.

It also has updated documentation that will be rendered to html and published @ https://microsoft.github.io/DiskANN/docs/python/0.6.0 after the release is completed.

0.5.0.rc5 Python

Includes fixes from PR #404, which either supercede #402 or include fixes for #400.

Also updates README used for PyPI publication to point to the python/README.md, not the root project's README.