---

starvector-rs

Rust crate for local StarVector inference on Candle.

Supported Matrix

• Models: starvector-1b-im2svg, starvector-8b-im2svg
• Weight formats: safetensors, GGUF
• Devices: CPU, CUDA (manual acceptance)

Effective matrix:

• 1B + safetensors
• 1B + GGUF
• 8B + safetensors
• 8B + GGUF

Current Runtime Split

• 1B: CLIP vision + BigCode decoder + BatchNorm adapter
• 8B: SigLIP vision + StarCoder2 decoder + LayerNorm adapter
• GGUF path uses quantized runtime for both 1B and 8B.

Build

cd D:\starvecntor\mk\starvector-rs
cargo build

CUDA build:

cd D:\starvecntor\mk\starvector-rs
cargo build --features cuda

CLI Usage

1B safetensors:

cargo run -- infer --model-dir ..\..\models\starvector-1b-im2svg --image ..\..\star-vector\assets\examples\sample-18.png --device cpu --max-new-tokens 64

8B safetensors:

cargo run -- infer --model-dir ..\..\models\starvector-8b-im2svg --image ..\..\star-vector\assets\examples\sample-18.png --device cpu --max-new-tokens 64

8B GGUF:

cargo run -- infer --model-dir ..\..\models\starvector-8b-im2svg --weights-gguf ..\..\models\starvector-8b-im2svg-full-q4_0.gguf --image ..\..\star-vector\assets\examples\sample-18.png --device cpu --max-new-tokens 64

CLI contract:

• Prints SVG to stdout.
• Writes SVG to file when --output is provided.
• Uses </svg> stop sequence.
• --device cuda is equivalent to --device cuda:0.
• If binary is built without cuda feature, --device cuda[:idx] fails explicitly.

Test Matrix

Default (cargo test, no env):

• No GPU required.
• No Python required.
• No local model artifacts required.
• Includes metadata/profile/mode selection and typed error mapping checks.

Opt-in local model tests:

• STARVECTOR_RUN_LOCAL_MODEL_TESTS=1
• Optional model path overrides:
  • STARVECTOR_MODEL_DIR
  • STARVECTOR_1B_MODEL_DIR
  • STARVECTOR_8B_MODEL_DIR

Opt-in Python parity tests:

• STARVECTOR_RUN_LOCAL_MODEL_TESTS=1
• STARVECTOR_RUN_PYTHON_ORACLE=1

Opt-in GGUF smoke:

• STARVECTOR_RUN_1B_GGUF_CPU=1
• STARVECTOR_RUN_1B_GGUF_CUDA=1
• STARVECTOR_RUN_8B_GGUF_CPU=1
• STARVECTOR_RUN_8B_GGUF_CUDA=1
• STARVECTOR_1B_GGUF=<path>
• STARVECTOR_8B_GGUF=<path>

Opt-in safetensors smoke:

• STARVECTOR_RUN_SAFE_1B_CUDA=1
• STARVECTOR_RUN_SAFE_8B_CPU=1
• STARVECTOR_RUN_SAFE_8B_CUDA=1

GGUF-First Manual Acceptance

Recommended manual flow before claiming full 8B support:

1. 8B GGUF CPU load + short generate smoke
2. 8B GGUF CUDA load + short generate smoke
3. 8B safetensors CPU load + short generate smoke
4. 8B safetensors CUDA load + short generate smoke
5. Explicit mismatch checks:
   • 8B model_dir + 1B GGUF
   • 1B model_dir + 8B GGUF

CUDA acceptance is manual and machine-dependent. It is not required for default CI-like test execution.

Verification Commands

cd D:\starvecntor\mk\starvector-rs
cargo fmt --check
cargo clippy -- -D warnings
cargo test
cargo build

License

Apache-2.0