feat: added flush of parity-db logs after block import

[Klapeyron]

Summary

Adds a background pipeline that flushes the ledger ParityDB log after each block import, exposes a small midnight-node-ledger API to perform that flush safely, and temporarily patches several midnight-* crates to a midnight-ledger branch that provides flush_log_pipeline on the storage backend.

Motivation

ParityDB keeps work in a log pipeline; flushing it regularly after imports reduces reliance on shutdown-only behavior and aligns with durability expectations for ledger state on disk.

What changed

• node/src/ledger_log_pipeline.rs (new)

  • Subscribes to every_import_notification_stream().
  • Uses an AtomicBool plus tokio::sync::Notify so wakeups coalesce with a drain loop: each wake runs flush_requested.swap(false) until quiescent, with the actual flush in spawn_blocking.
  • Wraps the flush in catch_unwind so a panic is logged and the next import can retry.
  • Logging target: ledger-parity-db-wal (trace for queue, debug for timing, error on panic/join failure).

• ledger/src/lib.rs

  • Adds flush_log_pipeline_on_default_storage() (ledger v7 and v8 paths).

• ledger/src/versions/common/storage.rs

  • Adds flush_log_pipeline_on_default_storage_if_exists() using if let Some(storage) = try_get_default_storage::<ParityDb>() and storage.with_backend(...) so the flush holds an Arc for the whole call and avoids a TOCTOU with default_storage() if default storage is dropped concurrently.

• node/src/service.rs

  • Spawns the pipeline after spawn_tasks, with the rest of full client setup unchanged.

• Workspace dependencies

  • [patch.crates-io] in root Cargo.toml points midnight-base-crypto, midnight-base-crypto-derive, midnight-serialize, midnight-serialize-macros, and midnight-storage-core at midnight-ledger branch feat/parity_db_pipeline_flush.
  • derive-where bumped to 1.6.1; Cargo.lock updated (including duplicate console / indicatif versions from the dependency graph).

🗹 TODO before merging

• Ready

📌 Submission Checklist

• Changes are backward-compatible (or flagged if breaking)
• Pull request description explains why the change is needed
• Self-reviewed the diff
• I have included a change file, or skipped for this reason:
• If the changes introduce a new feature, I have bumped the node minor version
• Update documentation (if relevant)
• Updated AGENTS.md if build commands, architecture, or workflows changed
• No new todos introduced

🧪 Testing Evidence

Please describe any additional testing aside from CI:

• Additional tests are provided (if possible)

🔱 Fork Strategy

• Node Runtime Update
• Node Client Update
• Other:
• N/A

Links