Standard Grant: Vup Vault - Personal Backup, Sync & Archive with indexd

Grants Complete
19

Final Grant Progress Report: Vup Vault

What progress was made on your grant this month?

  • Ran a full disaster-recovery drill on production Sia, twice: onboard
    (browser OAuth at https://sia.storage), back up a 51-file corpus
    (symlinks, empty files, unicode names), wipe the device, vup recover
    from the 12-word paper phrase plus one OAuth click. 51/51 files restored
    SHA-256-identical.
  • Shipped the redesigned vup CLI: verb-first grammar with vault:path#snap
    references; vup backup ~/Documents docs: creates the vault, records the
    source mapping and snapshots; restore addresses any snapshot or subtree;
    copy re-wraps per-blob keys so a vault’s master key is never shared and
    warns before widening the reader set; share, automate (schedule/watch),
    doctor, and staged-vs-durable gauges in status.
  • Hardened the always-on daemon: control RPC is served only on a
    loopback-bound endpoint behind a per-run token in a 0600 lock file; the
    public endpoint never accepts control connections. Covered by an E2E.
  • Landed the identity model: cold/warm key split anchored in a DID (paper
    phrase is the cold root), multi-device enrollment, device revocation,
    capability = keyset membership.
  • All of it landed as six reviewed PRs (#7-#12 in the table below), each
    green (fmt, clippy, full test suite) at merge.

Detail tasks worked on this month per milestone with the appropriate Pull Request(s) links

Milestone Task Status PRs Notes
Foundation s5_core: identity, vaults, key derivation, blob pipeline Completed #9 Content addressing, the zstd blob pipeline, key derivation, and the cold/warm identity + vault model everything builds on
M1/M3 foundation s5_registry, s5_blobs, s5_fs_v2 Completed #10 Convergent registry with live subscriptions + ACL (the sync backbone); two-ALPN public/ACL blob server with the F02 challenge; content-defined chunking + snapshot copy (the copy/share primitive)
M5 Background daemon (vupd) Completed #12 CLI-daemon RPC over authenticated loopback; systemd/launchd via vup service install
M5 Scheduled backups Completed #12 Per-vault interval schedule + notify-driven watch mode; automated E2E
M5 Performance optimization Completed #7, #11 Streaming flush (ingest RAM 5 GB to 200 MB on a 644 MB backup); ~8x concurrent restore; slab-aligned pack sizing matched to Sia’s sector-granular pricing
M5 Vup Web improvements In progress — wasm32 groundwork landed; UI blocked on the anonymous-share model, not the layout (see challenges)
M5 Comprehensive documentation Completed #11, #12 User guide, configuration reference, CLI workflows, key-recovery, share-links, packing format
M5 Publish crates This week — Runbook complete, manifests verified; usable and reviewable now via cargo install --git, crates.io push lands this week
M4/M5 validation Live recovery drill + gate re-runs Passed #8, #11 Production sia.storage, device wipe, 51/51 verified; recovery_e2e live x2
M5 validation Watch, publish, second device Passed #12 File created to published encrypted HEAD in ~2 s (criterion 30 s); cold second device resolves the content via the shared store

Link to an easy to test version or a demo video

The <15-minute vault (assumes cargo, Rust >= 1.85):

# 1. Install the `vup` binary (~3 min compile)
cargo install --git https://github.com/s5-dev/s5-rs vup_cli

# 2. Onboard: write down 12 recovery words, Enter for the default indexer
#    (https://sia.storage), one OAuth click, accept the service install
vup onboard

# 3. Back up: creates vault "docs", records the source, snapshots
vup backup ~/Documents docs:

# 4. Verify
vup list docs:
vup status

# 5. Restore, hash-verified per block
vup restore docs: ./restored

# Disaster recovery on a blank machine: step 1, then
vup recover

Test suite:

git clone https://github.com/s5-dev/s5-rs && cd s5-rs
cargo test --workspace
# live production gate (pre-provisioned test AppKey available on request):
S5_INDEXD_TEST_APP_KEY=<hex> cargo test -p s5_node --test recovery_e2e -- --ignored

Provide an overall summary of everything achieved during this grant

  • s5_store_indexd, the core deliverable: a full S5 Store backend on the
    indexd SDK (streaming, browser-OAuth app registration, cross-AppKey
    migration). S5 applications can use Sia without renterd.
  • A storage engine matched to Sia’s cost model: content-addressed pack
    bodies recoverable without manifests, perfect-hash pack index
    (wasm32-compatible), crash-safe staging WAL, durability barriers.
  • E2EE identity and recovery: 12 paper words anchor a cold/warm key split;
    recovery from total device loss drilled live on production, twice.
  • Multi-device sync through any shared store as an E2EE relay (devices never
    need to be online together), plus enrollment and revocation.
  • Sharing via E2EE links; copy re-wraps per-blob keys, so sharing never
    exposes a vault master key.
  • The vup CLI: backup, restore, copy, share, automate, mount, doctor, with
    resume, progress, watch mode, scheduled backups, and an installable daemon
    with a local-only authenticated control plane.

If there were challenges you experienced and/or any work you weren’t able to complete, list it here

  • Vup Web is the one milestone still in progress, and the blocker is the
    sharing model rather than the UI. A no-login browser share is not cleanly
    possible on indexd today: its share primitive delegates decryption but not
    payment (/objects/:key/shared returns slab metadata, not bytes, and the
    fetcher must pay hosts from a funded account), so an anonymous browser
    cannot retrieve the data. Login-required shares — where the recipient uses
    their own funded indexer account — do work, but that is not the
    frictionless experience the web UI is meant to deliver.
  • crates.io publishing lands this week; the release runbook and verified
    manifests are ready, and the code is already usable and reviewable today
    via cargo install --git.

What lessons have you learned as a result of this grant?

  • Drills beat tests: only a live wipe-and-recover run on the production
    network gives real confidence in a recovery story; it is now a permanent
    part of the release gate.
  • Compose instead of multiplying features: every sharing shape is
    vault + copy + automation, and capability is keyset membership. One rule,
    no parallel permission system.
  • Distinguish “staged” from “durable” everywhere: a user should never trust
    a backup that has not verifiably reached the network yet.
  • This was one of the first projects where I utilized AI-assisted development tools and while they made it possible to actually fully my dream result with all the bells and whistles, they also risked overengineering scope and architecture quite frequently if not steered correctly.

What are you most proud of about your work on this grant?

Actually achieving the goal of putting all the functionality of tools like rclone and restic that I use frequently into my own CLI, with a user experience that still works!

1 Like