Standard Grant: s5-indexd-js — JavaScript/TypeScript Storage Backend for indexd

Introduction

Project Name: s5-indexd-js — JavaScript/TypeScript indexd Storage Backend for Enhanced S5.js

Name of the organisation or individual submitting the proposal: Jules Lai (sole developer, founder of Fabstir)


Describe your project

s5-indexd-js is an open-source TypeScript library that implements the S5 blob storage interface against Sia’s indexd Application API, enabling any JavaScript/TypeScript application using Enhanced S5.js to persist data on the Sia network via indexd — without the application developer needing to run renterd or manage Siacoin directly. The indexd operator handles contract formation, host selection, and Siacoin funding on behalf of connected applications.

This is the JavaScript counterpart to redsolver’s s5_store_indexd Rust crate (being built under his approved Vup Vault grant). Where s5_store_indexd serves Rust-based S5 nodes and CLI tools, s5-indexd-js serves the browser and Node.js ecosystem — web applications, mobile apps via React Native, Electron desktop apps, and server-side Node.js services. Together, these two libraries ensure that the entire S5 developer community — regardless of language — has first-class indexd support on indexd’s public launch day.

The core deliverable is a TypeScript package (@fabstir/s5-indexd-js) that:

  1. Implements the S5 BlobStore interface against the indexd Application API (port 9982)
  2. Wraps @siafoundation/indexd-js (the official Sia TypeScript SDK, v0.9.0) with S5-compatible upload, download, and metadata operations
  3. Handles the OAuth-style app connection flow — registration, user approval, connect key management — with browser-friendly helpers
  4. Supports partial content retrieval (download ranges) for efficient streaming of large files
  5. Integrates seamlessly with Enhanced S5.js (@julesl23/s5js@beta) — providing an alternative storage backend to S5 portals, preserving all existing S5 protocol features (CIDs, registry, HAMT directories, encryption)

Why this matters:

Currently, JavaScript applications using S5 rely on S5 portals for blob storage. indexd provides an alternative: applications can connect directly to an indexd instance for storage operations whilst retaining S5’s content-addressing, encryption, and directory management client-side. Because indexd is significantly simpler to operate than a full S5 portal with renterd (no blockchain sync, no contract management, no slab repair), it becomes more practical for developers and organisations to run their own storage backend — or to choose from multiple indexd operators. This gives developers and their users greater optionality over where their data is stored.

This is particularly relevant for Platformless AI — the largest application currently being built on the Sia ecosystem. Platformless AI uses Enhanced S5.js for browser-side persistence operations including encrypted conversation history, vector database embeddings, document storage, AI-generated images, and agent session state. (STARK proof storage is handled by the Rust node software via s5-rs.) Platformless AI will be the first production consumer of s5-indexd-js, providing immediate real-world validation.

Enhanced S5.js is already adopted by other Sia Foundation grantees as their primary JavaScript storage client — for example, the Chi Voice grant (approved January 2026) explicitly uses Enhanced S5.js as its Sia integration layer for storing multilingual audio datasets. This establishes Enhanced S5.js as production ecosystem infrastructure with independent adoption beyond its original developer.

Example usage:

import { S5 } from "@julesl23/s5js";
import { IndexdStore } from "@fabstir/s5-indexd-js";

// Create indexd-backed storage
const store = new IndexdStore({
  indexdUrl: "https://indexd.example.com:9982",
  connectKey: "app_key_from_oauth_flow",
});

// Use with Enhanced S5.js — all existing code works unchanged
const s5 = await S5.create({
  blobStore: store,
  initialPeers: ["wss://[email protected]/s5/p2p"],
});

await s5.recoverIdentityFromSeedPhrase(seedPhrase);
await s5.fs.put("home/documents/report.pdf", fileData);
const data = await s5.fs.get("home/documents/report.pdf");

Browser OAuth flow helper:

import { connectToIndexd } from "@fabstir/s5-indexd-js";

// Opens indexd approval UI, returns connect key
const { connectKey, store } = await connectToIndexd({
  indexdUrl: "https://indexd.example.com:9982",
  appName: "My Sia App",
  appDescription: "A privacy-first notes application",
  appLogoUrl: "https://myapp.com/logo.png",
});

// connectKey is persisted by the application for future sessions
localStorage.setItem("indexd_connect_key", connectKey);

Who benefits from your project?

  • JavaScript/TypeScript developers building on S5: Every web, mobile, and Node.js application using Enhanced S5.js gains indexd as an alternative storage backend
  • The Sia Foundation: A reference JavaScript implementation that ships alongside indexd’s launch, demonstrating the SDK ecosystem’s completeness across both Rust and JavaScript
  • Platformless AI and its users: Production-validated indexd integration for the ecosystem’s largest application, storing encrypted AI conversations, vector embeddings, documents (text files, PDFs), and AI-generated images on Sia via the browser. Video generation support is planned but deferred from the MVP launch pending compliance with recent UK legislation on AI-generated video content
  • Future S5 ecosystem developers: Any application using Enhanced S5.js can adopt indexd with a single configuration change — no code rewrites
  • End users: Greater choice of storage backends means users and developers are not locked into any single portal operator

How does the project serve the Foundation’s mission of user-owned data?

s5-indexd-js advances user-owned data by giving JavaScript applications a direct path to Sia storage via indexd. Like any storage backend, an indexd operator could theoretically restrict access or go offline — but because indexd is significantly simpler to self-host than a full S5 portal with renterd (no blockchain sync, no contract management, no slab repair), it becomes much more practical for developers and organisations to run their own instance. This shifts the trust model: users and developers gain genuine optionality over who stores their data, rather than depending on whichever portal happens to be available.

Combined with Enhanced S5.js’s client-side encryption and content-addressed storage, s5-indexd-js ensures that:

  • Users own their data: Encrypted before leaving the browser, stored on Sia’s decentralised network
  • Users control their access: Can switch between indexd operators or self-host, without changing application code
  • Users verify their data: S5 CIDs provide cryptographic proof that stored data has not been tampered with
  • Developers have sovereignty: Open-source, MIT-licensed, no vendor lock-in

This library completes the user-owned data chain from browser to blockchain: Enhanced S5.js (client-side encryption + content addressing) → s5-indexd-js (direct indexd access) → indexd (contract management + Siacoin funding) → Sia hosts (decentralised storage).

Jurisdiction Compliance

  • Are you a resident of any jurisdiction on that list? No
  • Will your payment bank account be located in any jurisdiction on that list? No

Grant Specifics

Amount of money requested and justification with a comprehensive breakdown of expenses

Total Amount Requested: $39,200 USD

Category Amount Description
Developer fees $37,500 Development for 5 months ($7,500/month)
Infrastructure & testing $1,200 indexd instance for integration testing ($800), S5 node for E2E tests ($400)
Documentation $500 API documentation site

Budget justification:

  • Monthly rate reflects London-based senior developer costs and proven delivery track record (Enhanced S5.js completed 2 months ahead of schedule with all milestones exceeded)
  • 5-month timeline provides buffer for indexd release timeline alignment and thorough production testing
  • Senior-level TypeScript/S5/Sia expertise (sole developer with production S5 experience)

Timeline with measurable objectives and goals

Total Duration: 5 months

Milestone Strategy: Month 1 is dedicated to research, architecture, and design — indexd is still in active development, and rushing into implementation without deeply understanding the API surface, SDK behaviour, and browser constraints would lead to rework. Milestones 1–3 can proceed with the published @siafoundation/indexd-js SDK and API documentation (already public at indexd.sia.tech) even before indexd’s daemon source is publicly released. Milestone 4 requires access to a running indexd instance for integration testing. The 5-month timeline provides headroom for any slippage in the indexd public release schedule.

Milestone 1: Research, Architecture & Design (Month 1)

Due: ~4 weeks after grant start

Task Deliverable
indexd Application API deep study Comprehensive analysis of indexd Application API (port 9982): endpoints, request/response schemas, error codes, rate limits, authentication model. Document any gaps or undocumented behaviour
@siafoundation/indexd-js SDK exploration Hands-on evaluation of SDK v0.9.0: test all relevant methods, identify edge cases, assess browser compatibility, document any limitations
S5 BlobStore ↔ indexd API mapping Detailed design document mapping each S5 BlobStore interface method to indexd API calls, including data transformations, error handling strategy, and CID-to-object-key derivation
OAuth connection flow design Architecture for browser and Node.js authentication flows: app registration, user approval redirect, connect key receipt, secure storage, and session lifecycle
NPM package scaffold TypeScript project with ESM/CJS dual output, tree-shakeable exports, comprehensive type definitions, CI/CD pipeline, linting and test framework
indexd API client wrapper Type-safe wrapper around @siafoundation/indexd-js with error handling, retries, and timeout configuration
Architecture decision records Document key design decisions: adapter pattern, error handling strategy, retry policy, browser vs Node.js abstraction layer, connect key storage backends

Validation: Published design document covering full API mapping and architecture. NPM package scaffold builds for both browser and Node.js targets. SDK exploration report with documented findings (including any API gaps or undocumented behaviour to raise with the Foundation). indexd API client wrapper with unit tests against mocked responses.

Milestone 2: Core Storage Adapter & BlobStore Implementation (Month 2)

Due: ~8 weeks after grant start

Task Deliverable
Implement IndexdStore class implementing S5 BlobStore interface upload(), download(), exists(), delete(), stat() methods mapped to indexd Application API per Milestone 1 design
Download range support Partial content retrieval for streaming large files (maps to indexd’s download range API)
Connect key management Secure storage and rotation of indexd connect keys with configurable persistence backends (in-memory, localStorage, platform-specific secure storage)
Browser OAuth flow helper connectToIndexd() function handling app registration, user approval redirect, and connect key receipt
Node.js headless connection flow CLI-friendly authentication for server-side and CI/CD environments
Unit test suite 90%+ coverage with mocked indexd API responses

Validation: Unit tests passing against mocked indexd API. BlobStore interface contract tests (upload blob, retrieve by CID, verify content hash matches). All methods implemented per design document. Browser OAuth flow and Node.js headless connection flow tested.

Milestone 3: Enhanced S5.js Integration (Month 3)

Due: ~12 weeks after grant start

Task Deliverable
Enhanced S5.js integration layer Drop-in blobStore provider: S5.create({ blobStore: new IndexdStore(...) }) using @julesl23/s5js@beta
Integration tests with Enhanced S5.js Full path-based API tests: put(), get(), list(), delete() through IndexdStore backend
HAMT directory operations via indexd Verify HAMT sharding and directory operations work correctly with indexd backend
Media pipeline validation Verify Enhanced S5.js media operations (thumbnail generation, metadata extraction, colour extraction) store and retrieve correctly via IndexdStore backend

Validation: Integration test suite: create S5 instance with IndexdStore, perform all FS5 operations (file CRUD, directory creation, HAMT sharding at 1,000+ entries, encryption/decryption cycle, media pipeline). All 280+ existing Enhanced S5.js tests pass with IndexdStore backend swapped in.

Milestone 4: Production Hardening & Real indexd Testing (Month 4)

Due: ~16 weeks after grant start

Task Deliverable
E2E testing against live indexd instance Full test suite running against self-hosted indexd instance (or early-access instance if granted by Foundation)
Concurrent upload/download optimisation Configurable parallelism, connection pooling, upload chunking for large files
Quota management integration Respect and surface indexd per-app quotas; graceful handling of quota exceeded errors
Error recovery and resilience Automatic retry with exponential backoff, network interruption recovery, partial upload resume
Performance benchmarking Benchmarks for upload/download throughput, HAMT operations, and batch workflows via IndexdStore
NPM publication Publish to npm as @fabstir/s5-indexd-js with semantic versioning

Validation: E2E test suite passing against live indexd. Performance benchmark report published. Upload resume tested (interrupt at 50%, resume, verify complete file). Package published to npm.

Milestone 5: Platformless AI Integration, Documentation & Demo (Month 5)

Due: ~20 weeks after grant start

Task Deliverable
Platformless AI integration Integrate IndexdStore into Platformless AI’s browser SDK for encrypted conversation persistence, vector embedding storage, document storage, AI-generated image storage, and agent session state
Production validation Validate with real Platformless AI workloads at scale — concurrent users, large conversation histories, document uploads, image generation, batch vector operations
API documentation TypeDoc-generated reference with usage examples for every public method
Developer tutorial “Build a Privacy-First App on Sia with Enhanced S5.js and indexd” walkthrough
Example application Working example: React app with OAuth flow and file management via indexd
Community demo Live demonstration at Sia Foundation monthly community call showing Platformless AI running on indexd via s5-indexd-js
Final polish and release Address any issues discovered during production integration; publish stable v1.0.0 release
Upstream contribution Submit PR to Enhanced S5.js adding IndexdStore as optional built-in backend

Validation: Platformless AI successfully storing and retrieving encrypted conversations, vector embeddings, documents, and AI-generated images via IndexdStore in production. Documentation site deployed. Example application functional. Demo delivered at community call. Stable v1.0.0 published to npm.

High-level architecture overview and security best practices

Architecture:

┌─────────────────────────────────────────────────────────────┐
│                    Application Layer                         │
│                                                              │
│   Enhanced S5.js (path-based API, CIDs, HAMT, encryption)   │
│   (@julesl23/s5js@beta)                                     │
│                                                              │
├──────────────────────┬──────────────────────────────────────┤
│   BlobStore Interface│                                      │
│                      │                                      │
│  ┌─────────────┐   ┌┴────────────────┐   ┌──────────────┐ │
│  │ PortalStore │   │   IndexdStore    │   │  LocalStore   │ │
│  │ (existing)  │   │   (NEW - this    │   │  (testing)    │ │
│  │             │   │    grant)        │   │               │ │
│  └──────┬──────┘   └───────┬─────────┘   └──────────────┘ │
│         │                  │                                │
└─────────┼──────────────────┼────────────────────────────────┘
          │                  │
          ▼                  ▼
   ┌──────────┐     ┌──────────────┐
   │ S5 Portal│     │    indexd     │
   │ (HTTP)   │     │  (REST API   │
   │          │     │  port 9982)  │
   └──────────┘     └──────┬───────┘
                           │
                    ┌──────▼───────┐
                    │  Sia Network │
                    │  (hosts)     │
                    └──────────────┘

Data flow:

  1. Application calls s5.fs.put("home/file.txt", data)
  2. Enhanced S5.js encrypts data client-side (XChaCha20-Poly1305)
  3. Enhanced S5.js computes Blake3 content hash (CID)
  4. IndexdStore uploads encrypted blob to indexd via Application API
  5. indexd manages slab encoding, host selection, contract formation, and Siacoin funding
  6. Sia hosts store erasure-coded data
  7. CID and metadata stored in S5 registry (for directory structure)

Security practices:

  1. No plaintext leaves the client: All encryption happens in Enhanced S5.js before reaching IndexdStore — indexd only ever receives encrypted blobs
  2. Content verification: Every download is verified against its Blake3 CID hash; tampered data is rejected
  3. Connect key security: Keys stored via configurable backend (in-memory for ephemeral sessions, localStorage or platform-specific secure storage for persistent apps, never in plaintext config files)
  4. No credential leakage: OAuth flow uses URL fragments for sensitive data (not sent to servers); connect keys are scoped per-application
  5. Dependency minimisation: Minimal dependency tree; @siafoundation/indexd-js as sole external Sia dependency
  6. Supply chain security: Package published with provenance attestations; npm audit clean

Potential risks that will affect the outcome of the project

Risk Probability Impact Mitigation
indexd public release delayed beyond Month 4 Medium Medium Milestones 1–3 are fully functional with mocked API and published SDK docs. Only M4 requires live indexd. 5-month timeline provides buffer. I would request early access to the indexd repository (as was granted to redsolver for his Vup Vault grant) or self-host from available binaries
indexd Application API changes before stable release Medium Low Abstract all API calls through a thin adapter layer; changes require updating adapter only, not consumer code
@siafoundation/indexd-js SDK breaking changes Low Low Pin SDK version; adapter pattern isolates breaking changes. TypeScript types catch mismatches at compile time
Browser CORS restrictions with indexd Medium Medium Document required CORS configuration for indexd operators; provide proxy middleware for development. Coordinate with Sia Foundation on default CORS headers
Enhanced S5.js BlobStore interface needs extension for indexd-specific features Low Low Propose interface extensions upstream; maintain indexd-specific extensions as optional capabilities

Development Information

Will all of your project’s code be open-source?

Yes, MIT licence. Published as an npm package available to the entire S5 and Sia developer community.

Leave a link where code will be accessible for review

Do you agree to submit monthly progress reports?

Yes, following Foundation format with PR/commit links per task, as demonstrated throughout my Enhanced S5.js grant (monthly reports submitted consistently with detailed milestone tracking).

Do you agree to designate a point of contact for committee questions and concerns?

Yes, myself (Jules Lai).

Provide links to previous work or code from all team members

Completed Sia Foundation Grant:

Platformless AI (production Enhanced S5.js consumer):

Previous Sia Foundation Grants:

  • Fabstir Web3 Media Player (Standard Grant — Completed)

Have you developed a proof of concept for this idea already?

Yes — in the sense that the foundational pieces are all in place:

  1. Enhanced S5.js is the production SDK this library integrates with — I built it, I know the BlobStore interface intimately, and I understand exactly what needs to be adapted for indexd.

  2. Platformless AI is already storing encrypted conversations, vector embeddings, documents, and AI-generated images via Enhanced S5.js on S5/Sia. This grant adds indexd as the storage backend, and Platformless AI will be the first production consumer — providing immediate real-world validation.

  3. The indexd SDK and API documentation are already published (@siafoundation/indexd-js v0.9.0, @siafoundation/indexd-types v0.12.0, API docs at Indexd Application API). The interface mapping from S5’s BlobStore to indexd’s Application API is well-understood from studying both APIs.

Do you agree to participate in a demo at our monthly community call at significant milestones or after the grant’s completion?

Yes. The final milestone (Month 5) includes a live demo of Platformless AI running on indexd via s5-indexd-js.


Summary of Work Completed in Previous Grant

Enhanced S5.js (Standard Grant — Complete):

Deliverable Status Result
Path-based API (get/put/delete/list) :white_check_mark: Complete 10x simpler developer experience
CBOR serialisation (DirV1 spec) :white_check_mark: Complete Cross-platform compatibility with s5-rs
HAMT sharding :white_check_mark: Complete O(log n) at 1,000+ entries, tested to 100K+
Media processing (WASM + Canvas) :white_check_mark: Complete Thumbnails, metadata, colour extraction
Advanced CID API :white_check_mark: Complete 74 dedicated tests
Directory walker & batch operations :white_check_mark: Complete Recursive traversal, progress tracking
Connection API (mobile) :white_check_mark: Complete Status monitoring, reconnection
Documentation & upstream integration :white_check_mark: Complete 500+ lines API docs, mdbook format

Performance: 60.09 KB compressed (10.6x under 700 KB requirement). 280+ tests passing. ~800ms per S5 operation on real network.

Delivery: Completed 2 months ahead of schedule. Multiple milestones delivered early. Monthly progress reports submitted consistently with detailed PR/commit links.


Why This Grant, Why Now

indexd is the Sia Foundation’s flagship product launching in Q1 2026. The SDK ecosystem already includes Go (indexd-sdk), Rust (sia-sdk-rs), and JavaScript (@siafoundation/indexd-js) packages — but no S5 integration exists for the JavaScript ecosystem. Redsolver is building the Rust-side S5 integration (s5_store_indexd) under his approved Vup Vault grant. This proposal completes the picture for JavaScript.

Platformless AI — launching as the largest application on the Sia ecosystem — will be the first production consumer, storing encrypted AI conversations, vector embeddings, documents, and AI-generated images at scale via the browser. This provides the Foundation with a high-profile reference implementation demonstrating indexd’s capabilities from day one. Enhanced S5.js already has independent ecosystem adoption: the Chi Voice grant (approved January 2026) uses it as its primary Sia storage client, demonstrating demand for JavaScript-side S5 tooling beyond the original developer.

The timing is optimal: indexd’s SDK and API documentation are published, the TypeScript types package is at v0.12.0, and the daemon’s Q1 2026 release window aligns with this grant’s development timeline.

This grant also establishes a second independent maintainer with deep knowledge of the S5 JavaScript ecosystem, strengthening the Foundation’s long-term risk mitigation for this critical infrastructure layer.


Contact info

As part of Platformless AI, I am exploring a custom ZK chain using Polygon CDK in validium mode, where batch transaction data needs to be stored off-chain but provably available. Typically this uses a trusted Data Availability Committee (DAC), but using Sia via s5-indexd-js enables a cryptographically stronger alternative: the DA adapter calls Enhanced S5.js — configured with s5-indexd-js as its BlobStore backend — which stores batch data on Sia via indexd and returns an S5 CID as the DA commitment. The DA adapter itself has no knowledge of indexd; the entire indexd integration is transparently handled by s5-indexd-js, demonstrating how any application built on Enhanced S5.js gains indexd as a storage backend with a single configuration change. This also makes the DA node operationally practical — because indexd abstracts contract formation, host selection, and Siacoin funding, a DA node operator needs no knowledge of renterd or Siacoin management to store batch data on Sia.

A note on the JavaScript/TypeScript implementation approach

A technical question was raised about whether a pure JavaScript/TypeScript implementation of s5-indexd-js is justified, given that Rust-based WASM bindings could theoretically serve the same purpose. I wanted to address this directly for the committee.

There is currently no official JavaScript SDK for indexd. The only SDK in the SiaFoundation repository is a Python implementation built on Rust bindings. s5-indexd-js would be the first and only JavaScript/TypeScript path to indexd for web, mobile, and Node.js developers; filling a genuine gap rather than duplicating existing work.

On the pure JS versus WASM question, there are several concrete reasons why pure TypeScript is the right choice for this specific use case:

Bundle size — Enhanced S5.js is around 60KB. A compiled Rust WASM binary for equivalent functionality would be 1–3MB. In mobile and low-bandwidth contexts that difference is a real barrier to adoption, not a minor inconvenience.

React Native — WASM support in React Native is inconsistent and in many configurations effectively unusable. Pure TypeScript works identically across browser, Node.js, and React Native with no special handling.

Agentic AI process overhead — Platformless AI’s architecture includes AI agents calling other agents, each isolated in their own process space. In that pattern WASM overhead isn’t paid once, it’s paid per agent per invocation. At 1–3MB and potentially 300–800ms compile time per process, that compounds significantly when orchestrating dozens of concurrent agents. A 60KB TypeScript module with ~5ms load time is a meaningful architectural advantage.

Debugging — WASM produces opaque stack traces in production. TypeScript gives developers readable source maps, named functions, and the full browser and Node.js debugging toolchain. For a storage library integrated into complex production applications this matters day to day.

On encryption performance — ChaCha20 is not part of the Web Crypto API, which is a fair concern. However s5-indexd-js delegates encryption entirely to Enhanced S5.js, which already handles XChaCha20-Poly1305. In practice, Platformless AI uses Enhanced S5.js for large images and PDF documents well over 10MB with no noticeable performance issues — the dominant processing time is OCR, vision, generation, and RAG embedding operations, not encryption. For developers whose primary use case is frequent encryption of very large files, the Rust WASM implementation remains an option. The two approaches complement rather than compete with each other.

On scope — s5-indexd-js simply consumes the indexd Application API, the same public REST API available to any developer. It is a BlobStore adapter, the glue between Enhanced S5.js and indexd. There is nothing being reimplemented on the Foundation’s side and no maintenance burden placed on them. Developers already using Enhanced S5.js, including other Sia Foundation grantees such as Chi Voice, gain indexd as a storage backend with a single configuration change, staying entirely within their existing TypeScript stack.

A further clarification on client-side cryptography

A follow-up question was raised about whether a pure JavaScript implementation can handle the client-side cryptography required to keep the storage backend trustless.

The client-side crypto requirement is the same regardless of backend. Enhanced S5.js already handles all of it: encryption, key management, Ed25519 signing, CID verification. That doesn’t change whether the backend is an S5 portal with renterd or indexd via s5-indexd-js. The trustless model is inherent to S5, not to the choice of storage backend or implementation language. s5-indexd-js operates as a storage backend within Enhanced S5.js, so the crypto layer remains entirely under Enhanced S5.js’s control, unchanged. A Rust WASM approach would face exactly the same client-side crypto requirements.

To summarise the discussion: s5-indexd-js fills a genuine gap, there is no official JavaScript SDK for indexd. Pure TypeScript is the right choice for this specific use case given bundle size, React Native compatibility, agentic AI process overhead, and debugging considerations. The two approaches, pure TypeScript and Rust WASM, complement rather than compete with each other, serving different workloads and developer environments.

I hope this gives the committee the full picture ahead of Tuesday’s review.