Standard Grant: S5 Gateway & TypeScript Client

Grants Complete
15

Introduction

Project Name:
Enhanced s5.js – Advanced Directory Handling and Media Support

Applicant:
Jules Lai (sole developer, founder of Fabstir)

Describe your project

The official s5.js library provides fundamental S5 operations, but developers need higher-level features for efficient application development:

  • Simple path-based storage: “Store this data at /profile/settings”
  • Media handling: “Generate thumbnails for all images in /gallery”
  • Reliable updates: “Handle concurrent writes safely”

This grant adds these developer-friendly features directly into s5.js, making it easier to build robust applications on S5.

Planned enhancements

  1. Path-based API — getByPath(rootKey, path) / putByPath(rootKey, path, data).
    Traverses nested FS5 manifests, fetches or updates the target blob, and bubbles changed CIDs up to the root with a single registrySet.

  2. LWW-register wrapper — optional Last-Writer-Wins conflict-resolution around registry updates using the revision counter as timestamp.

  3. Thumbnail & media metadata pipeline — JPEG /WebP previews, basic EXIF/size detection and helper that returns { thumbnailCID, width, height, mime }.

  4. Directory utilities — Directory walker with optional limit / cursor pagination; integrates in-memory or IndexedDB manifest cache.

  5. FS5 sharding groundwork — implement the HAMT/B-Tree header fields marked TODO and add split / merge helpers so very large directories can be handled once the spec finalises.

These enhancements complement the existing s5-dev/web-proxy project, which provides client-side gateway functionality with verified streaming. While the web-proxy handles content delivery, my additions to s5.js will enable better directory browsing and media handling within that environment. The thumbnail pipeline and directory utilities are particularly valuable for web-proxy users, as they enable efficient browsing of large media collections directly in the browser. This maintains the web-proxy’s commitment to client-side verification whilst adding rich media capabilities that users expect from modern web applications.

All code will be submitted as pull requests to the upstream s5.js repository; a temporary fork lives under Fabstir until each PR is merged.

Who benefits?

Audience Benefit
JS/TS Web-3 developers Simple get/put path semantics and thumbnails instead of hand-crafting FS5 blobs.
S5 core library Gains high-level utilities without fragmenting into third-party forks.
Fabstir ecosystem Can store app bundles and user media with one call, paving the way for a decentralised app store.
Sia community Higher-level tooling encourages broader adoption of S5 + FS5.

Mission alignment

By making S5 easier to build on, more users will self-host data on Sia while retaining cryptographic control of their CIDs and registry keys—directly serving the Foundation’s goal of user-owned data.

Compliance

Resident of prohibited jurisdiction? No
Payment bank in prohibited jurisdiction? No

Grant Specifics

Amount requested

49 600 USD — paid in equal 6 200 USD monthly tranches (standard Foundation schedule).

Timeline & milestones (eight months)

Month Target date Milestone & deliverables Budget (USD)
1 7/2/25 Project set-up & design Repos, branching strategy, FS5 test fixtures; code-contribution guidelines & project board; tool-chain investigation; one-off business overhead 6 200
2 8/2/25 Path helpers v0.1 — basic get/put for single-level directories, unit tests 6 200
3 9/2/25 Path-cascade optimisation — multi-level update with single registrySet; docs & examples 6 200
4 10/2/25 WASM foundation & basic media — pipeline setup, code-splitting, image metadata extraction, baseline performance recorded 6 200
5 11/2/25 Advanced media processing — building on Month 4: JPEG / PNG / WebP thumbnails (≤ 64 kB average), progressive rendering, browser test matrix; bundle ≤ 700 kB compressed 6 200
6 12/2/25 Directory utilities & caching — walker with limit/cursor pagination, IndexedDB/in-memory cache, filtered listings, performance benchmarks 6 200
7 1/2/26 Sharding groundwork — HAMT header fields, split/merge helpers, integration tests 6 200
8 2/2/26 Documentation site update, demo scripts, screencast, forum feedback buffer, PRs merged upstream 6 200
Total 49 600 USD

The Foundation disburses 6 200 USD per month; internal line items show expected effort but may vary slightly while the total remains capped at 49 600 USD.

Technical Implementation Details

Media Processing Strategy

  • Primary Implementation (Month 4):

    • WASM module setup with code splitting
    • Lazy loading for optimal bundle size
    • Basic image metadata extraction
    • Browser compatibility testing suite
    • Performance baseline establishment

  • Advanced Features (Month 5):

    • Thumbnail generation with quality/size options
    • Support for JPEG, PNG, WebP formats
    • Progressive loading for large images
    • Fallback = skip thumbnail if local WASM fails
    • Comprehensive browser testing

Caching Implementation

  • Tiered Storage Approach:

    • In-memory LRU cache for frequent access
    • IndexedDB for persistent storage
    • Configurable cache size limits
    • Automatic cache invalidation
    • Browser storage quota management

  • Performance Targets:

    • Sub-100ms access for cached items
    • Efficient bulk operations
    • Minimal memory footprint
    • Graceful degradation strategy

Success Indicators

  • putByPath updates a deep file and results in exactly one registrySet.
  • Concurrent writes resolve correctly under the LWW wrapper (demonstrated in tests).
  • Thumbnail helper generates a preview ≤ 64 kB average size (95 th percentile) and correct MIME metadata for PNG, JPEG, WebP.
  • Directory walker lists 10 000 cached entries in ≤ 2 s on a typical laptop.
  • Thumbnail generation completes in ≤ 500 ms for a 1 MP JPEG on a mid-range laptop; larger images scale linearly.

Risks & Mitigation

Risk Mitigation
WASM bundle size exceeds targets Keep the thumbnail module ≤ 700 kB compressed
Browser compatibility issues Comprehensive testing matrix, graceful degradation paths, polyfills where needed
Thumbnail processing performance Progressive quality settings, worker thread implementation, cancel stale operations
Cache storage limitations Implement LRU eviction, respect browser quotas, clear warning when approaching limits
Fallback service availability Local processing first, clear error handling

*Fallback = If local WASM fails, thumbnail generation is skipped; no external upload is attempted.

Development Information

  • All code MIT-licensed.
  • Upstream PR target: https://github.com/s5-dev/s5.js (temporary fork at https://github.com/Fabstir/s5.js).
  • Existing helper libs: s5-encryptWasm, multiformats, sharp-wasm.
  • Monthly progress reports in the Sia Forum.

Community engagement & maintenance

Area Commitment
Progress updates Monthly forum posts.
Code reviews PRs opened early; iterate with maintainers.
Issue handling Best-effort review; no formal SLA.
CI GitHub Actions: lint, unit + browser tests.

Identity & Billing

Storage payments remain the S5-portal operator’s responsibility; no billing layer added.

References

Declaration

I certify that the information above is accurate and that all deliverables will be released under the MIT licence.

Contact info

1 Like