Standard Grant: S5 Network 2024

S5 Progress Report (December)

What progress was made on your grant this month?

  • The designer has been busy working on different components, concepts/views and utilities for the Vup Web UI, while I gave some feedback. Vup Web is slightly behind schedule now, but should finally get a nice MVP ready this month with all basic file management features working

  • Bluesky now has over 26M users which also resulted in significantly more load on my feed infrastructure and kept me quite busy putting out fires. So this month I built a new powerful open-source indexer in Rust to keep up with the network growth and have less issues in the future. The new indexer will also play a crucial role in bridging the AT Protocol and Bluesky data and media over to the S5 protocol as part of the archiver milestone planned later in 2025 for this grant

  • Attended the 38th Chaos Communication Congress and had some interesting discussions there around S5, the AT Protocol and other projects building decentralized protocols and networks

Links to repos worked on this month:

What will you be working on next?

  • Use the first finished design components to build the Vup Web UI with all basic features implemented and working

  • Continue working on the S5 TS/JS library and implementing all v1 specs

  • Design a landing page for the S5 protocol which gives a quick overview over which features are possible with S5 v1 and onboards developers to use it in their project (to be deployed on https://s5.pro)

Hello,

Thank you for your progress report!

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

Timeline Correction

The timeline listed in the second post of this thread is off by one month due to the grant taking a few weeks to be approved and onboarded, which resulted in the official timeline in the grant agreement being shifted by one month. Unfortunately I can’t edit the timeline in this thread anymore due to it being the second post (instead of the first), so instead I’m just posting the correct timeline here for reference:

Milestones and Timeline

Finish the S5 v1 specifications (2024-09-02)

Design, define and write v1 specifications for the different S5 data structures and features:

  • Blobs, already done: Blobs - S5 Network Docs
  • Registry
  • Encryption
  • File System (including metadata formats)
  • Identity System
  • Streams

In July, I will attend IPFS Camp 2024 (https://2024.ipfs.camp/) for exchanging ideas with the wider IPFS ecosystem and getting feedback on the final S5 specs, to make them the best they can be.

Adapt the S5 Dart and Flutter libraries to the new v1 specifications (2024-10-02)

In August and likely already earlier while working on the specs, I will adapt the existing Dart/Flutter libraries for S5 to use the new specifications. This will be very important for validating if everything works as expected and adding support for S5 to different apps like GitHub - LinwoodDev/Butterfly: 🎨 Powerful, minimalistic, cross-platform, opensource note-taking app (in collaboration with the developers), to see if the APIs are well-designed and the data structures efficient.

TypeScript Web Library (2024-11-02)

Build a TypeScript/JS library based on spec v1 in collaboration with @pcfreak to ensure the spec is sound and make all S5 features available for the most popular development ecosystem (the web and native JS). The libraries will use Rust via WASM for performance-critical things like hashing, signing and encryption (just like in the Dart libraries).

The TypeScript library will be designed to “just work” (like the s5 dart package), so devs can easily add it to their app to get features like a decentralized end-to-end-encrypted file system (compatible with Vup) and real-time communication between multiple users.

Build Vup Web (2025-01-02)

Using the new TypeScript libraries, build a lightweight web version of the Vup Cloud Storage app (fully compatible with the native version of course). This will also be the first implementation of the new user-friendly Vup UI designed by a professional designer.

Features

  • View and download shared directories and files (both public and encrypted)
  • Sign in with a S5 Identity
  • Upload files and manage the encrypted file system
  • Media Thumbnails

RHP4 support for S5 web apps (when available)

From https://sia.tech/roadmap:

RHP4 will also enable storage consumers to download and upload data directly in the browser without installing additional software.

When this feature lands in Sia, I will add it to the web-based and native S5 libraries (like TypeScript and Dart), to support more efficient direct downloads from the Sia Network.

Improvements to the S5 Node needed for Vup v1 (2025-01-02)

  • Improved Pinning API (for example background mass-pinning)
  • More powerful Account and Admin APIs, to make managing larger nodes with many users easier
  • Performance and scalability improvements

Get Vup to version v1 (native) (2025-03-02)

  • Update the native Vup App to match the new professional UI design of the web app
  • Migrate all data structures to the S5 specification v1 and the new libraries
  • Advanced two-way sync with file deletion and multi-device support
  • Advanced file sharing permission systems (for example read-only/write per file)
  • Add built-in audio player
  • Release on Linux, Windows, macOS, Android and iOS

Vup Collaborative Productivity apps (2025-04-02)

Webxdc (https://webxdc.org/) is a standard to bring privacy-friendly web apps to messenger chats in a simple file format containing HTML5, CSS, JavaScript and other asset files. But apart from that use case, this is also the ideal format to provide many different collaborative web apps for Vup (Web and Native) and other S5-powered apps. S5 can both be used as the transport layer (instead of a chat room) for real-time collaboration like drawing on a whiteboard or editing text, and it can also store files you edited permanently in the S5 file system. There are already many Webxdc apps made for https://delta.chat and other messengers and most of them should just work with Vup+S5.

From the Webxdc specification:

All authentication, identity management, social discovery and message transport is outsourced to the host messenger which runs a webxdc app container file and relays application update messages between app users, letting each app inherit offline-first and end-to-end encryption capabilities implemented by the hosting messenger.

In this case, the Vup app is the “hosting messenger”!

Some use cases possible with this:

  • Select a text file in your Vup App and share it for end-to-end-encrypted collaborative editing with other users by simply sharing a link with them. They don’t need to install anything due to the app being web-based and just need to open the link in their browser and start editing with you! In regular intervals, your local Vup App will persist the changes to your encrypted file system.
  • Same with task list editing, whiteboard apps, calendars, board games and more! Any productivity app that runs on the web and could use a file system.

The biggest advantage of this approach over just implementing collaborative productivity apps/features myself in Vup is that it saves me a ton of work and also makes the whole platform easily extensible by third-party developers with new and exciting apps.

Vup P2P (2025-05-02)

Add Iroh support (https://iroh.computer/) to Vup for keeping true p2p connectivity between multiple devices and users. (only trusted devices by default)

Use Cases

  • You want to stream a video file and one of your other devices already has it cached locally. Instead of needing to reach out to the Internet, your local Vup app will just connect directly to your other device and stream the file from there
  • If enabled, you can connect directly with other users you are editing a text file (or something else, see Webxdc above) with for improved latency and privacy (no server involved)

Archiver (2025-08-02)

Archiving makes sense for personal archive use cases, for preserving history and satisfying your data hoarder needs. It’s also a fantastic way to onboard massive amounts of useful data to the network.

The existing S5 Archiver will be extended to support all the following data sources natively:

  • Websites
  • YouTube Videos
  • AT Protocol data repositories and blobs (for example the entirety of Bluesky data)
  • GIT repositories
  • Twitch VODs and media.ccc.de

After that’s done, I will deploy a large Sia storage cluster with multiple S5 Nodes. This archival infrastructure will then be used to archive massive amounts of Creative-Commons-licensed YouTube Videos, different websites and CommonCrawl archives. It will serve as both a scalability test of the software and network, and as a useful archive.

All parts of the archive with a public license that permits it (like Creative Commons), will be publicly browsable using a S5-powered web app.

1 Like

S5 Progress Report (January)

What progress was made on your grant this month?

Links to repos worked on this month:

What will you be working on next?

  • Finish implementing full identity and account support in the S5 TypeScript library (currently ~70% done)

  • Continue working on Vup Web after the next components from the designer are available

  • Design a landing page for the S5 protocol which gives a quick overview over which features are possible with S5 v1 and onboards developers to use it in their project (to be deployed on https://s5.pro)

S5 Progress Report (February)

What progress was made on your grant this month?

  • Implemented pretty much all missing parts in the S5 TypeScript library
    • full identity implementation, with seed phrase generation, verification and recovery, key derivation
    • added identity-backed api, including upload, portal account registration and login
    • file system: added full identity support, plaintext upload, directory and file creation, home file system
    • registry write support
    • hidden db implementation for legacy compat with Vup portal accounts
    • added example script which mirrors all apps from the webxdc app store to a new directory in the s5 file system
  • My designer made solid progress on the Vup Web design components, quite a lot of new ones were added to the repository
  • Started sketching some ideas on how the S5 Protocol landing page could look like and speaking with @Julian who will work on it (I’ve been already working together with him in 2020 and 2021 for different Skynet apps (like for example the https://vup.app/ website), so really looking forward to doing it again).

Links to repos worked on this month:

What will you be working on next?

  • Add tests to the S5 TypeScript library in order to ensure everything is implemented correctly
  • Get the first proper release of Vup Web v1 out. The TypeScript library now already includes everything needed so that shouldn’t cause any delays, and all of the different components and designs are also finally coming together
  • In Vup Web, implement initial support for browsing and using webxdc apps (this is the first step for collaborative productivity apps)
  • Initiate work on the S5 landing page
  • Get the existing native Flutter-based Vup to version 1 and add features like advanced two-way sync (this milestone is now delayed due to Vup Web taking longer than expected)
1 Like

Due to poor time management on my part, I am unfortunately unable to publish my progress report for March on time today. I would like to apologize for this and will post it as soon as possible (no later than April 5th)

Hi @redsolver, thanks for the update. What’s the current status of your report?

S5 Progress Report (March)

What progress was made on your grant this month?

Links to repos worked on this month:

What will you be working on next?

  • Implement sharing in Vup Web, polish the user experience and add more features

  • In Vup Web, implement initial support for browsing and using webxdc apps (this milestone is now delayed)

  • Publish the new S5 landing page

  • Get the existing native Flutter-based Vup to version 1 and add features like p2p connectivity and advanced two-way sync

Hello,

Thank you for your progress report!

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

For reference, here are the current estimated completion dates for my milestones:

  • Collaborative Productivity apps (delayed to 2025-05-02)
  • Get Vup (Flutter) to version 1 (delayed to 2025-06-02)
  • Vup P2P (delayed to 2025-06-02)
  • Archiver (2025-08-02, still as planned)

This is not my progress report, the April report will be posted on 2025-05-02 as usual