Standard Grant: Sluby - Developer-First Video Infrastructure on Sia [Resubmission of SiaStream]

Grants Approved
6

Sluby, Progress Report 1 (June, Milestone 1)

Repo: GitHub - justmert/Sluby · GitHub

Milestone 1, Core Backend and Sia Storage Pipeline (due June 25): done.

What progress was made this month?

  • Milestone 1 is finished: I upload a video over TUS, FFmpeg transcodes it into four HLS renditions, every output is stored and pinned on Sia, and the object IDs are tracked in Postgres. Ran the full path on the Zen testnet, where one test video gives about 10 to 12 pinned objects, all visible on Siascan.

  • HLS uses FFmpeg single_file mode, so each rendition is one .m4s the playlist reads by byte range. That drops per-video Sia uploads from a few hundred to about N+2, and playback pulls only the bytes it needs.

  • Every Sia object ID maps back to its asset, rendition, and role (renditions and artifacts tables), which the storage breakdown and later cleanup build on.

  • The whole stack runs from docker compose up (backend, Postgres, Redis, Nginx).

Tasks this month

Solo repo, so work lands as commits on main.

Milestone Task Commit(s)
1 Backend with embedded TUS resumable uploads f83414a
1 FFmpeg fMP4 HLS, single_file, four renditions (1080p/720p/540p/360p) f83414a, 72fe226
1 Sia storage via the sia-storage SDK, packed uploads for small files f83414a, eeafc59
1 HLS manifest rewriting to Sia object IDs, byte ranges preserved 72fe226
1 Postgres schema (assets, renditions, artifacts, sessions, jobs) and BullMQ f83414a, 4397f35
1 Thumbnails and the Docker Compose dev stack f83414a

Problems and how I’m handling them

  • I kept a single SDK (sia-storage) instead of the two-SDK plus seal/open plan from the proposal. It handled both upload and download cleanly, so I dropped the extra bridge. Flagging it as a deviation; all Sia calls sit behind one module if I need the second SDK later.

  • The Zen testnet’s small host pool means too much upload parallelism hits “no more hosts available.” I capped concurrency and packed the small files, which should relax on mainnet.

  • Cold Sia reads take a second or two to first byte. The byte-range gateway and a local cache cover repeats for now; client-side retrieval and cache warming come in later milestones.

What’s next

Milestone 2, delivery gateway and REST API: write the OpenAPI 3.1 spec and build the endpoints against it, add the worker that reconciles the database against the indexer, and finish caching (cache-status headers and warming after a transcode).

And sorry for the late progress report @mecsbecs, thank you for your efforts.

1 Like