Grant Proposal: SiaShare

[mjmay08]

Introduction

Project Name: SiaShare

Name of the organization or individual submitting the proposal: mjmay08

Purpose:
This proposal is based on the idea from @parox for an end-to-end encrypted file sharing site that is very simple to set up and use: E2EE file sharing app

The plan is to create something very similar to https://wormhole.app/ but with data stored on Sia using renterd. A user can visit the site, drop some files on the page, and then get a link to share those files. Files will expire after a certain time period or a specific number of downloads.

The goal is to create a lightweight web service that anyone can run alongside renterd with minimal setup and configuration. The client application will use Uppy (https://uppy.io/) for the file upload experience, wormhole-crypto (GitHub - SocketDev/wormhole-crypto: Streaming encryption for Wormhole.app, based on Encrypted Content-Encoding for HTTP (RFC 8188)) for the file and metadata encryption, and the Uppy Tus integration to provide resumable file upload to the backend. The backend will be a simple webservice with a sqlite database and files will be temporarily cached on disk while they are being uploaded to the Sia network using renterd.

Grant Specifics

Project Timeline
In total I expect this proposal to only take a few months at most. The scope is limited but as I am only working on this part time I don’t want to promise anything shorter than that.
I have been working on a proof of concept already so that I could be certain this proposal would be a success. I hope to have a basic working version for the community to start testing a few weeks after the grant is approved.
After that I will work on customization(expiration, file size/count limitations, etc), UI improvements, documentation, and testing the scale of the service.

In addition there are many other enhancements that could be made as time allows such as:

• Packing files that are uploaded together into one to minimize the impact of Sia’s minimum sector size
• File preview in the browser without downloading
• Enabling Uppy integrations for services like Dropbox, Google Drive, and OneDrive so that user’s can import and share a file directly from one of those services.

Budget
I am requesting $5000 for the duration of the proposal to cover the part-time salary of myself as well as infrastructure costs to host an instance of this service for the next few months for the community to use while it is in development.

Potential risks that will affect the outcome of the project:
As mentioned above I have already been working on this project for a little while so that I could be confident in my ability to deliver it. The only risk I can see right now is that since I am building on top of renterd which is still beta, any issues with that integration could push back the timeline (however so far I have had no issues with the integration).

Development Information

Open-Source Commitment
The project source code shall be maintained in a public repository located at: GitHub - mjmay08/SiaShare
All code will be MIT licensed.

Progress reports
Yes, I agree to submit monthly progress reports. In addition I plan to provide more freqent updates on the Sia discord as this proposal is only expected to take a few months at most.

Contact info

Email: Will provide in DM on approval
Any other preferred contact methods: Sia Discord @mjmay08

Conclusion

I’m excited to contribute to the Sia ecosystem and hope that the grant committee will approve my proposal. I welcome any questions on details I may have missed above.

[parox]

Hi @mjmay08, very excited to see this!

I it would be really cool if the tool could replicate the wormhole feature where the receiver does not need to wait for the upload to finish to start downloading.

I forgot to add it to the Grant Idea but this is also very similar to @redsolver’s SkySend which also had an extremely simple user-friendly UX: GitHub - redsolver/skysend: Decentralized end-to-end encrypted file-sharing - that really made it one of the most useful/used Skynet apps. I think you are familiar with it, but definitely worth checking it out for further inspiration. Maybe @redsolver will port that to work with S5 and then we will have 2 great easy to use file sharing options on the Sia stack .

Other notes:

• Also its pretty cool how wormhole supports direct p2p transfers in certain situations.
• wormhole FAQ says they might open source the full app, if the code would be a useful reference for the streaming bits or integration with the wormhole-crypto library then it might be worth reaching out and asking about it.

[pcfreak30]

This feels somewhat similar to ideas I’ve had around BitTorrent and general filesharing of data on sia by knowing the sector and host data + encryption key.

I would recommend that you try to stick to the data formats S5 is creating which are based on blake3 for interoperability. With that, this could probably be further integrated into the ecosystem in the future.

Kudos!

[mjmay08]

@parox

I it would be really cool if the tool could replicate the wormhole feature where the receiver does not need to wait for the upload to finish to start downloading.

@parox

Also its pretty cool how wormhole supports direct p2p transfers in certain situations.

If I understand correctly I think both of these asks are solved by using WebTorrent? For sure that is how they do the p2p transfer, but that must also be how they accomplish the “instant file streaming”? I haven’t used WebTorrent before but that looks like something I could attempt as well.

@pcfreak30

I would recommend that you try to stick to the data formats S5 is creating which are based on blake3 for interoperability. With that, this could probably be further integrated into the ecosystem in the future.

Cryptography is not a strong suit of mine and the only reason I considered submitting this proposal is that this library (GitHub - SocketDev/wormhole-crypto: Streaming encryption for Wormhole.app, based on Encrypted Content-Encoding for HTTP (RFC 8188)) makes that part of the application pretty straightforward. I’d rather not risk doing the encryption incorrectly by implementing it myself using blake3. Maybe someone would be willing to make that change after my initial proposal is complete?

[pcfreak30]

The blake3 stuff already exists. However, blake3 is not an encryption library, it’s a hashing library (md5, sha1, sha256, etc). It is being used to create content-based file IDs (CID) like what IPFS does with S5 and Lume.

[steve]

Thanks for the proposal! I just wanted to reply quickly to let you know that we received this proposal a bit too close to our meeting tomorrow to take a proper look at it. We will review it in full at our next meeting on May 30th!