I didn’t have the idea, it was on the Sia - Grants page. It just happens that I know how to create Fuse filesystems and after giving a few reviews to the renterd API I realized it is possible.
I understand that the S3 protocol is slow and gets worse when an object ages.
As for rclone, it is designed to be used with siad not with renterd. There are quite API differences. I am almost positive, it won’t run. As I have said, I am not a GO coder; be my guest if you know GO.
My siafs will have caching at different levels. The FUSE API makes a lot of repetitive calls even when you do an ls /some/directory. Caching these calls not only will speed it up (as data won’t change if you pull the same meta-data twice in less than a second), but it will also prevent unnecessary SC burning (if you download many times a small/medium file).
Also, this project will give light to another one that I was advised not to disclose until this is done.
The renting side, which would use FUSE, is renterd, not hostd. As for rclone, Redsolver made a fork for renterd, but it was abandoned after S3 support was added.
If you think you can make something more performant, that would have merit. However, I also think it’s better to try to improve what exists rather than start over unless it is really required.
I remember using s3fs-fuse for a non-crypto-related project (VoIP) with AWS. The software hung until the file was retrieved (WAV files, 10-30 MB each). I can’t confirm or deny that siafs may be faster, the only thing I can confirm is that siafs will have caching in different levels, preventing unnecessary calls to renterd.
After this is done, I have other ideas that will speed up siafs. Including another like this one that will help other developers to adopt SIA in their projects.
Would you consider adding Finder/Windows Explorer Integrations to your original grant request or as another milestone?
As part of your caching strategy, I think this would be a welcomed feature to be able to sync/cache selected individual files/folders with a secondary click and keep some other files ‘online’ on the Sia Network, a little bit like Dropbox does it.
Thank you for the latest edits to your proposal. The committee has voted to approve this grant!
We’ll reach out to your provided email address to begin onboarding. It can take a couple of weeks to get new grantees set up, so please adjust your timelines accordingly. We’ll contact you after onboarding is complete to get the updated timeline and milestone target dates.
Thank you for providing your progress report in the template format, it helps us a lot! However, please shift your progress report deliverables to the 2nd of each month, covering the work done in the prior month.
If you’d like to provide additional info you can do that after those reporting requirements are met.
Regards,
Kino on behalf of the Sia Foundation and Grants Committee