Project Name: TrueNAS - Cloud Sync - Sia Integration
Project Lead: BoySanic
Developer/QA: BoySanic
Purpose
TrueNAS Core and TrueNAS Scale are open-source NAS operating systems primarily developed by iXSystems with additional contributors in the community. Cloud sync capabilities are available, which allow TrueNAS servers to sync their datasets to cloud remote endpoints using rclone.
Many users, including myself, use TrueNAS as their primary NAS and upload to cloud providers such as Amazon S3, Backblaze B2, and others. TrueNAS Core does have a rclone version with a Sia backend, but that backend relies on siad and will quickly become obsolete with the prevalence of the newer modular design of the Sia daemons. This project does entail updating both rclone and TrueNAS. I have confirmed that TrueNAS appears to use the upstream version of rclone and does not maintain its own fork of it, so this part should have fewer hurdles.
TrueNAS itself does not currently allow upload to Sia and doesn’t have the configuration needed exposed to the user.
My intention as of now is for this project to ensure Sia integration in the main branches of TrueNAS and rclone because a fork for Sia users to utilize that needs to be separately maintained would be too inaccessible, in my opinion. Whenever a change is made on either Sia or TrueNAS’s end, a further grant proposal may be put forward periodically to maintain the integration with varying complexity depending on what changes are necessary.
The reason this is beneficial to the Sia community is that this would not only give existing community members who happen to use TrueNAS the ability to back up their data to the Sia network, but it will expose an entire subsection of server administrators/homelab users to the copious amounts of storage available on Sia.
Development Information/ Open-Source Commitment
Since the project surrounds two existing open-source projects, the code will be open-source and available to the community.
The development forks will be accessible as follows:
GitHub - redsolver/rclone: "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files (already available rclone fork by redsolver)
Any code written by project lead/developer for this project will be available under BoySanic (BoySanic) · GitHub initially, and under the main projects after merge.
The project lead will make an effort to make sure users/committee/other developers know where to find code, either through direct communication or through project updates.
Project Timeline
The project shall have the following milestones, and timelines subject to change if things finish sooner or if integration into either external project stalls:
- Proof of concept TrueNAS instance (Jun '23 - Jul '23)
- Integrates Redsolver’s rclone fork
- Discovers the changes needed in TrueNAS to facilitate Sia as a backend
- QA changes to rclone made by redsolver, first PR to rclone (Jul '23 - Aug '23)
- Rigorously test redsolver’s rclone changes with renterd
- Begin PR process for both rclone and TrueNAS
- QA Sia changes to TrueNAS during the Core 14/Scale 23 release cycle (Sep '23 - Nov '23)
- Community involvement from those who use TrueNAS if they install an ALPHA/BETA version pre-release
- Will be used to catch issues both from Sia users and from TrueNAS users at large.
- Final changes merged into TrueNAS (Nov '23 - Jan '24)
- Will work with iXSystems to ensure rclone changes and TrueNAS changes do not cause regressions in their existing system.
- Developer available to maintain/educate and document the changes for the Sia community.
Risks
- Depends on Sia being feasible as a cloud bucket for TrueNAS to rclone to. Do not suspect this will be a major problem.
- Depends on rclone changes being merged. Sia backend already exists in rclone, redsolver updated it from siad to renterd in a fork, so those changes need merged. Do not suspect this is a major problem
- Depends on TrueNAS merging resolver and my changes to rclone and TrueNAS, respectively. This is the largest unknown as I have not worked with iXSystems in a development capacity but I have known them to be open to collaboration.
Budget
The project is requesting $42,000 USD for 7 months of development broken up as follows:
- 40,000 for developer salary, including research, development, QA, and integration
- 1,000 for sia integration testing (used to pay for siacoins)
- 1,000 for hardware resources (VPS hardware, bandwidth costs, etc)
Reporting
The progress of the development shall be reported on a monthly basis in the community Discord and on the Sia forum thread for the proposal.
Conclusion
The project lead is kindly asking the committee to review the proposal and encourages both the committee and community members to provide any feedback or questions they may have. Thank you very much for your consideration!