Grant Request: Proxmox Backup Cloud Extension

Grants Rejected
1

Project Name: Proxmox Backup Cloud Extension

Project Lead: Rodney Wild

Company: A-Z Computer Solutions

Purpose

We purpose to increase the amount of stored data on the Sia network. At the moment there are a low number of big renters uploading data to the sia network. We purpose to develop a cloud backup module for proxmox backup that would take proxmox backup data and hold a replication of it at an external service provider. This can either be done directly via a local sia host or via a satellite when that service is complete. Proxmox backup server is a great candidate for using sia because of its large data sets. Even a basic proxmox backup can be TBs in size and if freely adopted could put many petabytes of data onto the sia network. Backups out in the sia network add an extra level of redundancy to local backups should a disaster recovery scenario happen and allow businesses that use the PBC extension to have a greater level of recoverability from disaster situations. For example, if the business or data center where their primary backup data is housed burns down to the ground or gets flooded. When it comes to backups, we think its good idea to have the original running host, an on site backup and fairly recent off site backup just in case something happens for business continuity purposes.

Open-Source Commitment

The project source code shall be maintained in a public repository located on github.

Project Timeline

The project shall have the following milestones:

Development of the main cloud backup module (July – Dec 23)

Development of the Sia Transfer Plugin (Jan 24 - Mar 24)

Testing of the product and website development to support the project (Mar - May 24)

Risks

The project will be using renterd software as the base for the customization. The progress of Phase 1 will therefore depend on i. but since renterd is close to completion the risk is fairly low to non existent.
The other rise would be the loss of a developer once the project is started and not being able to find another developer. As I will be hiring with an agency I think that risk is low. Although I am not a developer, I have been a project manager for a number of projects in my 35 year career in IT, some development related and some in other areas, so this would not be my first project that I will be managing.

Budget

The project is requesting 55,000 USD, paid in Siacoins, for the following 12 months. The spendings can be itemized as follows:

50,000 USD shall constitute the salary of the developer, paid monthly or quarterly, at the discretion of the Foundation, at the beginning of each period. This person, will be a hired position.

1,000 USD shall be used to pay for the project infrastructure (test servers and environment, subscriptions, etc.) for the duration of the project, paid upon completion of Phase 1.

2,000 USD shall be used for testing of the product (we plan to have a number of people do testing of the modules and will cover a small subsidy to cover cost of people testing the product while it is in beta similar to the way the foundation has incentivized participation in the testnet).

2,000 USD paid to the project manager of the project to cover administrative costs

Reporting

The progress of the development shall be reported on a monthly basis in the community Discord.

Conclusion

The project lead is kindly asking the Foundation to review this proposal. He is also encouraging the community to ask as many questions as needed to improve the clarity of the project scope and the expected outcome. Our goal is to increase the amount of data on the sia network and increase the level of redundancy of backups in the proxmox virtual environment and provide for disaster recovery.

Development Information
Will all of your project’s code be open-source?
[Projects can use closed-source components, but can’t develop closed-source code. If any of your project’s code is closed-source, please describe what code and why.]

There is a possibility that there will be extra functionality developed via additional plugins for other cloud platforms. but these will be closed source and a premium feature. Sia will be the only open-source plugin developed for the cloud back project.

Leave a link where the code will be accessible for review.

Do you agree to submit monthly progress reports?
[Progress reports must be submitted monthly here in the forum.]
I agree to share monthly updates in the forum and on discord.

1 Like
2

Question I asked on discord:

afaik proxmox does not support any kind of plugins, so you will fork it and add a option to mount sia storage? why just not use rclone to mount sia to a folder on proxmox host?

1 Like
3

I understand the potential value of your project, but I have concerns about the feasibility of executing and maintaining the project without a developer already in place. Additionally, I would appreciate more technical details on how the cloud backup module will work, specifically how it will interface with Proxmox and Sia.

1 Like
4

You are right proxmox backup server but it does seem to have the ability to support modules (ie: the tape module that allows backups to tape drives). I purpose to add another module similar to the tape module and add plugin capability to support the different cloud platforms. The first one being Sia.

I have spent some time with rclone and it is not as easy to setup and use as the instructions and hookscripts suggests. This impedes the vast majority of people that might backup to sia from using the project. Also according to my read of the docs it does a full backup every time it backs up to any platform with rclone which is less than optimal.

1 Like
5

Hi Nate, there would be no difference between someone doing the dev work themselves and my hiring a developer to do the project. As mentioned in the proposal this is not the first project I have managed and this is not the first development project that I have managed.

I purpose to add another module similar to the tape module in proxmox backup server and add plugin capability to support the different cloud platforms. The first one is Sia. I have plans to add some premium features that will be closed source (not part of the project and not funded by it) and that would include some additional cloud plugins (s3 for example).

image
image1543×823 59.1 KB

1 Like
6

Thanks for your proposal! Just a note that we received this just after our cutoff for committee review for April 18th. It will be considered during the next committee meeting on May 2nd.

1 Like
7

Yep no problem, not in a hurry.

1 Like
8

Here is an article highlighting missing features in proxmox and cloud was one of those features.

1 Like
9

Hello Rodney,

Thanks for the proposal! The committee isn’t quite sold on the technical feasibility yet and is requesting additional information.

We’re not familiar enough with Proxmox to make a determination as to the logistics - it appears as if you’ll need to get the Sia integration accepted into their core code rather than being able to publish it independently. The lack of a developer is also an orange flag for us. Not having an experienced dev lined up can cause significant project delays as working with new blockchain code can be a daunting task, though we understand the possible Catch-22 of finding a dev before you have the money to find a dev.

The committee would like to see this proposal updated in the following ways:

  • Line up a developer to work on the project who can attest to the technical feasibility, or;
  • Submit a feature request to the Proxmox team explaining the project and getting their thoughts with more technical info on the integration.

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

1 Like
10

Thanx for the update Kino.

1 Like
11

I have reached out to the proxmox group and am awaiting a reply.

1 Like
12

Thanks @rdwild! We’re looking forward to hearing back.

1 Like
13

it looks like proxmox group is not going to respond to this request. Not sure why. But loaded a test instance of the backup server and I took a look at it and the web interface is javascript and seems to be easily extendable. In fact I added a cloud backup option as a test to make sure that it was as extendable as I thought. Here is a pic of my investigation.

image
image1543×981 73 KB

You will see a cloud backup option in the dashboard even though there is not a cloud backup option. Right now I am pointing it to the existing tape module but it could just as easily be pointed to a cloud backup module.

1 Like
14

I got your most recent post in discord. it is kinda looking like there is a bias against people that are not already developers. I withdraw my application. I might decide to close-source it at a later date. Thanx.

1 Like
15

Hi,
Even though I can understand your disappointment, what you say is not completely true.
When I applied with the Sia Satellite proposal, I was not a professional developer (which I am still not), but I already had a (really) Minimal Working Product available. I am not fully aware of what exactly convinced the Committee to approve my proposal, but I can make a good guess that it was one of the reasons.
It is of course up to you whether to go on with your proposal or not; my personal opinion is that you may be giving up too quickly.

1 Like
16

This would have been an easier pill to swallow if the foundation would have come up with a bit of $$$ to do the additional testing they are requesting to be done. I have already spent a fair amount of time ($$$) to allay their thoughts with regard to feasibility.

1 Like
17

You made a grant proposal for developing the full integration/extension, so for approval there needs to be some confidence that the project is even possible from a technical point of view. An example of existing third-party backup extensions to Proxmox could already be enough. If the research for this takes up too much time, you could also change the grant to be a smaller research-only grant for now (similar to for example Grant Proposal: Sia Atomic Swap with Adaptor Signatures Feasibility Study) and later apply for another one if the research turns out successful.

1 Like
18

mmmm, what would the foundation need from a feasibility study perspective? The web interface is javascript that is easily changeable which allowed me to set up an additional option on the left-hand desktop pane.

1 Like
19

Thanks for the update rdwild! The committee wasn’t sure that the change showed in the forum provides actual proof of feasibility - the ability to extend the local interface to provide cloud backup options doesn’t mean that Proxmox will accept your code contributions.

Without hearing from Proxmox, are you able to provide an example of an existing Proxmox extension that does something similar that similarly isn’t natively supported? This is what the committee would like to see to feel confident in the projects viability.

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

1 Like
20

As far as I am aware, no one else other than proxmox is working on the proxmox backup server. they do have a process to have code accepted into the main branch but just like with any other development project, there is always a chance they won’t accept the developed code. That is the risk of every open-source project as you probably know to well. The option if the code is not accepted would be to fork the original project.

The other option would be to wait and see if Proxmox develops a cloud module themselves.

1 Like