Standard Grant: Proxmox Backup Server Cloud Extension

Progress Report Form Sept 2024

This last month we have been working on getting a .deb file so that we can install on a server. We have managed to do that and have started testing the work product produced by the previous developer. Now that we are testing my current developer is suggesting there

Links to repos worked on this month:
No repos have been worked on this month due to the above issue.
What will you be working on this month? Fixing bugs discovered now that we have a working backup server with cloud backup module.

Please summarize your development goals into a few sentences or bullet points:
Until we got the issue of the .deb file resolved it was not possible to put a timeline on completion. Now that we have solved the above issue I have asked our new developer what the timeline would be to complete the project and he is suggesting that it would take an additional two months of development time to complete the project. Although A-Z Computer Solutions is committed to completing the project at a much slower pace without the Sia Foundation we are asking for additional support from the Sia Foundation to complete the project. In order to complete the project in a timely manner we would need 384 hours ($13,440) which includes a 20% contingency (which is standard when costing projects). It is noted that no extra money will be forthcoming to the project manager and any money not used will be returned to the Sia Foundation.

Snapshot of project running on running on Proxmox Backup Server by our developer:

Progress Report Form Sept 2024

What progress was made on your grant this month?

  • Cloud Backup API Connection Fixes: Successfully identified and resolved issues in cloud backup list menu API connections, enabling proper communication between the frontend UI and backend services. Fixed broken functions and replaced the old tape APIs with updated Rust dependencies for improved performance and stability.
  • Middleware Development: Began development of a middleware layer to facilitate smooth interaction between the cloud backup APIs and the Sia decentralized storage APIs. This was necessitated after discovering direct connections between these APIs were failing, and the middleware will enable a reliable and efficient integration of Sia’s decentralized storage with the Proxmox cloud backup plugin.
  • Sia API Integration Testing: Successfully tested Sia API integration using Postman. This confirmed that basic API interactions, such as data upload and retrieval, are functional. However, challenges remain in creating a full connection between the cloud backup plugin and Sia’s decentralized storage APIs, which is being addressed through middleware development.
  • Codebase Maintenance and DevOps Improvements: Built and tested the existing codebase throughout the month. Also completed ExtJS build DevOps tasks, ensuring that the development environment is fully operational with optimized CI/CD pipelines.

Phase 8: Beta testing and bug cleanup:

  • API Connectivity: Initial testing revealed that the cloud backup APIs could not connect to Sia APIs due to missing middleware functionality. This led to a pivot in development focus, where middleware is now being built to bridge the gap between the cloud backup plugin and the Sia decentralized storage network.
  • Rust Dependencies: Encountered issues with outdated Rust dependencies that caused failures in the original cloud backup functions and tape APIs. These were replaced with updated libraries, improving API performance and reliability for storage functions.

What was done in the code:

  • Renter API and UI connection: Worked on improving the connection between the renter API and the UI for better performance.
  • Updated files: pbs/src/api2/cloud/backup.rs, pbs/src/api2/cloud/mod.rs, pbs/src/api2/cloud/restore.rs
  • Encryption and Decryption: Implemented hash encryption and decryption algorithms to secure data.
  • Relevant code: www/cloud/window/Hashcrypt.js
  • Cloud Backup API Development: Added backup and restore functionality in the cloud backup UI and connected APIs.
  • Code: www/cloud/window/CloudBackupJob.js
  • Access Control & Backup Group Functions: Enhanced the access control editing interface and introduced functions for changing backup group owners.
  • Files worked on: www/cloud/window/ACLEdit.js
  • UI and Styling Enhancements: Improved the UI elements and styling for the Proxmox Backup Server plugin.
  • Stylesheet: www/css/ext6-pbscloud.css
  • Backup Scheduling Features: Added scheduling options for cloud backups within the user interface.
  • Code updated: www/cloud/backupworks/backupschedule.js
  • API Work for PBS Cloud Backup: Tested the API connection for cloud backups, ensuring that encryption and decryption functionalities are working.
  • Test code: src/cloud/test/alloc_writable_media.rs
  • Frontend Development with ExtJS: Continued working on the user interface using JavaScript and ExtJS for improved interaction and visualization.

Links to repos worked on this month:

Merged branch ‘develop’ of https://github.com/AZComputerSolutions/PBSCloudBackupModule

What will you be working on this month?

Finalizing Middleware for Sia Integration: Complete the middleware development and test its ability to support full cloud backup functionality with Sia. This will include successful data backup and retrieval via the Sia API, followed by a visual demonstration (screenshots) of the working solution.

Please summarize your development goals into a few sentences or bullet points:

Completion of Builds and GitHub Updates: Finalize the latest build after the middleware and Sia integration testing is complete, and push the updates to the project’s GitHub repository.

Further Testing & Optimization: Conduct more in-depth tests on the entire plugin to ensure robust performance under different conditions. Optimize both the API and UI to ensure seamless use of Sia as a decentralized storage solution within the Proxmox Backup Server environment.

I would like to get an answer with regards to the additional funds requested in last month’s report. Without those funds we are going to have to slow down the progress of development. I would also suggest that if the project goes into slowdown mode there is a definite possibility of losing our current developer as he could possibility move on to another project.

https://www.pbscloudbackup.com/docs/sia/sia-foundation-monthy-report-sept-2024/

Hey Rodney, this is on our radar for today. Should we assume full-time work for the 380 hours? About 9-10 weeks?

That is Correct Steve.

Hey Rodney, the committee requested that you put together a proposal for the additional funds request. Use the Standard template and throw it in there. I’ll make sure it’s at the top of the list for reviews.

Thanx Steve, greatly appreciated.

New Proposal Posted here: Standard Grant Proposal: Proxmox Backup Cloud Extension (Continuation) as requested.

Progress Report Form Oct 2024

What progress was made on your grant this month?

 Middleware Development for Sia Integration:

 Cloud Backup API Enhancements:

 Integration Testing and Code Maintenance:

 User Interface (UI) Improvements:

Phase 8: Beta testing and bug cleanup:

Documentation and Repository Management:

We updated and merged our work on GitHub, including all major middleware and API connection changes, code maintenance tasks, and ExtJS UI updates.

Cloud Backup Middleware Development

Objective: Build middleware to enable interaction between Proxmox Backup Server’s cloud backup APIs and Sia’s decentralized storage API, addressing connectivity issues.

Changes:

Developed the middleware layer to route API calls effectively and provide consistent data handling across platforms.

Added error-handling routines to manage connectivity errors and log middleware actions for debugging.

What was done in the code:

src/middleware/siaIntegrationMiddleware.rs: Main middleware file for API requests.

src/middleware/siaIntegrationTest.js: Middleware test script for Sia API functionality checks.

API Connectivity Enhancements

Objective: Improve cloud backup API’s performance and security by updating dependencies and fixing broken functions.

Changes:Replaced outdated Rust dependencies with optimized libraries for faster and more stable performance.

Fixed broken functions within cloud backup API to ensure proper backend/frontend communication.

pbs/src/api2/cloud/backup.rs: Main cloud backup API file.

pbs/src/api2/cloud/mod.rs: Module configuration for cloud backup.

pbs/src/api2/cloud/restore.rs: Restoration logic for cloud backups.

Encryption & Decryption for Data Security

Objective: Secure data interactions by implementing hash encryption and decryption routines.

Changes:Added encryption methods to secure sensitive data during transmission between cloud backup UI and Sia storage.

www/cloud/window/Hashcrypt.js: Contains hash encryption and decryption methods.

Cloud Backup UI Development

Objective: Improve the cloud backup user experience through API integrations and UI styling enhancements.

Changes:Integrated backup and restore functionality directly into the cloud backup UI.

Enhanced UI and styling for improved user experience and accessibility.

www/cloud/window/CloudBackupJob.js: UI logic for backup jobs.

www/css/ext6-pbscloud.css: Stylesheet for cloud backup UI.

www/cloud/backupworks/backupschedule.js: UI for backup scheduling options.

Access Control and Backup Group Management

Objective: Enhance user access control settings and allow management of backup group ownership.

Changes:Introduced features for editing access control within the UI.

Added functionality to change backup group owners directly from the interface.

www/cloud/window/ACLEdit.js: Access control settings.

www/cloud/window/GroupManager.js: Management of backup groups.

ExtJS Frontend Development and API Testing

Objective: Ensure compatibility and test the UI for Sia integration using ExtJS and API testing tools.

Changes:Continued refining the ExtJS frontend for seamless user interactions with the Sia API.

Conducted API connection tests to validate functionality for cloud backups.

src/cloud/test/alloc_writable_media.rs: Test file for API connections.

www/cloud/window/UIComponents.js: Various UI components.

Links to repos worked on this month:

Merged branch ‘develop’ of https://github.com/AZComputerSolutions/PBSCloudBackupModule

What will you be working on this month?

Beta Testing and Bug Fixes:

 Test Server Setup and Deployment:

 Middleware Enhancements:

 UI Refinements and User Experience Improvements:

 Documentation and GitHub Management

 Please summarize your development goals into a few sentences or bullet points:

Fix bugs and code dependencies changes in rust that make our code no longer compile . Get a test server setup and working. Document setting up a test server. So that anyone can do it would the code in our github site. I would also like to note that because of the pending approval by the Sia foundation (thank-you for your continued support) we have been running at full development speed this last month.

We were ask by the Sia foundation to shorten our monthly reports but as always, the full version of this document is available at https://www.pbscloudbackup.com/docs/sia/sia-foundation-monthly-report-oct-2024/