Foundation Core Development Proposal: Host Module (WIP)
- Contributing Authors(s):
- Start Date: 2020-01-13
- Category: Foundation / Technical / Host
- Tracking Issue: -
This a open formal proposal that aims to discuss the future of the host module on the Sia network. With the inception of the foundation, new resources will soon become available that will
enable additional development, fixes and improvements of the host module that were previously not economically viable for the parent company Skynet Labs to execute. Although development
of the host module will only be supported on
siad for the time being to more efficiently allocated the available resources, a proactive plan should be compiled that enables growth of the host module beyond general improvements or singular venture-based use cases. Effectively bringing the host module closer to the foundation. We propose an implementation that can be tweaked and optimized for different hardware and use cases the host wants to support on the Sia network.
The goal of Sia is to create a decentralized, permission-less and high performance global data storage network that can serve as the foundation for many different web3 (e.g. Skynet) and other applications.
As such, the host module is a crucial building block for the Sia network and directly affects any data related interactions. Being a for-profit company, the primary maintainer of the
host module, Skynet Labs (previously Nebulous Labs) has expressed the inability to maintain and improve the host module beyond what moves the needle financially for the company. With the
inception of the foundation it is therefore logical that the foundation dedicates a significant amount of resources to maintain and improve the host module in a way that is beneficial for
the entire Sia Ecosystem.
Current and future participants of the Sia network encompassing unknown network participants and known participants including Skynet Labs & Ecosystem, Filebase, StoreWise and Siacentral.
Feedback will be gathered by sharing this proposal in various Discord & Forum channels.
Some improvements include:
- Faster sync times (utreexo)
- Less lock-ups and timouts
- Lower latency for requests
- Faster transfer speeds
- Better issue reporting
- Lower memory, IO and CPU footprint
- Better scalability for datacenter/enterprise systems
Currently the Sia Foundation and Skynet Labs share the IP for the host and renter modules (source: discord). For the foreseeable future the
siad host module will remain the
primary host implementation. This is a logical decision considering the already existing expertise within Skynet Labs and the time it takes to onboard and train the new foundation engineers. As new foundation resources become available, we request the foundation dedicate some available resources to improve upon the current host implementation.
Standardized / Modular host implementation
As more projects start building on Sia, new use cases and requests will arise for specific host features. These host features might (1) be too specialized and cause
conflict with the Skynet Labs vision and goal for the
siad host; (2) too many features can make the host implementation unusable for low powered hardware devices.
Therefore we propose the foundation together with Skynet Labs begin work on designing a modular host implementation that is scalable, cross compatible and flexible enough to support the addition or removal of specific modules without causing host-renter interaction conflict. We envision hosts having the ability to, depending on the hardware available, activate
profiles optimized for specific use cases. For low powered devices (i.e. a RPI) a lightweight default implementation can be made available.
Many unknowns are still present for a modular host implementation: (1) how will renters known which host has which module activated?, (2) are there better strategies available to build
a host package that can grow outside of the scope of individual use cases or constraints?
Rationale and Alternatives
The rationale for these changes:
- Support different host implementations with different optimizations and use cases.
- Maintain a lean and high flexibility approach to the host package components.
- Guarantee cross-compatibility between host implementations and prevent renter - host interaction issues.
- Prevent lock-in to one single host implementation.
- Support datacenter and enterprise hosts.
We believe that host improvements will have a positive impact in driving growth of the Sia network. We also believe that a modular host will create a vivid ecosystem of highly optimized
hosts ready to provide the best service possible with their available infrastructure.
- More stable and performant host-renter interactions.
- Growth of the number of Sia hosts.
- Growth of data on the network.