Standard Grant: Lume Web 2024

[pcfreak30]

2023 Report

Timeline & Reflection

This year has been chaotic for Lume as the entire Sia community was affected by the collapse of Skynet. For us, it meant a significant change, of course, where instead of using Skynet, we had to figure out a way to replace it. For those unfamiliar with it, Skynet technology allows the upload and download of data through interoperable public portals without users having to run their nodes.

Lume, a project initially intending to leverage Skynet technology, had to decide how to continue its core mission of an open, user-owned web. Since Sia keeps constantly evolving, without further maintenance of Skynet, it was clear that we needed to find another way, as it no longer made sense to keep building on top of abandoned technology. Instead, similarly to some other projects, the way forward was to learn from Skynet’s mistakes and do it better. These efforts are visible not only on Lume, inspired by Skynet’s Kernel, but also on projects like S5 by @redsolver, a flexible peer-to-peer file-sharing network that Lume decided to support instead of re-inventing the wheel.

We re-purposed and expanded some of the technology in an attempt to solve the DNS problem of the decentralized web and become able to access domains across multiple distributed networks. This development effort revealed that support of each network is not easy as each network has its issues and research involved, resulting in more delays than we expected. And finally, we also needed more time to find reliable designers and developers to work on the consumer side of the project, which is mission-critical for product exposure and adoption.

Despite setbacks, we made significant progress on the consumer side and in the core backend technology.

• We successfully added support for the Handshake, Ethereum, and IPFS networks into Lume’s routing technology. Each implementation presented learning curves, roadblocks, and setbacks that must be navigated and solved. Ultimately, we achieved the goal of creating a working Lume prototype.
• We added support for bao & blake3 protocols, which was quite a complex challenge that we eventually successfully solved at the cost of skipping it for small files as it wasn’t required.
• We managed to get rid of needing Skynet’s portal by forking Skynet’s Kernel instead of trying to use it as-is, as the data formats between Skynet and S5 are incompatible. This engineering decision saved a lot of effort due to the instability of skyd.
• We figured out how to query ENS domains (on ETH) in a fully trustless way.

For the next part, let’s explain two significant components that Lume consists of and what makes it sometimes confusing to understand. Whenever used alone, it usually represents either the brand or the technology for decentralized web access. But there is one more component we are working on - Lume Portal.

Lume Portal is our interpretation of what a Layer 2 (L2) should look like in the spirit of Skynet, providing essential functions for other systems connecting to it. That also represents the current state, the portal being only a backend server with no user interface but allowing us to deliver the demo apps to users already. The well-designed Renterd APIs were surprisingly easy to implement, which we see as a great accomplishment by the Sia Foundation. We can already feel the impact of Sia’s ongoing re-design.

The portal software aims to fill a gap left in Sia’s ecosystem after the disappearance of Skynet Portals, and it seeks to onboard as many users and their data as possible. This will benefit both the Sia network and Lume, which, by pursuing more users, helps the Sia network’s growth and independence.

We also released a user-focused product, a Firefox browser extension, to provide access to the Lume technology and, by extension, Sia. At first, Firefox would provide everything needed for a fully decentralized experience. However, it didn’t work out as planned, not because it wouldn’t work, but because the target audience couldn’t care less about Firefox. Still, we consider this a success, as the goal of the extension wasn’t to drive immediate and broad adoption but to pilot and demonstrate the technology. This allowed us to receive crucial feedback, helping us to identify the fundamental problems and find ways to solve them.

We learned a lot throughout the year, including the realization that we must rely on something more transparent and censorship-resistant than the existing web browsers. It’s a constant fight, and we can’t tell if developers of those one day won’t block the APIs we use for our apps and extensions. This assessment of the macro situation led us to introduce a new long-term plan to commit to developing a Lume Browser. In the short term, we will still ship Lume Technology through other accessible means already available on the web.

Our journey so far has led us to create some demo apps. As stated before, the Firefox extension turned into a flop, so instead, we focused on the creation of a simple UX for a web login into Lume and created two demos, a web3 browser app and an NFT gallery app, powered by Sia’s renterd, which stores and serves the browser code required to communicate with all networks, including Lume’s P2P net.The web3 browser app also supports serving Sia-powered websites via S5. Renterd, accessed through Lume’s Portal, will also enable long-term user storage for applications accessed through Lume Technologies in the future.

Our intention has shifted towards bringing as much data to Sia as possible through the Lume Portal, supporting most (if not all) of S5’s network abilities and technologies like Bluesky and IPFS. This approach will ensure the broadest adoption and interoperability.

We believe the Lume Portal (as Layer 2) will have its own means of syncing data via hypercore such that it imports the data and owns the contracts like the original Skynet. Since it’s using renterd, it means that ALL data is stored on the Sia network unless the user defines different or multiple options through alternatives we support as well, like S5. Using hypercore in the portal effectively means using the P2P tech we spent significant time on to help power the L2. At the same time, we’re also developing a community platform, “web3.news”, which will serve as the hub for the whole industry with easy ways for users to host community sites on the Sia network. Doing this could be through subdomains or their domain, based on the communities’ interest in deciding how this evolves. So, web3.news covers our intention for a community effort this year. While we planned to do more, we realized we needed to build a community before anything else, and we will be focusing on that first.

Looking at the larger picture, Lume, both as a brand and product, continues to evolve in alignment with its original vision and principles of a decentralized internet. We are satisfied with the progress made so far, and the results promised to match, if not exceed, the expectations set earlier in the original grant proposal for the year 2023. It’s also clear that many proposed details have changed, but that’s unavoidable, given the dynamics and challenges we faced on the ecosystem level.

Current State

The project currently consists of several functional components, all working prototypes. We have set up the core technology to offer a P2P DNS system for web3 domains, showcased through a Firefox extension and a web app, and continue focusing on enhancing the web app experience and the underlying technology. Everything we learn in the process also helps us shape the plans for a browser-based solution that we’re committed to developing in the future.

We intend to improve or entirely redo much of the code as we discover better technologies or if the project’s short/medium-term goals change. We know that the entire ecosystem is evolving on many fronts, and discoveries are made constantly, making continuous refactoring unavoidable, especially for projects like Lume, integrating several other technologies in active development. However, everything functions as intended, and plans are in place to iterate on the next steps.

Finally, we would like to finish this part of self-reflection by examining how we use Sia today and what lies ahead. As mentioned above, this year’s largest effort was on routing technologies, which might be less attractive for users looking just for storage but is crucial for further development and Sia-related features. Currently, the Sia network is used for downloading the Browser App’s code and creating an anonymous account used in the process. It can also serve sites hosted on the Sia network via S5. The browser app demo showcases our progress, and you can find both the link and source code in the Deliverables section below.

Demo Videos

The following is a quick reference to all demo videos, which can be found in the deliverables as well:

• Firefox Extension:

Firefox Extension Demo

• Web3 Browser App:

Web3 Browser App Demo

• NFT Gallery:

NFT Gallery Demo

Relay

The relay is a Node.js P2P application that functions as a P2P node. It features a plugin system and some configuration management. The latest version is available at https://git.lumeweb.com/LumeWeb/relay/src/branch/develop. Marketing for the relay ceased due to a strategic pivot.

Plugin Name	Description	Link
ETH	Provides consensus data to the ETH client	https://git.lumeweb.com/LumeWeb/relay-plugin-eth/src/branch/develop
S5	Acts as a partial S5 node for registry services	https://git.lumeweb.com/LumeWeb/relay-plugin-s5/src/branch/develop
Handshake	Acts as a proxy for serving HNS peer connections	https://git.lumeweb.com/LumeWeb/relay-plugin-handshake/src/branch/develop
IPFS	Acts as a proxy for serving IPFS peer connections	https://git.lumeweb.com/LumeWeb/relay-plugin-ipfs/src/branch/develop
IRC Discovery	Acts as a means of allowing peers to discover the relay by pubkey	https://git.lumeweb.com/LumeWeb/relay-plugin-discovery-irc/src/branch/develop
Lavanet	Enables the relay to sponsor badge services, which are effectively a sub-billing account so the end user can query paid RPC while still being trustless	https://git.lumeweb.com/LumeWeb/relay-plugin-lavanet/src/branch/develop

Every plugin has a matching client-side codebase.

Portal

Next, We would like to visit the state of the portal itself, which currently has a live web app and demo for consumers to experiment with the basics of the Lume tech.

https://git.lumeweb.com/LumeWeb/portal/src/branch/develop

The portal consists of a Golang daemon and API server. It features:

Feature	Description
TUS Support	We might have a broken feature, as we last tested it a while ago.
Basic File Uploading Support	Primarily used for kernel module code.
Accounts with Email and Pubkey Login	Allows users to log in using either their email address or public key.
Basic File Tracking	Used for keeping track of files.

Kernel & Browser

The Kernel (libkernel and libmodule) was forked and combined into a single library. The following are the components:

Component	Description	Link
libkernel with libmodule	This component represents the merging of libkernel and libmodule into a single library.	Link
Kernel	Refers to the core Kernel itself.	Link
A re-implementation of skt.us	This is a hosted part of the Kernel in web2, trading off trust for accessibility.	Link
Hosted kernel worker	Enables Kernel modules to run in their own hosted iframes, providing them with dedicated CPU threads and overcoming the original Skynet Kernel module limitations.	Link

You can see that Lume is on course with the mission to bring decentralized DNS and distributed file hosting to the online consumer. Over time, We’ve developed, implemented, and maintained a growing library of open-source protocols that will serve the Lume mission while also bringing attention to fundamental technologies provided by Sia and building a community of interested developers and consumers to the Lume product, which I will talk more about below.

Community

This year’s primary focus is on web3.news, which we plan to have functional as an MVP by the end of the year despite the project holding many assets for future community efforts. We will delay the implementation of additional features until we reconstruct the basic primitives we are currently using, such as S5. The community will guide the direction in which web3.news takes off.

Source code: https://git.lumeweb.com/LumeWeb/web3.news.git.

Design files

As this project is entirely free and open-source, all design files based on the community funding are available here: https://git.lumeweb.com/LumeWeb/design.git.

Deliverables

Demo	Description	Link(s)	Video
Firefox Browser Extension	An early prototype extension for Firefox. It was a successful Proof of Concept from a technical and R&D perspective but needed more adoption. The extension doesn’t include much of the R&D from this year, like the S5 registry. A video is available showcasing its experience.	Git Repository	Link
Web2 -> Web3 Browser App	The first fully decentralized Web3 Browser/Gateway using service workers, similar to archive.org. It supports IPFS, Sia websites (via S5), and HNS and ENS domains. It uses the Lume Network, Portal, Kernel, and S5 network to load content trustlessly. PR release gained exposure.	Web App, Git Repository	Link
NFT Gallery	Part of a series of toy demos. It uses ETH and IPFS to find ERC721 NFTs for a given ENS address. The demo represents an early stage that will evolve in the future.	Toybox, Git Repository	Link

These demos may have minor UX bugs and not support edge cases or every site/ETH account. Consider them polished prototypes with plenty of room to grow.

[pcfreak30]

Grant Proposal

Introduction

Project Name: Lume Web

Name of the organization or individual submitting the proposal: Hammer Technologies LLC

Describe your project:

Lume Web is our way of stitching together the vast web3 landscape. From decentralized networks to DeFi and crypto, we’re building a hub for everyone to access web3 content easily without relying on any central power. Currently, Lume Web consists of:

• A peer-to-peer node that we’ve developed with Node.js.
• Lume Portal, written in Go, taking cues from Skynet.
• Browser tech is based on our fork of Skynet Kernel.
• Forum and a community news platform for the web3 crowd, with some help from the tech behind Sia and S5.
• A web app demo supporting IPFS, Sia (via S5), HNS, and ENS.

Our aim with Lume is to be that one-stop gateway for all web3 content in a genuinely decentralized way.

This past year has been a bit of a maze, figuring out our place in the community development scene and how to adapt to other projects that are on a similar path.

Along the way, we’ve had to switch gears a few times, especially as new tech and libraries came into the picture, not to mention some exciting opportunities that we didn’t see coming.

We’re getting ready to take Lume into its next chapter and excited about where we’re heading.

Our goals for 2024:

• Superportal Software Development
  • Rewrite MVP software to support multiple web3 protocols.
  • Begin with S5 protocol implementation in Golang; collaborate with @redsolver.
  • Use subdomains for each protocol to serve as API servers.
  • Prioritize starting with S5 and IPFS, with plans to include more protocols.
  • Aim for portal software enabling paid hosting and competition with large closed-source firms, maintaining democratic/decentralized access.
  • Introduce a single subscription service for diverse data management.
• Billing and Subscription Integration
  • Add billing support, initially incorporating Stripe, avoiding crypto transactions for users.
• User Interface and Marketing Site Creation
  • Develop a web dashboard and a marketing site for enhanced user interaction and portal promotion.
• Kernel System Upgrade
  • Rewrite the kernel system with a superior underlying framework. Explore and possibly utilize https://github.com/GoogleChromeLabs/comlink for browser communication and kernel RPC system.
• Relay Server Refactoring
  • Rewrite the relay server to incorporate https://github.com/GoogleChromeLabs/comlink and evaluate Bun.js to create a more straightforward all-in-one binary.
  • Shift towards using web workers/worker threads for improved efficiency.
• Web3.news Community Platform Development
  • Continue development and periodic redesigns to maintain alignment with portal updates.
  • Adapt and improve based on community feedback.
  • Engage in targeted marketing to build a community base.
• File Sharing Functionality between Portals
  • Implement hypercore technology for a decentralized file metadata-sharing system.
  • Focus on on-demand contract creation for file imports and redundancy vs. pre-creating contracts like skyd did.
  • Keep P2P logs/blockchains public to ensure data persistence and resilience.
  • This approach intends to expose the sector data to the community for redundancy and ensure the portal can track who is accessing the data. S5’s network takes a different approach to this, which may make tracking user usage more complicated and does not currently share sector data.
• Browser Extension and Desktop Proxy Application Development
  • Design a two-part gateway system with a browser extension for Chrome and related browsers for proxy management and dashboard access.
  • Create an Electron application leveraging kernel technology in web workers.
  • Ensure the extension facilitates web3 domain to IP address resolution and includes DANE SSL support.
• web3browser.io Application Advancement
  • Continue to improve web3browser.io to promote the Lume & Sia brands and mission and to ease users into decentralized web technologies.
• Community Hosting Service Management
  • Set up a robust hosting service for the community, offering paid services and free usage subsidized by the Foundation.

For more information, you can visit https://docs.lumeweb.com/ or check out a snapshot of the documentation at the time of writing at https://git.lumeweb.com/LumeWeb/docs.lumeweb.com/commit/9018cb11bf357cb581e6dadc7ffafba8f8c93c4a.

Additionally, as the Lume project is slowly maturing, we are attempting to be more structured and organized in project management.

Who benefits from your project?

The project benefits:

• Sia community by:
  • Enabling an L2 portal software that gives access to multiple Web3 protocols.
  • Starting a hosting service for the community that does not require self-hosting.
  • Evangelizing the Sia network and spreading awareness that it is the data backbone of web3.
  • Evolving and growing a community platform for the wider web3 community powered by Sia, thus spreading awareness in the process.
• Data hosters:
  • Providing cheap, reliable, decentralized storage via multiple distribution protocols to the broader community.
  • Democratizing access to a FOSS file portal software that will enable pinning on multiple protocols, ensuring censorship resistance.
• DApps/Web Apps:
  • Enable webapps to be hosted on Sia through several distribution protocols, which should ensure censorship resistance.

How does the project serve the Foundation’s mission of user-owned data?

It creates an actual decentralized open web by unifying all ecosystems and providing a platform for users to own their data while exploring the digital seas. At the heart of the web is data. Data is to the web what oil is to the world. It is currently the most valuable thing among the world’s nations.

The web needs paid data storage to ensure things stay online, and users need private data storage that retains their privacy and self-sovereignty.

Sia solves that problem, which is a central element of the project.

Grant Specifics

Amount of money requested and justification with a reasonable breakdown of expenses:

Total Budget Request: $91,000

Upfront Budget Request: $49,750

The initial funding needed includes:

• Derrick Hammer’s Q1 Salary: $13,750
  • This is Derrick Hammer’s first quarter salary.
• Project Budget for the Year: $36,000
  • Requested in full to efficiently manage contractor payments and adapt to variable project demands. The breakdown of this amount is as follows:
    • Contractor Payments: Essential for project milestones, with unpredictable quarterly needs.
    • Marketing Expenditures: Allocated monthly, subject to change based on the marketing agency’s advice.
    • Infrastructure: Allocated monthly, subject to change based on testing and scaling needs.
    • Storage: Allocated based on demand.
    • Code Certs and MacBook: One-time development costs.

Detailed Expenses Breakdown

1. Salary Expenses: $55,000

• For Derrick Hammer
• Justification: This represents an inflation-adjusted increase from the previous year.

2. Annual Project Budget: $36,000

• Used MacBook Laptop: $1,000
  • Purpose: For Mac-native software development.
• Code Certifications (Apple and Microsoft): $1,000
  • Purpose: To ensure app security and safety.
• Design and Development Work: $20,000
  • Projects Covered: Includes development of portal dashboard, Chrome extension, desktop proxy, web2 browser app (web3browser.io), web3.news, and potential marketing websites.
• Marketing Efforts: $9,000
  • Breakdown:
    • $4,500 for SEO Services.
    • $4,500 for Social Media Management.
  • Goal: To enhance awareness of Sia, HNS, and Lume through grass-roots marketing.
• Server Infrastructure Costs: $4,000
  • Note: This is subject to change based on testing and scaling needs.
• Sia Network Subsidy for Storage: $1,000
  • Contingency: Additional funds may be necessary if demand exceeds expectations.

Marketing and Development budgets may be re-balanced throughout the year based on the projected needs of the project

Timeline with measurable objectives and goals

The items mentioned follow the OKR (Objectives and Key Results) methodology. Budgets represent Derrick Hammer’s salary only:

Q1 Milestone

Budget	Objective	Key Results
	Rebuild foundational systems	• Goal: Rewrite portal software using jape, go migrate, and tusd. • Validate: Test operational capacity of S5 protocol and dnslink functionality.
		• Goal: Refactor relay software to use Bun.js and comlink. • Validate: Ensure plugin functionality post-refactor and integrate mock plugin.
		• Goal: Redevelop kernel software to incorporate comlink. • Validate: Set up new infrastructure for isolated testing.

Q2 Milestone

Budget	Objective	Key Results
$13,750	Develop Dashboard & Website	• Goal: Create a user-friendly dashboard. • Validation: Launch functional dashboard and integrate with existing systems.
		• Goal: Design and develop the website with specific sections. • Validation: Create responsive frontend and complete development within a quarter.
	Gather User Feedback and Improve Continuously	• Goal: Collect and iterate on user feedback for dashboard and website. • Validation: Implement improvements and establish a regular feedback loop.
	Implement Billing Functionality with Stripe Integration	• Goal: Integrate Stripe plans and pricing management. • Validation: Configure Stripe plans and ensure user subscription process.
		• Goal: Establish user onboarding with Stripe Checkout. • Validation: Test payment processing and subscription creation.
	Enable File Metadata Sharing Between Portals with Hypercore	• Goal: Integrate Hypercore for metadata sharing. • Validation: Implement and test Hypercore functionality.

Q3 Milestone

Budget	Objective	Key Results
$13,750	Update web3.news to Align with New Infrastructure Changes	• Goal: Integrate new infrastructure into web3.news. • Validation: Implement updates and confirm compatibility with comprehensive testing.
	Upgrade web3browser.io and web3toybox.com Demos	• Goal: Enhance demo features and functionality. • Validation: Prioritize enhancements, implement them, and conduct thorough testing.
	Develop Browser Extension and Cross-Platform Web3 Proxy App	• Goal: Develop browser extension for web3 proxy app management. • Validation: Ensure browser compatibility and test functionality.
		• Goal: Create Electron-based web3 proxy app with DANE support. • Validation: Develop app, ensure cross-platform functionality, and test secure connections.

Q4 Milestone

Budget	Objective	Key Results
$13,750	Integrate IPFS Support into the Portal	• Goal: Integrate IPFS support into the portal. • Validation: Implement changes for IPFS, verify user interaction, and test functionality.
		• Goal: Ensure API compatibility with major IPFS providers. • Validation: Enhance APIs for interoperability and conduct comprehensive testing.
		• Goal: Add IPFS support into the user dashboard. • Validation: Integrate functionalities and gather user feedback.
		• Goal: Maintain portal’s performance with IPFS. • Validation: Monitor performance and conduct stress testing.
	Get Project Hosting Ready for Production	• Goal: Prepare hosting infrastructure for production. • Validation: Set up servers, ensure infrastructure health, and confirm monitoring.
		• Goal: Establish a streamlined deployment process. • Validation: Create and document deployment process, and verify with test deployments.
		• Goal: Secure production environment and maintain stability. • Validation: Implement security and monitoring, conduct security checks.
	Enhance and Expand web3.news	• Goal: Integrate community feedback into web3.news. • Validation: Establish feedback mechanism, implement enhancements, and monitor user satisfaction.
		• Goal: Support platform migration to web3.news. • Validation: Develop migration tools, ensure successful content transfer.
	Diversify Integrations and Starter Projects	• Goal: Build integrations with additional web3 services. • Validation: Develop integrations, test compatibility, and document for users.
		• Goal: Create a suite of starter projects. • Validation: Develop starter projects, test deployment process, and collect user feedback.

• Management of web3browser.io and web3.news can occur as necessary throughout the year, separate from the projected development work.
• The progression and timing of milestones may be adjusted or reordered in light of emerging risks, unforeseeable events, or community input.
• The arrangement of milestones is structured based on their dependencies.

Potential risks that will affect the outcome of the project:

• The S5 development might encounter technical hurdles in the transition to Golang, causing project delays and increased complexity.
• Pending issues with Bun.js, as detailed at Bun Fails to Load sodium-native NAPI · Issue #4170 · oven-sh/bun · GitHub, could result in the temporary reliance on node.js with worker_threads or a web workers alternative. The option to use WASM only partially eliminates this risk.
• Operational challenges with Stripe for payment processing might force a shift to alternative payment processors, affecting transaction handling.
• Acquiring a code signing certificate could face unexpected setbacks, potentially impacting software release and credibility.
• Unanticipated complications with renterd could arise, which have yet to be specified but could affect the project’s storage or networking functions.
• The project may face interoperability issues within the third-party ecosystem, particularly with hyper, Ethereum (ETH), InterPlanetary File System (IPFS), lavanet, or other networks that might be integrated in the future.
• The ABAO support might not be ported over from Rust to the required platform by @nemo within the necessary timeframe, as discussed in Bao Tree Support · Issue #17 · lukechampine/blake3 · GitHub, potentially affecting cryptographic or hashing functionalities.
• Developmental difficulties with IPFS on the portal could hinder incorporating support for its protocol, leading to potential limitations in functionalities or interoperability.
• If data is served anonymously over web2, there might be a need to implement anti-abuse measures to ensure data integrity and compliance with regulatory standards.

Development Information

Will all of your project’s code be open-source? Yes

Leave a link where the code will be accessible for review.

Lume Web Git Service

Do you agree to submit monthly progress reports? Yes

Contact info

Email: [email protected] or the email the foundation already has on file.

Any other preferred contact methods: Discord @pcfreak30

Early submission

We are aware that we are submitting this about 1-1.5 months before our current grant is completed. Our justification is that we need to ensure continued funding in 2024 (Jan) and cannot afford to have a large gap, as we have no other significant active income at this time. Lume Web is our full-time mission.

Additionally, our plan is for the end of our 2023 grant to finish with the web3.news MVP as a deliverable.

[mike76]

I understand there has been a lot of ups and downs this year.
This is your quote on Discord from November 13:

mine is not production ready yet and will be rewritten soon

Please correct me if I am wrong. You spent 80,000 USD of the grant money on learning various blockchains and realizing that Lume as you had planned it would not work. Now you are going to request additional funds.

Please indicate what you have achieved through the last year specifically regarding the Sia network.

[pcfreak30]

You spent 80,000 USD of the grant money on learning various blockchains and realizing that Lume as you had planned it would not work.

This is true and false, though it is out of context a bit. The portal is an MVP that works. I do not consider it production-ready, and my next steps are to rewrite it to have the portal support many protocols. Even if I did not rewrite it, it would still need more work to mature as-is. The comment you are referring to is based on my 2024 plans. It is not to say that the portal software does not work.

But Lume as I had planned would not work is incorrect unless you are referring to using Skynet, which I had to avoid, and that was a clear early risk that forced a pivot. The browser extension was a flop, as many in the Sia community reacted as such. That was a larger pivot, even though I completed the work. I have never stated that Lume is a failure or otherwise would not work.

All software is now MVPs that work but need more maturity and iterations to improve.

• The kernel technology: I see an opportunity to rewrite many internals while keeping the same high-level API. This will make the system much easier to work with.
• The portal: Needs a large refactor/rewrite to support many protocols.
• The relay: It needs a rewrite so that the plugins can run in parallel and ideally use a better JS runtime, which can also help with packaging it when it is time to market that component. That does not mean rewriting everything since much of the effort has already been done, but it does mean a large refactor.

There are three critical places where Sia is used now:

• The portal itself is used to upload and download all browser code with an anonymous account.
• S5 Network/websites. The browser demo supports accessing Sia-powered websites via S5. I am running my own S5 node and have lumeweb.com uploaded to it (to renterd), and lumeweb HNS can be used to access the site.
• And, in progress, is the community component this year, web3.news, which will have Sia- hosted websites aggregated there. That would be the 3rd important effort.

The following components have an indirect usage of Sia:

• The kernel tech downloads all files from the portal, which uses renterd.

The non-sia components, meaning they don’t have direct usage of Sia, include:

• The P2P network technology
• ETH support
• HNS support
• Lava RPC support
• S5 Registry (does not store on Sia like on Skynet).

Getting non-sia parts working was vital to working with the sia parts. It is an interconnected puzzle that requires everything to fulfill its goal.

So, in the big picture, Sia is powering the system and is being used as planned and as I originally proposed in my 1st grant. With the following in place, the next efforts that will connect this further to Sia are:

• IPFS support in the portal: With both the Foundations’ efforts and @alvinpreyes grant for IPFS support, I will be implementing it into the portal to support the protocol and create an L2 software that supports this, thus creating an opportunity, and, ideally, demand from that ecosystem.
• S5 support: Creating an S5 node directly into the portal to communicate with the S5 P2P network. This will ensure full support from Vup, @redsolver projects being built, and other community projects using S5.

After that, creating a portal dashboard, using the hyper P2P tech to sync files between portals, running a hosting service, evolving the demos, evolving web3.news, and creating the next end-user product.

So again, Lume works, is using Sia, and is ready to move to its next stage.

[PxC]

I feel the scope was too open ended for 2023. I recommend for 2024 one thing is the focus and the timeline and accomodations are geared for a Q1 2024 release of a finished product that fully incorporates Sia.

Personally, my preference is to get the Sia portal back to what skynet was intending as it served as a potential building block for other ideas. This project alone will have numerous ongoing challenges but could be monetized to alleviate the headaches.

[parox]

Thank you for the review on your past year. Like others, I would like to offer some constructive feedback and advice, I hope it can help you refine your grant and find success for your project.

Lume’s first year of funding was largely you working with dozens of different technology tools and networks and very little direct interaction with Sia software and APIs, thats totally fine, but now I would like to see any follow up Lume grant present:

1. A drastically narrower scope
2. A drastically clearer focus on Sia

The new grant proposal seems to learn very little from your retrospective on the previous year: Rewriting new portal software, rewriting the skynet kernel again ?, creating a web3 news service ??, running a cloud hosting service of some sort? Building with brand new alpha technology like Bun.js? This comes off as a rough list of brainstorming ideas that needs significant refinement and critique.

I would encourage you to cut 90% of it and focus on just one of these areas, and do so with a clearer focus on how Sia fits in. The proposal as-is does not set you up for success. Like your previous year of work, it wanders through far too many technologies and does not provide the necessary focus to deliver something tangible and useful to the Sia community. It is good to be optimistic, but having so many shiny new things to chase around prevents you from doing any single one of them well. Once you find success in one or two areas, you will naturally continue on to more things, with an even better understanding.

It is great that you have a grand vision for Lume as a company, but it would much more valuable and clear to the Sia community if you could keep your subsequent Sia grants much more narrow and focused, and related to Sia.

[pcfreak30]

Thanks @parox and @PxC

I can agree that a fair number of issues were hit this year. For my following proposal, I tried to be much more structured.

On a high level, though, from my perspective, the efforts I am doing big picture can be broken down into a few categories:

Sia Portal

The portal is now an MVP, and I have had to build it while the whole ecosystem is in flux. As you know, Lume started as an L3 project that got rugged with the rest of the community post-Skynet.

Right now, though, I have a clear direction on where to take that effort for the Sia community.

• Needs to be re-designed to support multiple protocols starting with S5.
• Needs to have IPFS support as a second protocol.
• It needs a user dashboard and billing.
• The community needs a reliable Skynet-like hosting service; thus, marketing is required.
• I see that A second approach to sync data via portals is needed, which uses hyper for the metadata.
  • To explain more, hyper has something called hypercores, which are P2P blockchains. S5’s approach shares links, but that may not work in all cases for tracking usage on uploaders and downloaders?
  • So the idea is to put all file metadata into a hypercore and keep it seeded. If a portal goes down, anyone can make a new contract and pay for the sectors as long as someone is keeping the log online.
  • This also solves data syncing between files in a different approach by copying ownership of the data rather than redirecting it to the host providing it or just downloading it from that host to your node.

The community efforts (web3.news) and L3 tech, which include the demos and the kernel tech, all rely on the L2.

The P2P relay is required as part of the L3.

Now let me clear up more confusion:

web3.news

This is already in progress, and an MVP is planned for this year. However, it will require further effort afterward. I view this as critical for the broader community to organize better and see the projects in the ecosystem, Sia or not Sia.

This site serves as a marketing effort and a public good that the Foundation (and Sia Community) can champion. The overall goal is to have project owners host their sites on this platform, and since the only way to have your project on web3.news is by hosting it on the Sia Network, then it is set to drive uploads and engagement with the tech.

Rewriting the kernel

Forking the kernel actually did not change a ton of the internals. Over time, a few technical approaches changed, but it was not a significant rewrite unless you consider merging code bases. I copied a lot of code, but many Skynet references are still in there.

What I have found, though, is that there is a better way to handle the low-level postMessage communication and simplify a lot of the guts by delegating it to the comlink library, which is about three files as an abstraction. I’m aware Skynet used post-me at some point, too (a similar library), though it wasn’t for the kernel.

As this is an internal change of the kernel and not high-level APIs, I don’t expect any module code to need changes other than deploying new versions.

Rewriting the relay

You make a point about Bun.js being early, though a key reason I have looked at it is that it replaces Vercel pkg, and both the web workers and worker threads use the same implementation.

I intend to rewrite the internals and probably clean up some technical debt in the process so that plugins use the same comlink to communicate with the parent process and aren’t blocking each other. All plugin code has been written, but it would have to be re-packaged for the lack of a better word.

Honestly, if bun causes issues, I can put it in node.js, but the long-term goal is to have it apt-get installable, not have a mess of files on the operators’ system, and not have the build process be a pain.

I saw Bun as something better to use, primarily as it supports native modules.

Cloud hosting

I intend on running a community service on the portal, with usage and the infra subsidized by the Foundation.

That is the cloud hosting service of some sort I am referring to. There is no solid Sia hosting option yet in the spirit of Skynet, and the DevOps code for such a service does not exist either.

Network Integrations

I am not looking to explore other blockchains unless something caused ETH or HNS to break. I am not a blockchain developer; those efforts were complicated as-is, and I would rely on upstream developers to assist. Any future efforts to integrate networks trustless would require the blockchain to meet the project halfway. I also do not see many chains ready to take that challenge. If i did integrate a new chain, it would have to take a few days, not a month.

Big Picture

These components/initiatives break down into L2 and L3 efforts. The long-term goal is a new internet sandbox and building all the needed infrastructure to get to that goal.

I started the route I did because I was building for Skynet, and then I had to improvise and figure out a new path.

Despite the chaotic journey, I stuck to what I promised in my original grant.

I explain all this to try and break everything down better on what does what and what my intentions are.

If the consensus is to focus on all L2 efforts first, I can re-prioritize Lume’s roadmap.

However, I also feel a second priority should be for the community site, web3.news to evolve, as it serves to help expand brand awareness of both Sia and Lume. In the macro, I feel we really need a place to get organized due to all the regulatory challenges. I view this as a grassroots effort that benefits everyone.

As it stands, I am planning to build the web3.news backend to use S5 tech and let things evolve from there.

Lastly, I hope the Sia community understands the long-term value and need for building a censorship-resistant web. Web content will not stand the test of time as long as ICANN is used.

Skynet did part of it, but it built it within a web2 sandbox and was caught off guard when it got heavily censored. We need to get back to that point, but better; we also need to go beyond it, or web3 stays DINO (decentralized in name only).

---

Based on my feedback, I will post a new proposal for the year with a smaller scope and revised budget below. I will focus only on creating a solid portal L2, building the DevOps, and launching a community hosting at the end. I will not be developing web3.news significantly further based on this, but I will maintain it as part of hosting expenses and deal with bug fixes. I will also launch a community for it sometime next year. I will focus on further funding for it after the portal is developed.

[pcfreak30]

Introduction

Project Name: Lume Web

Name of the organization or individual submitting the proposal: Hammer Technologies LLC

Describe your project:

Lume Web is our way of stitching together the vast web3 landscape. We’re building a hub for everyone to access decentralized content easily without relying on any central power.

Our aim with Lume is to be that one-stop gateway for all decentralized content in a genuinely decentralized way.

Our goals for 2024:

• Multi-protocol Sia Portal Software Development
  • Rewrite MVP software to support multiple decentralized protocols.
  • Begin with S5 protocol implementation in Golang; collaborate with @redsolver.
  • Use subdomains for each protocol to serve as API servers.
  • Add support for IPFS, with plans to include more protocols.
  • Aim for portal software enabling paid hosting and competition with large closed-source firms, maintaining democratic/decentralized access.
  • Introduce a single subscription service for diverse data management.
• Billing and Subscription Integration
  • Add billing support, initially incorporating Stripe, avoiding crypto transactions for users.
• User Interface and Marketing Site Creation
  • Develop a web dashboard and marketing site(s) for enhanced user interaction and portal promotion.
• File Sharing Functionality between Portals
  • Implement hypercore technology for a decentralized file metadata-sharing system.
  • Focus on on-demand contract creation for file imports and redundancy vs. pre-creating contracts like skyd did.
  • Keep P2P logs/blockchains public to ensure data persistence and resilience.
  • This approach intends to expose the sector data to the community for redundancy and ensure the portal can track who is accessing the data. S5’s network takes a different approach to this, which may make tracking user usage more complicated and does not currently share sector data.
• OAuth Integration with Ticket System
  • Enable linking accounts to a 3rd party ticket support system with SSO.
• API Documentation
  • Create full documentation of the supported portal API’s.
• Community Hosting Service Management
  • Set up a robust hosting service utilizing Sia’s decentralized storage, offering both paid tiers of storage services and a free but limited storage plan.

As the dashboard will be the most significant frontend effort, we are looking to get the following designed and implemented:

• Account Management
  • Login Screen (username/email, password fields, two-factor authentication)
  • Account Dashboard (profile and security settings, notification settings, account status)
  • Password Recovery Process
  • Account Creation/Sign-Up Process
  • Account Deletion/Closure
  • Design and User Experience Considerations (user-friendly interface, responsive design, security, accessibility)
• Subscription Process with Stripe Integration
  • Plan Selection Screen (overview of subscription plans, plan details)
  • Stripe Redirection for Payment (Stripe checkout integration, pre-filled information)
  • Confirmation Before Redirection (review and confirm plan selection, terms of service)
  • Post-Payment Processing (success and failure handling, email confirmation)
  • Subscription Management (dashboard for subscription status, modify/cancel subscription)
  • Design and User Experience Considerations (simplicity in plan selection, seamless Stripe integration, responsive design)
• IPFS File Management (powered by Sia)
  • File Explorer Interface (navigation pane, main viewing area, breadcrumb trail)
  • File and Folder Operations (upload, create, delete, rename, move)
  • IPFS-Specific Features (pin files/folders, retrieve IPFS hashes)
  • Search and Filter Functions (search by name, type, metadata; filter by size, date, etc.)Bun * File Previews and Details (previews for specific file types, detailed file metadata display)
  • Design and User Experience Considerations (intuitive layout, responsive design, performance optimization)
• S5 File Management (powered by Sia)
• File List Interface (displays file name, size, type, last modified date, flat file listing without traditional folders)
• Manifest-Based Navigation (virtual folder structure navigation for directories/apps)
• File Operations (upload, download, delete, rename files, manage manifests for virtual directories)
• Search and Filtering (search by name, type, metadata; filter by size, date, etc.)
• File Previews and Details (previews for specific file types, detailed file metadata display)
• Public Key Management
  • Key Registration and Setup (key upload interface, key verification process)
  • Key Management Dashboard (list of registered keys, add/delete key options)
• Admin Panel
  • Dashboard (overview and statistics, quick access panel)
  • User Account Management (subscriptions, billing)

For more information, you can visit https://docs.lumeweb.com/ or check out a snapshot of the documentation at the time of writing at https://git.lumeweb.com/LumeWeb/docs.lumeweb.com/src/commit/e5520d24023895d14d3f950f9f3f45b7d34b265a.

Additionally, as the Lume project is slowly maturing, we are attempting to be more structured and organized in project management.

Who benefits from your project?

The project benefits:

• Sia community/users by:
  • Enabling an L2 portal software that gives access to multiple decentralized protocols and stores data on the Sia network.
  • Starting a hosting service for users who do not want to run a node themselves.
  • Evangelize the Sia network and spread awareness that it is the data backbone of web3.
• Data hosters by:
  • Providing cheap, reliable, decentralized storage via multiple distribution protocols to the broader community.
  • Democratizing access to FOSS file portal software that will enable pinning on multiple protocols, ensuring censorship resistance.
• DApps/Web Apps by:
  • Enabling web apps to be hosted on the Sia network through several distribution protocols, which should ensure censorship resistance and grow network demand.

How does the project serve the Foundation’s mission of user-owned data?

For 2024’s focus, it provides a reliable platform for Sia users to host their data for both S5 and IPFS-based protocols. It also creates a community service based on the software built.

The web needs paid data storage to ensure things stay online and censorship-resistant and users need private data storage that retains their privacy and self-sovereignty.

Grant Specifics

Amount of money requested and justification with a reasonable breakdown of expenses:

Total Budget Request For the Year: $89,000

Upfront Budget Request: $47750

The initial funding needed includes:

• Derrick Hammer’s Q1 Salary: $13,750
  • This is Derrick Hammer’s first quarter salary.
• Project Budget for the Year: $34,000
  • Requested in full to efficiently manage contractor payments and adapt to variable project demands. The breakdown of this amount is as follows:
    • Contractor Payments: Essential for project milestones. Work must happen gradually throughout the year to build the frontend as the backend is created, and the effort will not be predictable per quarter.
    • Infrastructure: Allocated monthly, subject to change based on testing and scaling needs.
    • Storage: Allocated based on demand.
    • Marketing: Marketing will be created throughout the year to drive interest and awareness of the project, of Sia, and through the launch

Detailed Expenses Breakdown

1. Salary Expenses: $55,000

• For Derrick Hammer
• Justification: This represents an inflation-adjusted increase from the previous year.

2. Annual Project Budget: $34,000

• Design and Development Work: $20,000
  • Projects Covered: Includes development and launch of a public portal with Sia-powered storage services and marketing websites.
• Marketing: $9,000
  • $2,000: SEO and Content Writing for marketing sites
  • $2,000: Social Media Marketing
  • $5,000: High-quality Introduction/Explainer marketing video
• Server Infrastructure Costs: $4,000
  • Note: This is subject to change based on testing and scaling needs.
• Sia Network Subsidy for Storage: $1,000
  • Contingency: Additional funds may be necessary if demand exceeds expectations.
• Marketing and Development budgets may be re-balanced throughout the year based on the projected needs of the project
• A hosting subsidy for running the community service will be requested in a grant for 2025

Timeline with measurable objectives and goals

The items mentioned follow the OKR (Objectives and Key Results) methodology:

Q1: Build core portal: $13,750 (Paid as part of up-front payment)

#	Objective	Key Results	Validations
1	Develop Modular Architecture	Complete architectural redesign with support for multiple protocols	N/A
2	Implement S5 and DNSLink Protocols	Integrate and test S5 and DNSLink functionality	Performance evaluations and small-scale user feedback sessions

Q2: Integrate hypercore for file syncing: $13,750

#	Objective	Key Results	Validations
1	Integrate Hypercore for Enhanced Metadata Sharing	Complete integration of Hypercore for metadata access and sharing	Internal testing for integration Portal should be able to import metadata from a dead portal Portal should be able to import a file from another live portal

Q3: Add IPFS Protocol and Billing: $13,750

#	Objective	Key Results	Validations
1	Incorporate IPFS Support	Integrate IPFS into the portal	Technical checks and early user feedback Should be able to browse Sia-stored IPFS content from public gateways
1	Integrate Billing	Complete integration of Stripe billing for subscriptions	Ensure basic billing and subscription functions work

Q4: Community Hosting: $13,750

#	Objective	Key Results	Validations
2	Establish Hosting Environment and Monitoring Systems	Set up hosting environment and monitoring systems	Stress testing of hosting environment and monitoring system evaluation

• The progression and timing of milestones may be adjusted or reordered in light of emerging risks, unforeseeable events, or community input.
• The dashboard will be built gradually throughout the year as milestones are hit.

Potential risks that will affect the project’s outcome:

• The S5 development might encounter technical hurdles in the transition to Golang, causing project delays and increased complexity.
• Operational challenges with Stripe for payment processing might force a shift to alternative payment processors, affecting transaction handling.
• Unanticipated complications with renterd could arise, which have yet to be specified but could affect the project’s storage or networking functions.
• The ABAO support might not be ported over from Rust to the required platform by @nemo within the necessary timeframe, as discussed in Bao Tree Support · Issue #17 · lukechampine/blake3 · GitHub, potentially affecting cryptographic or hashing functionalities.
• Developmental difficulties with IPFS on the portal could hinder incorporating support for its protocol, leading to potential limitations in social media functionalities or interoperability.
• If data is served anonymously over web2 from the portal directly, there might be a need to implement anti-abuse measures to ensure data integrity and compliance with regulatory standards.

Development Information

Will all of your project’s code be open-source? Yes

Leave a link where the code will be accessible for review.

Lume Web Git Service

Do you agree to submit monthly progress reports? Yes

Contact info

Email: The email the Foundation already has on file.

Any other preferred contact methods: Discord @pcfreak30

Early submission

We are aware that we are submitting this about 1-1.5 months before our current grant is completed. Our justification is that we need to ensure continued funding in 2024 (Jan) and cannot afford to have a large gap, as we have no other significant active income at this time. Lume is our full-time mission.

Additionally, our plan is for the end of our 2023 grant to finish with the web3.news MVP as a deliverable.

[pcfreak30]

To answer your request, the current iteration of the portal is very basic. no web UI/dashboard, and the large uploads probably would not work. So, there is no easy way for others to use it outside as an API server. In a rewrite, the UI will take nearly the whole year from my hired team as the backend gets fully built.

The issue with “mysky” is that it is the version of Skynet that failed. It is the trust-required version of faux web3 before the kernel.

Serving Skynet how it existed with a CDN and anon file downloads is asking to get yourself censored. I already experienced this and am trying to learn from past errors. If I had to allow this, it would have to include anti-abuse and work in a way that couldn’t be used maliciously… I got the email reports from upstream of bad actors using portal.com/file or file.portal.com as phishing pages.

The software could be modified to allow it, but I will not focus on re-creating Skynet exactly how it was since that would be repeating mistakes.

[Danger]

Being asked, I provided feedback to pcfreak in a private conversation that was similar to what others mentioned here. It’s nice to see the original plan changed and focused more on the Portal technology.

I have good understanding for his focus on web3. While the general public can think whatever they want about web3 and even I don’t have a positive opinion on most of the web3, its development matters. Decentralized storage at the present has only limited reach, but decentralized internet will unleash it completely. However, decentralized internet CANNOT really exist without decentralized storage and that’s why Portal using Sia as the storage backbone should be the absolute priority now, everything else can wait.

Get the main feature (uploads/downloads with minimal UI) functional, start early testing in closed alpha, fix the Sia (S5) implementation bugs and then start adding more features and quality of life stuff, followed by a kind of “closed” beta, which will let anyone join, but do this without marketing just by posting it in the Sia community to monitor its interest, reach and get some people in to solve issues that weren’t visible with just few testers. Because when this launches for real and starts doing any marketing, you want people (and especially those with followers/communities behind them) to not just have a “good enough” experience. You want them to love it, use it and share it.

As I suggested in a DM, Skynet’s free tier giving 100 GB free would probably ruin you, so make the free tier like 5-10 GB and have the first paid tier being at something like 100 GB for $0.99. Maybe it’s just my opinion, but I think it would help the adoption. It’s a detail now, but important one I feel.

Also, one thing that I don’t see anyone talking about, but I think that not many people have experience with the dark side of Skynet, which is the blocklist and its management. I can easily see that the Portal development, but mostly its maintenance, will present bigger challenge than expected and I don’t see it mentioned in the proposal/budget. I’m sure everyone remembers the post on Sia blog about obstacles of existing internet infrastructure when it comes to stuff that people upload and share. Even if Lume has different approach and the technology is different, someone will still need to handle it and it shouldn’t be distracting the developer.

Don’t expect to be done with portal in 3 months, that’s where the real challenge will begin. In Q1, I would expect maybe closed alpha that I mentioned above, then community-beta in Summer and maybe an actual launch in Q4 after everything has been tested and fixed, including the payment methods and at least a basic ticket system (this is actual business, so support system must be in place).

And finally, I want to comment on the original proposal’s Browser which was I believe idea for future. I think this is something we really need, but it’s still early and can wait. What we need NOW are some products that are easier to attract users and the Lume Portal sounds just like that, to fill the hole left after Skynet Portals disappeared.

[mike76]

I agree with @Danger. Marketing budget needs to be restricted or removed completely from this proposal, at least for 2024. Focus should be made on the development and getting something working without issues first.

[parox]

Hi thank you for thoughtfully considering all the feedback, the revised goals focusing on only the Portal look much better, and doing a solid job on the portal is more than enough work for a year. I would really like to see you focus on making the best portal experience possible, and once that is locked in, move on to the experimental browser software which can benefit from the solid portal infrastructure. Until the portals are implemented and adopted the browser kernels etc are premature.

Here are some more revisions/suggestions for the portal development section and logic phases that are really clear and build well on each other, where each step can be demonstrated, deployed, and shipped to users before you move on to the next. A specific feature/milestone that you omitted that I would like to see done very early on is basic Sia renterd support before moving on to S5 and IPFS. Hope this breakdown helps as you refine your proposal.

Phase 1: Core Portal development (This grant)

1. Accounts system

Before getting into protocols, get the basic core logic in place.

1. Users can create accounts, authenticate, view their own data.
2. Users each have their own file explorer that will be the basis for viewing user files. The file explorer will be a unified view of files where files can be downloaded and shared via any supported protocol.

2. Store and retrieve

First make sure solid storage and retrieval is in place. Start with native renterd support, then add S5 and IPFS.

1. Begin with basic Sia support. User should be able to store and retrieve files directly from Sia/renterd.
2. Move on to S5 support. Users should be able to store and retrieve any files via the S5 protocol which can directly leverage renterd’s S3 interface.
3. IFPS support. Users should be able to store and retrieve any files via the IPFS protocol. Note: you will be able to leverage our upcoming IPFS node software here.

3. Discovery and sharing

Once files can reliably be stored and retrieved via supported protocols. Move on to portal-to-portal discovery and sharing. Because S5 is largely a file discovery network, and you will already have it integrated start there as a basis for sharing.

1. Users can add files to their file list via S5 CID. This leverages the S5 network for file discovery.
2. Users can add files to their file list via IPFS CID. This leverages the IPFS network for file discovery.
3. Users can add files to their file list via any CID, discovered over Hypercore.

Phase 2: Service development: billing, ticketing, etc (Maybe this grant)

Move on to billing, ticketing, etc after you have a working and viable product. Before this, test your portal with community members etc.

Phase 3: Novel decentralized patterns: browser kernels, forks, etc (Definitely not this grant/year)

After you have built and refined a powerful Portal software that has seen some adoption, move on to bigger picture longer term projects that can build on the now available infrastructure.

[pcfreak30]

@parox

Sia renterd support before moving on to S5 and IPFS

Sorry, to clarify, everything will be directly using renterd, with the portal handling everything. So when I say S5 or IPFS, I was implying direct use of renterd in the backend. S5 may support many backends, but I’m doing the opposite. It is sort of what @alvinpreyes or you are doing with IPFS/Sia, but unified into one multi-protocol system. Implementing S5 means porting the networking code, which isn’t as complicated as hyper or libp2p, especially as I have a TS version now, and porting the API HTTP endpoints.

Based on this, is why I have the milestones set that I do. It is expected that things should be stable by the time any community service is launched.

Please let me know, knowing that, what you think should be different?

Thanks.

[pcfreak30]

Also, another clarification. Hypercore is probably 50/50 on discovery vs redundancy. Portals can find files from the p2p net via other portal logs or S5… But it is also intended so that if a portal goes down, that log of sector data can stay seeded BitTorrent style by a 3rd seeder node + community, and anyone can secure the data.

I just want to ensure the reasons for the functions are correctly understood.

[pcfreak30]

@parox

Users should be able to store and retrieve any files via the S5 protocol which can directly leverage renterd’s S3 interface.

Sorry, I just now saw this reading.

I currently am not planning on using the S3 object API unless there’s a good reason to use it over the worker object API. I am planning on storing all data in the same way that S5 does with blake3 hashes. blake3 will power most things regardless.

To do anything else would be to define custom APIs, which I want to minimize. I may create APIs needed for the dashboard/admin, but I want the dashboard to use the same APIs as everyone else so that the system will be trust minimized (The opposite of the original Skynet).

For IPFS, the implementation will depend on seeing the W.I.P. efforts being done now. I also don’t see a reason to use the S3 indirection there.

So, it is not clear to me what you mean by native renterd support.

Thanks.

[pcfreak30]

One other detail. The ticketing system I am most definitely not building. I will do an SSO auth to an open-source system via OAuth and integrate support in the portal configuration. I’m sticking to the portal, not the aux systems.

[pcfreak30]

@mike76 @Danger, I can agree to reduce the marketing and remove the social media. The video, I believe, will be beneficial, and the SEO and content writing would be needed for the marketing sites, and they would be seen as frontends to the portal (not the dashboard).

I mention multiple sites as I have plans for a second marketing site to benefit the community, but I cannot publicly give info on it (PvP). However, I can provide it to the foundation privately through existing channels.

All of this assumes that community services will be accepted as part of the grant in Q4, as a dashboard does not include a front-end website.

As for the blocklist, that should be relatively easy to implement as an API, db table, UI, etc. The complexity comes from automation, AV scanning, and the systems Skynet had.

[Jiggythegrey]

Interesting updates on Lume @pcfreak30, looking forward to everything Lume is bringing to the space.

[Kinomora]

Hello @pcfreak30,

Thanks for your proposal! The committee is requesting more information and some changes to this grant proposal:

First, the committee has been in favor of streamlining proposals lately, and as a result would like to see the marketing budget removed entirely, as well as the removal of the marketing websites portion of the “Design and Development Work” in your budget request. The Foundation is hesitant to provide marketing funds for projects that haven’t been built yet or are still in development, and we’re still speaking with our lawyers to determine under what guidelines offering marketing funds as part of grants is reasonable.

Next, the Foundation will not provide the entirety of the Project Budget up-front as we did last year, instead preferring to divide your entire budget request by four and providing the first amount up-front as usual. We would then provide the rest after successful completion of subsequent milestones. Your milestones are currently listed in three month increments, so the timing lines up well there.

Finally, we’d like to see more demonstrable milestones. We definitely appreciate your format of Objective, Key Result, and Validation, but would prefer to see more ways to concretely and easily verify progress at each three month mark.

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

[pcfreak30]

I am requesting feedback and clarifications on 2 things so far.

Marketing

I think there may be confusion over this definition and my intentions. From my perspective, I view a front-end website effort, meaning design and code, like the original Skynet portal, as a marketing site.

I view marketing efforts like what I listed as SEO, content, or videos as separate things.

Generally speaking, a portal needs some front end to be usable. So I would like to understand where the line is that makes that be considered marketing such the foundation does not want to fund it at this time. A front end for the portal is generally required for usability. I intended to re-create the Skynet portal in spirit, including a frontend site that could have pricing but would also have login/register and an about, etc.

The second marketing site I was potentially planning would be the same portal but targeted at a different market in web3 that I could see potentially falling more under the Foundations’ definition.

I was told there might have been confusion about the marketing site’s purpose, so I am, in turn, also confused about the new limitations in this situation.

Thus, I request feedback from the committee on this.

Milestones

I have created some updated milestone validations based on advice from the Foundation and am requesting feedback to know what needs to be improved, to be more detailed, or to know what’s missing.

---

Q1: Build core portal

#	Objective	Key Results	Validations
1	Develop Modular Architecture	Complete architectural redesign with support for multiple protocols	N/A
2	Implement S5 and DNSLink Protocols	Have a functioning S5 node with DNSLink support	Should be able to upload and download small files Should be able to upload and download large files over TUS Should be able to login/register with S5's account API's Should be able to import a file from another peer and store in the portal's contracts Should fully support all HTTP node APIs. Admin APIs may be skipped or mocked until they can be routed internally for the dashboard functions. Should fully support the S5 registry Should have API documentation, in collaboration with redsolver as needed

Q2: Integrate hypercore for file syncing

#	Objective	Key Results	Validations
1	Integrate Hypercore for Enhanced Metadata Sharing	Complete integration of Hypercore for metadata access and sharing	Portal should be able to import metadata from a dead portal Portal should be able to import a file from another live portal Multiple peers should be able to seed the logs of portals

Q3: Add IPFS Protocol and Billing

#	Objective	Key Results	Validations
1	Incorporate IPFS Support	Integrate IPFS into the portal	Should be able to browse Sia-stored IPFS content from public gateways Should be able to upload files to the portal *Undecided*: you should be able to download files from the portal. This depends on the needed APIs for IPFS, as it would be for logged in only users, and should use a standardized API. *Undecided*: Implement hypercore sharing of IPFS files. This will be determined when implementing if it makes sense to do so.
1	Integrate Billing	Complete integration of Stripe billing for subscriptions	Ensure basic billing and subscription functions work Ensure support SSO works with ticketing system

Q4: Community Hosting

#	Objective	Key Results	Validations
2	Establish Hosting Environment and Monitoring Systems	Set up hosting environment and monitoring systems	Should have monitoring setup of the portal usage metrics based on existing grants Should have any possible alerts setup, storage, or money, based on existing grants, and implement if needed, assuming the required webhooks exist. Should have devops infrastructure created and documented for others to setup a portal Should be successfully stressed tested that large data can be both uploaded and downloaded