Grant Proposal: Lume Web

Hi everyone, I’m here to vouch for Derrick. I’ve personally witnessed his commitment to & passion for Lume Web. Discussions with Derrick & exploration of his code have led me to believe in his technical capabilities. I can also attest to his support for Sia and Skynet over alternatives.

I believe his contributions are very valuable & would love to see him funded.

This is a project we should all support! The space needs more projects like this. PCfreak30 has a clear vision and I believe many of us agree that this should be developed.

This space has problems that must be addressed if we want this space to stay “ours” - our own. PCfreak30 has the integrity need to head these problems. We need more “creators” giving back to the space rather than trying to create a quick buck.

I think I can speak for everyone when I say, I tip my hat to you friend and keep the course steady; we believe in you and what you are doing.

Not to be biased, but anyone who truly cares about web3 should support Lumeweb in all possible ways. Derrick takes pride in this project and this funding will go a long way to fuel the journey of enhancing data ownership, web freedom, accessibility, and anti-censorship to end users in a low to no-friction experience. Go lume!!

Conditionally approved - The committee approves this grant subject to the following conditions:

  • Scope : We ask that you remove the web browser fork milestone. The scope of this grant is already large enough without it. Feel free to apply for a separate browser-focused grant after other milestones have been completed.
  • Payment : Provide justification for why payment must be received as a lump sum, particularly your salary. We are willing to provide some payment up-front, but not 100%.
  • Timeline : Provide milestones for each quarter of 2023. We need at least a high-level list of objectives for comparison to the results you report.
  • Reporting : We ask that you provide progress updates on a monthly basis.

This conditional approval is recommended by the Grants Committee and will subsequently need to be approved by the Sia Foundation Board - the expectation is that the Board will approve the funding recommendation of the Committee.

5 Likes
  • First, the scope I can agree on is large. Some of that was telling my intentions, more than what I feel WILL get done in a year. This will be getting reduced.
  • Second payment upfront has been more due to market volatility/risk so far in the past. But I can recognize the foundation will not have the issues I have had to mitigate for. What I am interested in, though, is getting my salary in at-least quarterly payments and getting the project funds upfront since I will need to handle that as needed. Where that money gets spent specifically is not known yet, as I do not have any contractors lined up. The salary part is more, so I have a financial buffer for myself and am not relying on month-to-month payments. I would like six months, but I can settle for three.
  • Timeline, I can agree with that. PM is my weak spot. My one question, though is, will my grant/payment be withheld if I fail to meet an objective in a quarter, if I have to change course due to the community or a macro situation that requires a new priority, or if I have to re-order the priority of work?
    • There is potential for black swan events as well, like if there’s something political between the Foundation and Skynet Labs that forces me to change course to ensure Lume survives :upside_down_face:

So just looking to ensure I have all expectations set.

Kudos.

1 Like

Hi pcfreak30,

The Committee has approved the delivery of the Grant in quarterly installments for your salary, $12,500 per quarter for 4 quarters, as well as an up-front payment of $30,000 for project funding.

Additionally, it was decided that falling short of an objective or milestone would not constitute disqualification for the Grant as long as progress toward the objectives is shown via progress reports and code submissions. If the scope or the viability of the objectives changes significantly, the Grant may need to be adjusted appropriately.

Finally, we’re happy to confirm that the board has approved the Committee’s recommendation should all the conditions be met and agreed upon.

1 Like

Below is an updated set of goals/scope and timeline for 2023.

Proposed Scope/Goals/Milestones

  • Browser Extension (2022-forward) - continue building, testing, and evolving.
  • Portal Development - making web3portal.com the primary community Skynet portal and improving how usage and costs are tracked and ease of use.
  • Educational & Community Sites - design, plan, and launch community-managed sites with the provision of assets already secured for the project.
    • A potential list of sites for 2023 includes:
      • cryptoguide.org
      • web3guide.org
      • web3.news
      • btcguide.org
      • web3forum.xyz
      • defiforum.xyz
      • web3ethos.org
      • web3storage.com
    • Half of these sites should be completed, with at least some community crowd-sourced content, but it is possible that all the above sites will be done in 2023.
    • These sites have been given a rough priority order based on the community from the following poll, and descriptions for each can be seen: Lume Web - Education & Community Site Priorities
    • The community priorities will change over time as more people give their voice, so the poll will act as a rough consensus guidance on what website to build next.

Quarterly Timeline

End of Q1 2023:

  • Publish the first version of the browser extension with a basic UX and project-hosted relay node
  • Complete DevOps and CI infrastructure for building and distributing web relay software packages at web3relay.io
  • Trustless download support in Skynet
  • Account creation via the extension to web3portal.com
  • Get relay software on multiple platforms, such as Digital Ocean, Vultr, Linode, DAppNode, Flux (hopefully), and more. This could take most of the year to coordinate and develop since it relies on other projects or companies. This may include custom UX development to have GUI too.
  • Deal with various portal tasks as needed and work with the foundation on skyd.

End of Q2 2023:

  • Continue expanding the relay network in both platforms and in distribution and work to get project partners.
  • Continue development on relay based on feedback and testing
  • Continue development on extension based on feedback and testing
  • Start on automated code testing systems for code bases if things are stable enough. If not ready, this will be punted to a future quarter.
  • Spend effort on better monitoring for portal usage, cost, abuse, and readiness for the sia v2 fork. The exact details of actions needed for the portal are unknown unknowns. Still, the portal must be considered stable and performant to handle all community and paid traffic. This may also include (open) development on the portal dashboard and billing to keep operations running smoothly.

End of Q3 2023:

  • Continue efforts from Q1/Q2
  • Start planning, designing, and developing community sites based on community priority.

End of Q4 2023:

  • Continue efforts from Q1/Q2/Q3

Q3/Q4 has risks of significantly changing if issues with the Skynet infrastructure layer force a pivot to keep Skynet alive.

2 Likes

Hello,

I am requesting for my first salary payment ($12,500) to be delivered in 2022 in Siacoin and to delay the $30,000 payment to January 2023 (First quarter) in Siacoin.

Thanks.

1 Like

Hello pcfreak30,
The Foundation is excited to let you know that the Committee was satisfied with your refined scope and proposed timeline. Additionally, your request for the variation on funding ($12,500 upfront then $30,000 Q1-23 then $12,500 Q2-23 thru Q4-23) was approved and you should expect communication regarding paperwork to be provided to you by the Foundation shortly.

We also understand that you are unsure whether the Grant would be more appropriately provided as either Siacoin or USD though that is not an important condition of the grant and can be discussed with the Foundation as needed.

Regards,
Kino on behalf of the Foundation and Grants Committee

3 Likes

Hello,

Due to the recent black swan and (extremely) bad timing by the Foundation in sending my first salary payment, I am requesting an additional $2,000 USD, as I received a bit under $10,500 USD between the flash crash and exchange fees.

Thanks.

3 Likes

December Progress Report

Repos:

Progress:

Progress so far has been switching to the Protomux and Protomux-RPC libraries and building a new RPC protocol based off of it from the original RPC design. This has the benefit of enabling any plugin to define a custom protocol due to the new PluginAPI.protocols at relay-types/plugin.ts at 7c3c193b4ac571dd7e66a56e2289cfcb5c29d9ae - relay-types - Lume Web Git Service and relay-types/swarm.ts at 7c3c193b4ac571dd7e66a56e2289cfcb5c29d9ae - relay-types - Lume Web Git Service

The entire PluginAPI has been redone as well and now supports namespaced events via eventemitter2 relay & plugin wide. Example: relay/plugin.ts at 1b78d0e696f57520fb717d839c648e400d7bbe83 - relay - Lume Web Git Service

Next, various DHT libraries have been created and will likely continue development. It is still being determined if all will be used, so testing is needed to determine the best RPC cache performance strategy.

The RPC client side is split between two packages. The rpc which overrides a few things from protomux-rpc, and rpc-client, which used to be dht-rpc-client and has been near-completely rewritten.

Several DevOps packages/repos have been created for CI that are docker tooling or other automation and are deployed and hosted by the Git server itself (Gitea).

Next, I have started on a peer discovery system that is very simplistic in design and modular. I started with what felt most common sense, BitTorrent, but utilizing the DHT BEP44 spec did not work in the browser because the relay needed a WebSocket proxy. Using torrents for this would be too easy to DDoS and abuse the tracker systems based on a dummy INFOHASH, so this is being abandoned but not deleted since it can still serve use outside the browser, even if that’s redundant.

The first helpful peer discovery source as I call it, is IRC. There are IRC servers that support websockets now! This means browsers can directly talk to a supported IRC network. Lume has found an initial IRC network willing to partner with and host the project and is fully informed of our mission.

At the core, all a discovery source does is return a host and port for a given pubkey. So you could theoretically go as far as using snail mail to “dial in” to the network. In the future, I look forward to finding many out-of-the-box/unorthodox methods to enable censorship resistance. As long as the browser can directly connect and do so without an account, API key, or security restrictions, it can be used as a means to discover a peer/relay.

Lastly, work has started on a registry relay plugin. This will replace the Sia registry for everything. The Sia registry may find use in the future, but as it is very niche and economically expensive at scale, it can only have very targeted use cases. What is being designed is an implementation of @redsolver’s registry system in S5, which itself is still a WIP. This is DHT based, and it is expected that paid sia portals and other actors who can afford to host some or all registry entries will be the ones to take on the burden.

The plugin will support Memory, LMDB, and Redis backends.

One final thing to know is Lume is now operating with BIP39 and BIP44 and has registered a coin ID at Add Lume Web to SLIP-0044 by pcfreak30 · Pull Request #1469 · satoshilabs/slips · GitHub to ensure adoption and interoperability with the industry. Lume will not be using “MySky” in its current form, and anything moving forward will just be a “Lume Web” account. Specific components or approaches of MySky tech may be used, but they will only be internal details.

The coin ID was not randomly picked, so kudos to the person who solves the puzzle :wink:

Outreach

I have done outreach to a few early projects to make them aware of Lume Web and the tech being built to onboard and partner with them in the coming year or later. One established naming project also outreached to me.

What’s next

Continue developing the registry plugin, refactor ecosystem libraries as needed, and then start on the browser side of development, which will require much rebuilding to shift to the new primitives.

It has also been confirmed by @nemo that getting trustless, A.K.A. verified streaming will be possible in sia. S5 currently uses blake3, and while it has been stated that blake2b is possible for this, blake3 would be the ideal route and ensure unity given its nature and the GitHub - oconnor663/bao: an implementation of BLAKE3 verified streaming library.

The sooner this can be ready, the better for the project.

Renterd will either need to be forked, or, if possible, a plugin system created so projects such as Lume or @mike76’s Sia Satellite can extend without forks or needing to use the renter as a library.

The direction of Lume is pivoting such that almost nothing will be compatible with what I now consider Skynet v1. S5 is doing the same and is breaking compatibility. The future “Skynet” idea will likely turn into a fork as a Layer 2 project with a unique name. Those details are still undecided, but it is clear Skynet is dead in its current form, and only the bones, if any, will be picked.

As such, future timeline plans have changed such that there will be no plans to maintain the existing portal software after the hard fork or keep uptime for it. Two parallel systems will likely exist. The new software can be ported to the Skynet v1 portal stack to maintain continuity at a fundamental level, if practical. Otherwise, the portal will be shut down until it is ready, which will also mean partial Lume downtime.

The current primary focuses of Lume as of now are:

  1. Web3 Hypercore P2P network and related ecosystem software
  2. Browser extension and related ecosystem software
  3. Educational and Community Sites
  4. A new web portal software stack

The portal stack will overlap @mike76’s Sia Satellite or use some of the code. It is too early to say. This may also become a project for 2024. But it will be required for Lume as a network to function since web accounts will use sia for the browser extension. If it is done this year, it will take priority over the extension work since it is effectively the backend service that the browser needs to save data. The extension will have already had its initial release.

Overall these unknowns are the bright red line initially outlined for Q2 of 2023 (this year now) on where things may go crazy.

There have already been many bumps in the road, and many more are expected, given the unknown. The overall vision and goals of the project are the same, but how we get there has changed and may continue to change to ensure we can get the best outcome.

This concludes my report :upside_down_face:

Thanks! The committee appreciated the extensive update and is pleased with your progress.

January Progress Report

Repos worked on

Progress:

Progress this month has felt somewhat slower, but some crucial achievements have been made.

I successfully got Handshake SPV (HSD node) to sync over the P2P network to a full node using a new library that acts as a SOCK proxy. This is very important as it will form the basis (with PluginAPI.protocols) for other nodes (blockchain and non) to communicate/bridge on the network. This took a while to do (and slowed me down) because, like sia, the node takes time to sync before I could proceed. I attempted to shortcut it, but I have found that future work on the HSD node software will be needed to improve the sync process (not utreexo) for relay operators, as this is a critical aspect of deploying a node for web3 users. This may be through supporting a bootstrap download of the chain.

Next, I got peer (p2p) node discovery working over IRC. This is the first of many peer discovery methods long term. This is also specifically working in the browser. It takes around 1 second to do the discovery actions.

Next, I have rewritten much of the p2p browser code (kernel modules and related libraries). I have also started forking some of the Skynet kernel code. However, not all of that will be used as a fork, possibly until after the L2 has been completed.

Other things done are creating misc library helpers and other aux client code.

Next:

  • Continue browser development.
  • Get Handshake syncing and querying in the browser, then work on the DNS system and integrate it back into the extension.
  • Start on UX development when far enough along, and hopefully have an MVP out by the end of Q1.

This concludes my report

Thanks @pcfreak30! This is a great report and we’re going to model our monthly report criteria after this moving forward.

Thanks pcfreak30! Going forward, please submit your monthly report ideally by the 2nd of every month. Thank you!

February Progress Report

Repos worked on

Progress:

I have gotten handshake syncing and querying fully working in the browser. This includes querying it with the DNS layer/kernel modules.

Significant work has been done on the DHT communication as well as the related client libraries that are used for DHT and DNS. Redundancy/resiliency and connection retries/reconnecting were added to the different layers up to the HNS module itself.

Next, I have started researching and implementing IPFS. I actually got a decent-sized roadblock with doing this and got no real help solving it. I made a breakthrough today (3-2-23) and so I hope that can get finished this month.

Next, I started on a website rebranding and redesign via contracting. This will hopefully finish within 1-2 months.

Next:

  • Implement IPFS support
  • Implement ETH RPC support and ENS DNS
  • Test HNS, IPFS, and ENS together and integrate them into the extension
  • Work on extension UX once all subsystems are functional

A Q4 release may not be hit due to the roadblocks encountered, especially if ETH takes 1-2 weeks of work to figure out.

This concludes my report

The committee has reviewed your report and is pleased with your progress. The committee did note that they would like to better understand Lume’s progress as it relates to Sia, as this report was largely unrelated to Sia development.

So let me clarify some things:

  • The kernel modules are stored on sia via my portal. Given the current ecosystem crash with Skynet and renterd being made, I plan on whitelisting specific kernel CID’s to have the extension modules operate. So sia is being used practically in the backend. This has not changed from how Taek designed it.
    • Nearly everything in the extension is powered by the kernel system. IPFS, HNS, ENS, DNS, IRC discovery. All modular logic is done like this vs. embedded in the extension code itself. So sia is storing nearly all code that is powering the system.
  • I have decided to not do anything related to web3/lume accounts on the portal or further develop the sia integration on the L3 browser side given the hard fork and the fact that I have to build a new L2 and completely dump Skynet v1. It is a waste of energy to do so. I am waiting on renterd to be in beta too.
  • Much of what is required right now is at high level to interop with the wider ecosystem to achieve Lumes goal and to promote sia in the industry. HNS, IPFS, and ENS will be the base minimum required for a usable web3 extension and experience (DNS) for a tech demo/prototype release. Other integrations will come but at a later time. Playing that by ear mostly.
  • I will be getting the extension out, the relay out, and marketing sites for awareness for both of them. That is the current roadmap.
  • After doing this, I will be switching back to building the L2 layer, building the client code needed for that, and wiring that back up to the L3.
  • I am taking this route to stick to getting the extension out as planned in an MVP, as a semi-centralized service, with the expectation of things breaking in a few months and having to transition to the new tech being built by the foundation, me, and the community.
  • I will also be looking to see at what point the community/education/news sites make sense this year. Given the rapid ecosystem and community changes, I am focusing on getting the extension, basic relay, and new branding/awareness for the project done before asking what is next.

I hope this makes it more clear why I am not directly focusing on sia right now, even though it has involvement and will be a foundational aspect of the project mid to long term, as I originally proposed in my grant.

Kudos :wink:

From a technical perspective, great, I’ll let other delve into the usability of this. What I want to know, is how you’re going to attract users to pay money?

Early on in the Sia project, Lightspeed essentially laid out a business model that at the time was Filebase. We were then and still are too busy to take creating something like Filebase, so we’re very excited they did. In the end though, the thing they did that others have not, is (a) have a way to pay that users already know and use (credit cards and paypal etc) and then (b) give them tools they’re use to using to interface with the medium of storage.

These two things we said from the beginning would be required for Sia to be successful and they failed to do those things, and still fail to do those things. Providers who have, have gained traction and customers.

Lastly, of that $80k, how do you intend to the market the end result?

1 Like

Thanks for asking what’s often seen as the elephant in the room.

I did touch on this in my original grant, and just to be clear in case you only skimmed this thread, my grant was approved and I’m several months into development.

First, know that I’m largely anti-VC based on my own experience and observations from the past few years. I think VCs are a good route to build tech fast, if they do not end up controlling it, the IP, or have direct power. They often blow up though, so the risk in some cases is more theoretical and principled. I might compare that with the AI wars we have now where everything is closed source in a few AI labs. The tech rapidly evolves, but it stays owned by the few.

Now. I know every VC crypto/web3/etc company does develop in the open thanks to the foundational ethos, but in exchange, they often find other means of control or keeping power or ROI’ing that’s not in the interests of the community.

This is not a hard rule that VCs are evil… But it is a guideline based on what I see as the crypto/web3 VC playbook, so there are going to be exceptions and good-faith VC-funded projects. But there are very few of them IMHO.

Due to this… and the fact I come from a Linux FOSS background in my views, I just rather avoid all that shit show altogether and stick to the community to ensure what we build is by and for the community.

So, I am doing things with the community giving me a chance to make their investment, if you want to call it that, give them something that can solve a LOT of issues with Sia and the wider ecosystem.

And yes, I am aware of a lot of Sia’s history, drama, etc. I was not involved in the early days but after a year of being initiated into the cult… I mean community (just kidding guys :P), I have found all of Sia’s skeletons and why Sia has had a rough decade.

That said, unlike what many seem to have made into a meme, sia being allergic to marketing, I am realistic and I straddle both the worlds of Web2 and Web3.

Sia to an extent, I see as always a background player, and marketed to bigger players, and power users, but not retail.

So as for the how? One large route is FOSS/grassroots marketing. By creating public good services that are also hosted on sia, or store data on sia… The project solves a number of wider issues that start with education. In the process, Lume Web as a brand creates awareness for our mission and starts the needed network effect.

This is why I have invested in many web3/blockchain and ICANN assets. Creating open, community-driven, and community-built resources that serve as onboarding for new users and those skeptical or frankly calling us all scammers, as well as advancing the education of those in the community by crowdsourcing those who can help, but that may not be devs.

I have already spent a good % of the foundation grant on a new project brand/logo, website, and 2 other marketing websites, one for users, and one for those who want to support/host the infra of the new web.

I have a lot more planned as well. I have long-term ideas too, but they can only happen once Lume is actually fully working.

As for the business model of Lume as a community-focused business, I will be starting at some point (no ETA) by simply offering web accounts (private data encrypted by your identity key, which I can never decrypt) on the portal for a small yearly fee, and scale the storage plans offered as the project grows. Eventually, it will be webapps (aka dapp) too.

So simply put, web3 hosting services.

This is one prong of sustaining the project long-term outside having grant funding.

The other is that I will be creating a web3 domain registry with assets that I have acquired on Handshake. This, of course, requires the tech to be stable for the domains to be useful, just like in the current web.

I may get other revenue streams from services, but that’s more of an unknown, unknown right now.

I have also disclosed all my plans on the longer-term business aspect to the foundation and gotten guidelines of what they are ok with. So I have been fully transparent with the community regarding my plans and actions.

I hope this answers your criticism, which based on Sia’s history, is warranted to ensure we don’t fall into the same traps with failed experiments and some community members who have left or are exiled due to their own actions.

Kudos!

1 Like