2023 Report
Timeline & Reflection
This year has been chaotic for Lume as the entire Sia community was affected by the collapse of Skynet. For us, it meant a significant change, of course, where instead of using Skynet, we had to figure out a way to replace it. For those unfamiliar with it, Skynet technology allows the upload and download of data through interoperable public portals without users having to run their nodes.
Lume, a project initially intending to leverage Skynet technology, had to decide how to continue its core mission of an open, user-owned web. Since Sia keeps constantly evolving, without further maintenance of Skynet, it was clear that we needed to find another way, as it no longer made sense to keep building on top of abandoned technology. Instead, similarly to some other projects, the way forward was to learn from Skynet’s mistakes and do it better. These efforts are visible not only on Lume, inspired by Skynet’s Kernel, but also on projects like S5 by @redsolver, a flexible peer-to-peer file-sharing network that Lume decided to support instead of re-inventing the wheel.
We re-purposed and expanded some of the technology in an attempt to solve the DNS problem of the decentralized web and become able to access domains across multiple distributed networks. This development effort revealed that support of each network is not easy as each network has its issues and research involved, resulting in more delays than we expected. And finally, we also needed more time to find reliable designers and developers to work on the consumer side of the project, which is mission-critical for product exposure and adoption.
Despite setbacks, we made significant progress on the consumer side and in the core backend technology.
- We successfully added support for the Handshake, Ethereum, and IPFS networks into Lume’s routing technology. Each implementation presented learning curves, roadblocks, and setbacks that must be navigated and solved. Ultimately, we achieved the goal of creating a working Lume prototype.
- We added support for bao & blake3 protocols, which was quite a complex challenge that we eventually successfully solved at the cost of skipping it for small files as it wasn’t required.
- We managed to get rid of needing Skynet’s portal by forking Skynet’s Kernel instead of trying to use it as-is, as the data formats between Skynet and S5 are incompatible. This engineering decision saved a lot of effort due to the instability of skyd.
- We figured out how to query ENS domains (on ETH) in a fully trustless way.
For the next part, let’s explain two significant components that Lume consists of and what makes it sometimes confusing to understand. Whenever used alone, it usually represents either the brand or the technology for decentralized web access. But there is one more component we are working on - Lume Portal.
Lume Portal is our interpretation of what a Layer 2 (L2) should look like in the spirit of Skynet, providing essential functions for other systems connecting to it. That also represents the current state, the portal being only a backend server with no user interface but allowing us to deliver the demo apps to users already. The well-designed Renterd APIs were surprisingly easy to implement, which we see as a great accomplishment by the Sia Foundation. We can already feel the impact of Sia’s ongoing re-design.
The portal software aims to fill a gap left in Sia’s ecosystem after the disappearance of Skynet Portals, and it seeks to onboard as many users and their data as possible. This will benefit both the Sia network and Lume, which, by pursuing more users, helps the Sia network’s growth and independence.
We also released a user-focused product, a Firefox browser extension, to provide access to the Lume technology and, by extension, Sia. At first, Firefox would provide everything needed for a fully decentralized experience. However, it didn’t work out as planned, not because it wouldn’t work, but because the target audience couldn’t care less about Firefox. Still, we consider this a success, as the goal of the extension wasn’t to drive immediate and broad adoption but to pilot and demonstrate the technology. This allowed us to receive crucial feedback, helping us to identify the fundamental problems and find ways to solve them.
We learned a lot throughout the year, including the realization that we must rely on something more transparent and censorship-resistant than the existing web browsers. It’s a constant fight, and we can’t tell if developers of those one day won’t block the APIs we use for our apps and extensions. This assessment of the macro situation led us to introduce a new long-term plan to commit to developing a Lume Browser. In the short term, we will still ship Lume Technology through other accessible means already available on the web.
Our journey so far has led us to create some demo apps. As stated before, the Firefox extension turned into a flop, so instead, we focused on the creation of a simple UX for a web login into Lume and created two demos, a web3 browser app and an NFT gallery app, powered by Sia’s renterd, which stores and serves the browser code required to communicate with all networks, including Lume’s P2P net.The web3 browser app also supports serving Sia-powered websites via S5. Renterd, accessed through Lume’s Portal, will also enable long-term user storage for applications accessed through Lume Technologies in the future.
Our intention has shifted towards bringing as much data to Sia as possible through the Lume Portal, supporting most (if not all) of S5’s network abilities and technologies like Bluesky and IPFS. This approach will ensure the broadest adoption and interoperability.
We believe the Lume Portal (as Layer 2) will have its own means of syncing data via hypercore such that it imports the data and owns the contracts like the original Skynet. Since it’s using renterd, it means that ALL data is stored on the Sia network unless the user defines different or multiple options through alternatives we support as well, like S5. Using hypercore in the portal effectively means using the P2P tech we spent significant time on to help power the L2. At the same time, we’re also developing a community platform, “web3.news”, which will serve as the hub for the whole industry with easy ways for users to host community sites on the Sia network. Doing this could be through subdomains or their domain, based on the communities’ interest in deciding how this evolves. So, web3.news covers our intention for a community effort this year. While we planned to do more, we realized we needed to build a community before anything else, and we will be focusing on that first.
Looking at the larger picture, Lume, both as a brand and product, continues to evolve in alignment with its original vision and principles of a decentralized internet. We are satisfied with the progress made so far, and the results promised to match, if not exceed, the expectations set earlier in the original grant proposal for the year 2023. It’s also clear that many proposed details have changed, but that’s unavoidable, given the dynamics and challenges we faced on the ecosystem level.
Current State
The project currently consists of several functional components, all working prototypes. We have set up the core technology to offer a P2P DNS system for web3 domains, showcased through a Firefox extension and a web app, and continue focusing on enhancing the web app experience and the underlying technology. Everything we learn in the process also helps us shape the plans for a browser-based solution that we’re committed to developing in the future.
We intend to improve or entirely redo much of the code as we discover better technologies or if the project’s short/medium-term goals change. We know that the entire ecosystem is evolving on many fronts, and discoveries are made constantly, making continuous refactoring unavoidable, especially for projects like Lume, integrating several other technologies in active development. However, everything functions as intended, and plans are in place to iterate on the next steps.
Finally, we would like to finish this part of self-reflection by examining how we use Sia today and what lies ahead. As mentioned above, this year’s largest effort was on routing technologies, which might be less attractive for users looking just for storage but is crucial for further development and Sia-related features. Currently, the Sia network is used for downloading the Browser App’s code and creating an anonymous account used in the process. It can also serve sites hosted on the Sia network via S5. The browser app demo showcases our progress, and you can find both the link and source code in the Deliverables section below.
Demo Videos
The following is a quick reference to all demo videos, which can be found in the deliverables as well:
- Firefox Extension:
- Web3 Browser App:
- NFT Gallery:
Relay
The relay is a Node.js P2P application that functions as a P2P node. It features a plugin system and some configuration management. The latest version is available at https://git.lumeweb.com/LumeWeb/relay/src/branch/develop. Marketing for the relay ceased due to a strategic pivot.
| Plugin Name | Description | Link |
| ETH | Provides consensus data to the ETH client | https://git.lumeweb.com/LumeWeb/relay-plugin-eth/src/branch/develop |
| S5 | Acts as a partial S5 node for registry services | https://git.lumeweb.com/LumeWeb/relay-plugin-s5/src/branch/develop |
| Handshake | Acts as a proxy for serving HNS peer connections | https://git.lumeweb.com/LumeWeb/relay-plugin-handshake/src/branch/develop |
| IPFS | Acts as a proxy for serving IPFS peer connections | https://git.lumeweb.com/LumeWeb/relay-plugin-ipfs/src/branch/develop |
| IRC Discovery | Acts as a means of allowing peers to discover the relay by pubkey | https://git.lumeweb.com/LumeWeb/relay-plugin-discovery-irc/src/branch/develop |
| Lavanet | Enables the relay to sponsor badge services, which are effectively a sub-billing account so the end user can query paid RPC while still being trustless | https://git.lumeweb.com/LumeWeb/relay-plugin-lavanet/src/branch/develop |
Every plugin has a matching client-side codebase.
Portal
Next, We would like to visit the state of the portal itself, which currently has a live web app and demo for consumers to experiment with the basics of the Lume tech.
https://git.lumeweb.com/LumeWeb/portal/src/branch/develop
The portal consists of a Golang daemon and API server. It features:
| Feature | Description |
| TUS Support | We might have a broken feature, as we last tested it a while ago. |
| Basic File Uploading Support | Primarily used for kernel module code. |
| Accounts with Email and Pubkey Login | Allows users to log in using either their email address or public key. |
| Basic File Tracking | Used for keeping track of files. |
Kernel & Browser
The Kernel (libkernel and libmodule) was forked and combined into a single library. The following are the components:
| Component | Description | Link |
| libkernel with libmodule | This component represents the merging of libkernel and libmodule into a single library. | Link |
| Kernel | Refers to the core Kernel itself. | Link |
| A re-implementation of skt.us | This is a hosted part of the Kernel in web2, trading off trust for accessibility. | Link |
| Hosted kernel worker | Enables Kernel modules to run in their own hosted iframes, providing them with dedicated CPU threads and overcoming the original Skynet Kernel module limitations. | Link |
You can see that Lume is on course with the mission to bring decentralized DNS and distributed file hosting to the online consumer. Over time, We’ve developed, implemented, and maintained a growing library of open-source protocols that will serve the Lume mission while also bringing attention to fundamental technologies provided by Sia and building a community of interested developers and consumers to the Lume product, which I will talk more about below.
Community
This year’s primary focus is on web3.news, which we plan to have functional as an MVP by the end of the year despite the project holding many assets for future community efforts. We will delay the implementation of additional features until we reconstruct the basic primitives we are currently using, such as S5. The community will guide the direction in which web3.news takes off.
Source code: https://git.lumeweb.com/LumeWeb/web3.news.git.
Design files
As this project is entirely free and open-source, all design files based on the community funding are available here: https://git.lumeweb.com/LumeWeb/design.git.
Deliverables
| Demo | Description | Link(s) | Video |
| Firefox Browser Extension | An early prototype extension for Firefox. It was a successful Proof of Concept from a technical and R&D perspective but needed more adoption. The extension doesn’t include much of the R&D from this year, like the S5 registry. A video is available showcasing its experience. | Git Repository | Link |
| Web2 -> Web3 Browser App | The first fully decentralized Web3 Browser/Gateway using service workers, similar to archive.org. It supports IPFS, Sia websites (via S5), and HNS and ENS domains. It uses the Lume Network, Portal, Kernel, and S5 network to load content trustlessly. PR release gained exposure. | Web App, Git Repository | Link |
| NFT Gallery | Part of a series of toy demos. It uses ETH and IPFS to find ERC721 NFTs for a given ENS address. The demo represents an early stage that will evolve in the future. | Toybox, Git Repository | Link |
These demos may have minor UX bugs and not support edge cases or every site/ETH account. Consider them polished prototypes with plenty of room to grow.