Standard Grant: Lume Web 2025

Grants Approved
5

Out of curiosity: is the Upload button not working on https://account.testing.pinner.xyz, or was that the intent?

Regarding your new proposal, I am requesting you to create a long-term roadmap of your project, similar to what the Foundation has but looking into the years ahead. The reason I am asking this is that you keep submitting a new proposal for the coming year at the end of each year, and this is fine but it does leave a feeling that this is going to last forever. Please spare us all the frustration.

6

I just tested this in Brave (Version 1.70.119 Chromium: 129.0.6668.70) and was able to upload video files fine, even in incognito mode. I’d need to reproduce the reported issues to properly fix them if there’s actually a problem.

Regarding the roadmap, I’ve shifted my approach based on community reactions from last year and what I’ve learned from new data and experience this year. Instead of focusing on moonshot ideas about where I want things to go, I’m concentrating more on immediate and medium-term goals. This shift actually came from the community’s request to focus on the portal and bring more immediate value to Sia.

For the period after 2025, I do have other integration ideas from ecosystems I’ve researched. Some of these aren’t ready yet, and others would need significant time to port code to golang.

Ideas I’ve researched for potential integrations:

  • Iroh support (requires go port)
  • TBD Web5 (likely requires go port)
  • Syncthing support (act as a cloud peer, in golang already)
  • ActivityPub support
  • RemoteStorage.io
  • SolidProject.org
  • Hyper/Dat (requires go port)

And for application-level problems:

  • Docs
  • Email
  • Password manager
  • GitHub Pages/Vercel/Cloudflare Pages alternatives
  • Taking Siashare and making a golang version using S5/IPFS & other tech, either as an L3 or portal plugin (undecided) - basically run a simple end-user filesharing service like Wormhole
  • WordPress-related integrations (web 2.5)

But I’ve learned that nothing goes exactly as planned, and what seems like a good idea now might change later. Some of my thinking from 2023 evolved by mid-this year, and I ended up adapting what I built to make it more flexible, ensuring I could bring more features to Sia (which ultimately brings Lume more long-term funding through storage sales).

At this point, any future ideas I might share are just that - ideas. Things are very likely to change at the macro level which will impact the project. While Lume’s big picture goals will eventually return to the high-level web3 work I started with (which will honestly require a fair amount of experimentation), right now I’m following what the community requested: bringing demand to Sia, which obviously supports Lume as well. My focus going forward will be on bringing data to Sia as the community requested, while funding Lume’s bigger picture goals through paid services. The yearly grant proposals reflect this iterative approach - each focused on concrete deliverables that build toward sustainability rather than locking into long-term predictions that might not serve the community’s evolving needs.

I’m also planning to launch feedback.lumeweb.com to give the community a more direct role in guiding the project’s direction.

7

Okay, let me ask you a blunt question. For how many years are you planning to ask funding for development on Lume, and how much time shall pass before you deliver a working product?

8

Lume is already a working product, though we’re continuing to add polish and evolving features with my team. Our immediate plans include:

  • Soft launch EOY for Q1 2025 to work out issues
  • Implementing anti-abuse measures to address liability concerns
  • S5 integration either this year or potentially Q1, depending on redsolver’s development timeline

Regarding funding expectations:

  • 2025 will see us generating sales while continuing development
  • I may request 2026 funding if organic growth in 2025 isn’t sufficient for sustainability
  • There will be two portals:
    • The community portal will continue being grant-funded, and I may seek continued funding for it as a community service even after Lume itself no longer needs project-wide grants
    • The paid commercial portal through pinner.xyz

I understand the Foundation’s need to evaluate KPIs and see return on investment through storage demand. I don’t expect continued funding without demonstrating value - if we’re not generating ANY sales by the end of 2025, that’s concerning, and if we’re not successful within three years (2025-2027), that would indicate a fundamental issue with our execution or market fit. At that point, I would need to return to consulting work to pay bills while continuing the project part-time if the foundation decided to stop funding.

As Danger recently joked, I have ideas reaching toward 2030, and while these align with our original grant’s vision of broader ecosystem integration, I recognize that continued Foundation funding should be tied to delivering tangible value to Sia.

We’re committed to becoming self-sustaining through storage sales and paid services rather than relying indefinitely on grant funding.

9

Also, given you had issues regarding the upload button. Im posting this screen capture for the record to show that upload is working.

10

Thanks for your proposal to The Sia Foundation Grants Program.

After review, the committee has decided to approve your proposal. They’re excited to see the continuation of Lume and the integrations you’re building this year. Congratulations!

We’ll reach out to your provided email address for onboarding. This shouldn’t take long unless your info has changed from last time, but you may still need to adjust your timelines.

11

Progress Report for January

What progress was made on your grant this month?

  • Had to handle weird issues with TUS uploading on the pinner.xyz portal and double check uploading was working properly in general
  • renterd took a lot longer to sync then expected
  • Rest of the month has been spent on rebuilding the billing support.
    • The existing implementation after reviewing things turned out to be badly designed without a full understanding of killbills state machine system.
    • The justpay-written plugin needed a lot of changes to properly use the killbill apis.
    • This included significant rewrites to the hyperswitch java plugin, and dealing with upstream library issues. Most of these updates are still in WIP branches, but the hard parts of the billing backend systems should be done.
    • The billing plugin also has had significant changes and will have more in the future.
  • A design for the abuse report webapp has been created and ready to be ported for the Q1 anti-abuse priority.

Summarize any problems that you ran into this month and how you’ll be solving them.

While I currently consider the billing in a 90-95% ready state, I am intending to do another refactor of the webapp code base and create a frontend framework/foundation with module federation. Due to this, I am punting the billing support until after this is done with the goal of it being finished by the end of Q1.

This will also be used as a basis for the admin webapp/plugin as well.

I don’t wish to put effort into the web part of the billing further and need to scrap it. This also means further UX work will be punted until there is a better webapp foundation.

As a secondary concern, hyperswitch’s cloud pricing has changed, and went to target enterprise, which means I will have to self host a card vault and go through required compliance measures to bring billing online. I was considering it as a temporarily way to deploy faster, but thats no longer in the cards, it seems.

Links to repos worked on this month:

What will you be working on next?

  • I will be bringing the free portal online and doing any required devops work along the way.
    • I am also intend to be doing some server migration efforts and restructure things. Looking to adopt proxmox and put infra inside virtualization, which would allow use of another layer of redundancy/safety as well as using the proxmox sia plugin with an outside renterd instance.
  • Will work on the webapp foundation priority
  • Will work on S5 support assuming no blocking issues from redsolver. I have also made the decision that S5 will be an API-first service upfront with no UI support in the webapp given most planned app use cases will use the API. a UI will be added based on feedback. This is to reduce the eng time needed to bring S5 demand online while balancing other Q1 priorities.
  • Will do needed research on the anti-abuse and create a game plan for it. Will start those efforts if I have the time.
    • Know that anti-abuse will be a subsystem of the admin panel… which will require the new web foundation, so all of this is being prioritized on importance and is very inner-connected.
  • Will likely be doing some portal core design updates and possible plugin API changes, As stated in the past, these will continue as deemed needed for the evolution of the system, and is a large reason why no v1 in semver exists yet.
12

Progress Report for February

What progress was made on your grant this month?

  • Created the bulk of the portal webapp framework.
  • Start migration of webapp components and business logic to a core plugin
  • Started on refactor/restructuring of golang S5 codebase
  • Got the abuse report webapp UI ported to react
  • Started on the UI & code of the abuse admin
  • Did research on the requirements for the abuse service
  • Worked on core portal design changes needed to improve current data models and ensure an abuse service + content scanning can be supported.
  • Got pinner.xyz live. Will be able to get the free portal online this week.

Summarize any problems that you ran into this month and how you’ll be solving them.

The devops for the portal took much more effort then expected, as I had to effectively recreate all infra, and ran into new operational problems to be solved, due to both virtualization being used, and other black swans. Some of these are k8 related issues and others are more akash related issues.

Nothing right now is a further blocker for the portal infra, and the rest will be brought online soon.

Links to repos worked on this month:

What will you be working on next?

  • Getting the free portal infra online 1st week of march.
  • Getting the abuse plugin completed, which will also involve the admin plugin, which needs work too.
  • Working on S5 support further if there are no upstream blocking issues from redsolver.
  • Completing the billing support, ideally launching it if time permits.

My hope is to complete all needed priorities this quarter, but obviously the abuse comes 1st per my grant priorities. So there is a risk that S5 and billing will be worked on in Q2.

13

Progress Report for March

What progress was made on your grant this month?

  • Got lumeportal.com operational (https://github.com/LumeWeb/lumeportal.com-devops)
  • Got backend of abuse plugin completed
  • MVP mocks of both abuse report form and admin ui are completed
  • Further webapp framework architectural work
  • Further portal core design changes to meet the needs of the abuse plugin
  • A Go client library for NCMEC (CSAM Report API)
  • Worked on libraries that standardized rest API HTTP processing and filtering
  • Created unit testing packages for the core portal and mocks

Summarize any problems that you ran into this month and how you’ll be solving them.

  • Roughly 1 week of time was burnt on trying to create a unit test framework library to streamline testing of database queries in plugin code. This was stopped after a week as the requirements turned out to get more complex. As a result, no time has been left to get the webapp parts of the plugin completed.
  • I realized the NCMEC service only allows for reporting content, not for getting any hashes or being told if content is on a blocklist (content scanning). I will have to evaluate on how to integrate reporting CSAM content in the abuse reporting processes, and that may be de-prioritized if required while working on other tasks. ClamAV, however, has been integrated as a content scanner.

Links to repos worked on this month:

What will you be working on next?

  • Completing the webapp ui’s and testing
  • Refactoring anything needed in the abuse plugin
  • Evaluate integrating the NCMEC support
  • Getting the IPFS plugin synced with core portal changes as well as abuse support
  • Continue working on getting the dashboard ported to the new webapp framework as a frontend plugin.
  • Work on S5 and/or Bluesky planning if there is time available

The portals may be online but I expect them to undergo large upgrades, and so I will not be promoting the community portal until things are ready since many systems are in a transition regarding architectural or framework improvements. Specifically, the dashboard and the IPFS plugin will need testing after they get updated to the new core changes.

Q1 Milestone Report

A majority of the milestones has been completed.

  • Report Intake
    • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/service/email.go#L146
    • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/pkg/email/pipeline.go
    • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/pkg/email/imap_client.go
  • Documented API endpoint with authentication
    • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/api/abuse_api.go#L79
    • Swagger document & middleware integration is still TODO
    • Support includes ARF reports https://www.rfc-editor.org/rfc/rfc6692.html
    • Support includes parsing email threads as well
  • Report Processing
    • Automated classification and routing
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/pkg/email/classifier.go
    • Unique case ID generation
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/service/email.go#L491
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/api/dto/abuse_report.go#L28
  • Case Management
    • End-to-end workflow tracking
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/api/admin_extension_case.go
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/db/models/case.go
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/service/case.go
    • Basic analytics/reporting capability
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/api/admin_extension_case_analytics.go
    • Content scanning
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/pkg/scanner/clam_scanner.go

I am estimating needing a week to complete the milestone, which is mainly the webapp aspect. I will keep the committee updated on progress and if I end up under-estimating that ETA. I have largely been able to leverage AI to accelerate development on both design and development, So I do not believe that I will require an excessive more amount of time for this milestone.

14

Hi pcfreak30, I am checking out the status of Lume, looks like progress was made on the Q1 anti-abuse milestone, let me know the best way to check out these features on pinner.xyz once they are complete. Also, when I tried out pinner.xyz and ran into a few different issues. The app gives me many 400, 401, and 403 errors as I navigate around, try to create API keys, or upload files. Reviewing your upcoming plan, I would suggest figuring out how to keep the portal and existing functionality stable throughout all the development and refactoring cycles, and how to prioritize doing that community promotion sooner than later, I think you will a gain an invaluable source of continuous feedback and direction from a small group of active users.

15

Thanks for the detailed feedback on pinner.xyz and checking in on the anti-abuse work.

Regarding the errors (400s, 401s, 403s) you hit on pinner.xyz – you’re right, that definitely needs fixing. I’ll dig into those issues once I wrap up the current anti-abuse priority.

A lot of the instability you’re seeing stems from a major refactoring effort that’s actually mostly complete now. The core foundation (web framework updates, module federation, new abstractions, database migrations) is largely in place. The current phase involves updating the different application pieces – like IPFS support, Admin tools, and the Abuse system – to work as plugins within this new structure. This is where the remaining work and temporary breakage lies. Until these plugins are synced up, the APIs might be inconsistent.

Specifically for IPFS, it needs some dedicated effort (a few days probably) to fully integrate with the new internals because I changed how some things work. I did most of the heavy lifting for that back in February in a branch, but it needs revisiting and testing.

Be aware that you will be able to test things on lumeportal.com first before pinner.xyz. My immediate infra focus for the free, community instance will be lumeportal.com. That’s where I’ll concentrate efforts to provide a stable running service for wider use and feedback once these updates settle, before doing whats required on pinner.xyz.

So, the plan for the anti-abuse features is to release them as an updated Admin plugin once they’re aligned with the new core and the priority tasks are done. The abuse webapp side will have to get updated into a new web plugin when im done with it this month as well.

One thing to note is the dashboard frontend won’t be updated right away. Porting the React code and its IPFS/Uppy integrations into a plugin is a separate step that will happen after the backend stabilization. So, the web UI might feel a bit ‘legacy’ for a while even after the backend APIs are updated.

I agree with you on stability and community feedback. This whole refactoring effort, while causing short-term pain, is precisely to build that solid foundation needed for long-term stability on lumeportal.com. Once things are running smoothly there, it will be the right time to really push for community promotion and gather that continuous feedback you mentioned. The goal is to get past these major foundational changes soon so we don’t keep hitting these kinds of breakages and can focus on building out features on a stable base, particularly for the community on lumeportal.com.

Today I will be actively working on the admin webapp side of things and can give the committee an update at the end of the week on where I am at for the milestone.

16

Hello,

Thank you for your progress report!

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

17

A bit late, but enough progress has been made that ill summarize what has been done and what is left in terms of my milestone:

  • Have spent a decent more time on the webapp framework as several gaps were found that needed to be implemented.
  • Have been working on some needed UI components in a high level UI library thats being made that will also be foundational for future UI efforts to have a sort of design system.
  • Got most of the abuse dashboard ported into plugin-format.
  • Got a MVP mock abuse report webapp and a small user side dashboard for viewing a report by an access key created.

Remaining:

  • Porting the mock into plugin format
  • Testing both UI plugins
  • Testing APIs work correctly with the webapp
  • Do system testing with inbound emails
  • Creating swagger
  • Potentially other miscellaneous tasks

My timeline I have found to be off mainly b/c I have had remaining foundational work required.

I will provide another update on progress for this when it makes sense to or my next monthly report, whichever comes 1st.

18

Progress Report for April

What progress was made on your grant this month?

  • Continued chipping away at the webapp framework and the Go aspects for plugin loading.
  • Made headway on the UI component library – building those essential legos.
  • Spent some time wrangling miscellaneous JS tooling.
  • Knocked out about 50% of the abuse webapp frontend.
  • Did a bit of recon on the Bluesky ecosystem and the current state of PDSs.

Summarize any problems that you ran into this month and how you’ll be solving them.

This month’s main blockers were some unforeseen engineering timesinks, mostly tangled up in React and getting those UI legos just right. Here’s the breakdown:

  • React Wrangling:

    • Hit several low-level architecture snags while building the report webapp.
    • Tooling Headaches: Module federation tooling sent me on a few goose chases. Had to sink a fair bit of R&D time over the last month or two to get it cooperating. The hope is to eventually migrate to Rolldown, as they’re building native MF integration which should simplify things.
    • Routing Shenanigans: Ran into issues with react-router, navigation, and how they mesh (or don’t) with module federation. Specifically, index routes aren’t playing nice, forcing manual layout management in some spots :upside_down_face:. I’ve patched things up with workarounds for now, but this needs a proper fix down the line to use the router correctly.
    • Context Fun: Also navigated a variety of fun issues related to React contexts along the way.

  • UI Component Deep Dive:

    • As mentioned before, I’m building out reusable UI legos. The long-term goal is clear: need the ability to spin up tables, dialogs, and forms rapidly without repeating myself (DRY!). Future plugins, like file managers, will also build on these primitives (especially tables).
    • To achieve this, I’ve been investing time in creating solid components and React-based APIs (like useDialog) to make future UI development faster and more consistent.
    • Where the Time Went: Honestly, most of the energy this month poured into this lower-level UI foundation rather than getting stuck on specific application logic. Given I foresee potentially up to 10 different portal plugins, each potentially needing its own web UI, this foundational work is crucial. My experience building monolithic apps tells me this upfront effort saves massive amounts of time avoiding repetitive basic UI builds later.
    • Looking Ahead: Assuming no more surprise React ambushes (famous last words, especially since I’m kind of inventing the plugin system integration as I go), I don’t think the abuse app has much longer. After that, porting the other UIs (Dashboard, IPFS) to plugins should be quicker.
    • The Vision: Over time, this library will likely mature into a more formal design system for Lume, similar in spirit to what the Foundation is building for the Sia apps.

To give the committee a concrete look at the components that have been taking up this foundational effort, here are direct links to the code as of today:

On a related structural note: I’ve also consolidated the apps down. Now there’s primarily just the portal-app-shell (https://github.com/LumeWeb/web/tree/2f5e8b1ad5220dc735582317d3bf454750eab467/apps/portal-app-shell). This shell can be configured at build time (basically just needs a page title and app name/ID) to become any of the needed webapps, as the actual functionality gets loaded via web plugins.

I know this section got lengthy, but I wanted to properly communicate the technical details behind the project delays, the reasoning for the decisions (especially the UI component investment), and what exactly has been consuming time.

Links to repos worked on this month:

What will you be working on next?

  • Finish the remaining abuse milestones (frontend, admin, and testing email processing & routing).
  • Get the existing dashboard and IPFS webapp code migrated into plugins.
  • Circle back to IPFS functionality and fix anything broken during the current refactoring chaos.
  • Dive deeper into Bluesky research and testing:
    • Current plan is leaning towards using rsky (https://github.com/blacksky-algorithms/rsky), a Rust-based PDS. Why? Honestly, the thought of managing distributed SQLite, even with Litestream, gives me a bad gut feeling about DevOps pain… and my plate’s pretty full already :P.
    • Need to grok the atproto data structures to map out the plugin plan. This likely means running both rsky and maybe the reference PDS side-by-side to experiment and learn.
    • rsky uses Postgres and S3-compatible blob storage, which sounds like a decent foundation to integrate with the portal.
    • The approach will likely be an atproto portal plugin that communicates with a separate PDS server (like rsky) via an API. Embedding isn’t really feasible since there’s no mature Go PDS :cry:.
    • Heads up: rsky is actively developed, so some Bluesky features might be in flux or missing (e.g., I saw OAuth support is still being worked on). The good news is the Blacksky project seems to have solid community momentum and funding (https://opencollective.com/blacksky). I believe the critical PDS functions are mostly there, but I’ll re-evaluate this path if major roadblocks appear.
  • Take a look at the billing and S5 tasks list and figure out what’s feasible to tackle soon.
  • Gather user feedback – if I can get updated systems deployed in May. This might slip depending on progress.

As requested previously, here’s a very short, unedited video demonstrating some tangible progress. It shows the abuse report submission flow working. Next steps for this feature are implementing login to view submitted reports, plus adding commenting and file upload capabilities (likely using uppy).

Apologies again if this report feels long, but transparency about the work, the roadblocks, the reasoning, and the path forward felt important. Hoping the foundational work being laid now pays off in faster progress soon!

19

@pcfreak30 Just a quick note that we’re sending to all current grantees.

When submitting progress reports, especially ones that contain work towards your milestones, development work must be easy to access and review. Please directly link to work done, and if that report says “support added for feature x”, please link either to a branch or to a commit range showing the diff for that feature.

Moving forward, we will be assuming that progress that is not linked to directly was not made.

20

Progress Report for May

What progress was made on your grant this month?

  • Worked on query/sort/filter library
  • Given all of the planned plugins for the portal, I ended up going on a side quest and rewriting the entire http layer which uses the go echo router at the base. Took longer then I wanted, but the goal was:
    • To not use a unmaintained router (gorilla)
    • Create a framework for using the routing API to describe itself so swagger.yaml can be dynamically generated and served.
    • In the process of this several repos were worked on:
      • httputil library: Still mostly client side http utilities
      • gswagger fork: Middle layer that does a lot of the openapi work
      • portal-router: a new library
      • Created a new portal-middleware library that extracts out from portal and is a new near-complete rewrite of all the required HTTP needs.
  • Did further webapp framework engineering:
    • Found simpler solutions to past react issues
    • Created unit testing for many core libraries and some of the UI components
    • Started an effort on storybook
  • Took another side quest and ended creating a small testing framework for portal plugins. I have been working in unit test code with the abuse plugin and dashboard and found it was getting to be too much duplicated effort. Goal is to streamline that aspect of development as much as possible.
  • Worked on the abuse plugin. Mainly upgrading it to use new libraries, new routing, verify tests work. Still needs to use the new testing library.
  • Worked on the dashboard plugin. Mainly upgrading it to use new libraries, new routing. Still has a bit of work left.
  • Worked on the abuse plugin. Mainly upgrading it to use new libraries, new routing.

Summarize any problems that you ran into this month and how you’ll be solving

No issues this month other then I hit a lot more foundation work being required, unexpectedly, in the process of trying to build the admin ui & abuse admin.

Links to repos worked on this month:

What will you be working on next?

  • Finish the abuse (I honestly am hoping this takes under a week, the react parts on this are not complicated as I see it)
  • Focus on atproto and figure out its requirements
  • Get the dashboard webapp onto the new framework (my account, etc)
  • Get IPFS back in sync with core, and get a web plugin for it ported over from the legacy webapp.
  • Get the community server operational for user testing.
  • Work on S5

As of right now I am definitely viewing my milestones as being pushed back overall and my expectations are that I will primarily be doing a lot of plugins/integrations in a short period, so I foresee the next 3-4 months as being very compressed once I am able to get everything back online again. That is why I am trying to make that effort as minimal as possible so if im creating a ton of plugins they aren’t hard to make… be it protocol, informational or misc.

Lastly for a visual on milestone progress here is a screenshot of the abuse admin dash, no real data yet.

Selection_2399
Selection_23992257×1000 44 KB

21

Quick Update: Abuse System MVP & Framework Progress

This is not my monthly report for June

Here’s a short video demoing a very rough MVP for the admin side of the abuse system.

The good news is that I’ve gotten unit tests passing for both the DB and API code, all running on the testing framework I built. The core web framework also has its unit tests from May.

On the tech side, two key things:

  1. DB Testing: I learned a hard lesson from a previous side quest: Don’t try to mock the DB. The testing framework now uses temporary SQLite DBs, runs all migrations, and tests against a real structure. It just works.
  2. UI Rewrite: In the process of building this, I rewrote the layout for the admin/dashboard UI (the shared GeneralLayout component). This includes a new menu and profile menu UI.

All major webapp framework challenges are now largely solved. The tooling will likely evolve to a rolldown plugin for module federation, but the path forward is clear.

While there’s a lot more that could be done here, and the TS codebase could use some love, these are low-priority tasks against everything else required this year.

I will be focusing next on:

  • Doing more internal refactoring on subsystems like events, config, cron, db, and possibly more. I want to get to something I will be more happy with before I view it as as final design for a while (as many plugins will depend on it)
  • Creating an abstraction so I can more rapidly create plugins and handle the most common actions (upload, pin, dl). This may include adding to the helper libraries.
  • Get the IPFS plugin using this abstraction and get full unit testing on it, ideally also get it in compliance with the IPFS pinner spec test suite.
  • Get the portal dashboard ported over to the new framework.

I have also found that for the atproto integration, community tooling has been made in several languages and there is both a golang tool at indigo/cmd/goat at fa0e9d0066c10c010f28053a4bd0fa49337a6562 · bluesky-social/indigo · GitHub and a rust tool at GitHub - NorthskySocial/pds-migration: Backend Application for user PDS migration. So leveraging these will largely solve the required backend code for migrating user accounts over, and that then becomes more of a question of the a webapp plugin for a wizard, etc.

I may or may not get to any significant progress on atproto this month. I will be largely focusing on getting IPFS back online and usable again.

That said, I do expect to get my milestones back on track based on the effort done already and what I plan on doing to improve the speed of upcoming integrations.

If the committee has any questions or concerns about my milestones, please let me know.

22

Progress Report for June

What progress was made on your grant this month?

  • Created a MVP of the admin side of the abuse.
    • Please note that I am considering the abuse work done for now and will revisit it once priorities deem it important to do so.
  • Redesigned and rewrote the config manager subsystem
  • Redesigned and rewrote the event bus subsystem
  • Moved from dbmate to goose for DB migrations
  • Added full unit testing and integration testing to core portal services. Majority of tests are passing, with low priority issues to be handled at a later date.
  • Started on re-design of IPFS plugin, especially with the data model.

Summarize any problems that you ran into this month and how you’ll be solving them.

Please summarize your issues into a few sentences or bullet points:

N/A

List repos worked on this month with links to PRs and relevant commits:

What will you be working on next?

Please summarize your development goals into a few sentences or bullet points:

  • IPFS plugin
  • User dashboard webapp
  • Getting lumeportal.com ready for test users
  • Nostr integration

Grant Timeline & Q3 Focus

This section details significant shifts in our development timeline and priorities for Q3, driven by recent challenges and strategic considerations.

Current State & Adjustments:

  • Delays: Grant progress has been heavily impacted by the foundational rebuild of the webapp portal, which required creating a new architecture.
  • Abuse Plugin: To prevent further derailing the grant and keep core priorities on track, the abuse plugin MVP is being considered completed and will be evolved when required.
  • atproto (Bluesky) Punted: Due to significant project and engineering risks, coupled with high complexity and unknowns, atproto will be the last integration tackled in Q3.
  • Billing: While not part of my milestones, I am not expecting to be doing any sales this year as that requires a few weeks of work that I likely no longer have a buffer for.

Improving Development Velocity:

Despite the initial delays, we’re building momentum:

  • Stable Portal Framework: I now have a robust and stable Golang-side portal framework, including a powerful testing library that has already delivered massive gains in efficiency. This means portal-side development should take less time moving forward.
  • Helper Library: I’ll be creating a helper Go library during the IPFS plugin rework, which will significantly speed up the development of subsequent integrations.

General Development Observations:

Through this process, I’ve consistently observed three key patterns:

  • Webapp Development Challenges: Webapp development often takes longer due to immature and evolving tooling that lacks features like Hot Module Reload (HMR), slowing down iteration cycles.
  • Protocol Integration Core: The majority of integration effort lies in enabling the portal to communicate with a new protocol and completing any necessary database modeling. Once that’s done, the rest of the work is far more standardized.
  • DevOps Overhead: DevOps efforts requiring new infrastructure components can consume up to 10x the time of typical R&D. Improving this area requires its own dedicated research.

Revised Q3 Integration Plan:

My broad aim is to complete each integration within 1-2 weeks, though some may extend to 3. atproto is currently estimated to require the most time. The current priority order is:

  • IPFS (Target: Mid-July):
    • Will include the dashboard, file manager, and API keys.
    • Will be compliant with the pinner spec, allowing use with IPFS desktop via an API key.
  • Nostr (Estimate: 1-2 weeks):
    • Based on NIP-96 and NIP-98 (if thats deemed needed), this is the simplest integration as it’s fully HTTP-based.
  • LBRY (Estimate: 1-3 weeks):
    • Research indicates a need to extract and adapt code from the reflector.go project to create a new library for network communication. After that, development should be similar to IPFS/S5.
  • atproto (Bluesky) (Estimate: 3-5 weeks):
    • Highest Risk: This integration currently requires running a separate server, as a feasible Go port isn’t possible within a reasonable timeframe (even if the grant timeline were on track :cry:).
    • Operational Complexities: The TypeScript reference implementation uses SQLite databases per account, which poses significant operational challenges in a cluster environment.
    • rsky Option: As previously mentioned, rsky is a potential Rust-based alternative, but it’s still in active development.
    • Research Focus: This quarter will require substantial research to determine the optimal design and infrastructure decisions for this plugin.

Q3 Strategic Shift: Delivering Tangible Results

Responding to community feedback and looking ahead to 2025 grant obligations, I’m restructuring Q3 to focus on tangible progress that the Sia community can actively use. While the grant to date has centered on compliance and foundational efforts, Lume will be delivering concrete, usable results this quarter.

S5 Integration on Hold:

The one exception to delivering “most” integrations is S5, which is being put on hold. This was not part of the milestones. However, Due to significant, ongoing changes to the S5 specification, I cannot risk spending 1-2+ weeks (or more) porting s5.js into a Go version only to have that effort rendered useless by a “rug pull” from a rapidly evolving spec. While supporting Vup is a very large, long-term priority for Lume, it’s currently too risky to build on until the specification stabilizes and provides a clear, dependable foundation.

1 Like
23

hey @pcfreak30 I’m trying to learn more about your LumeWeb. On my adventures I have ran into a few things I’m hoping you can help address:

1) When I click “Download Extension” on the landing page of the website https://lumeweb.com

Screenshot 2025-07-05 084001
Screenshot 2025-07-05 0840011555×119 8.43 KB

It takes me to this unrelated website:
https://drivelatino.com/2020/12/nissan-armada-2021-ahora-con-mayor-potencia-seguridad-y-tecnologia.html

Screenshot 2025-07-05 084116
Screenshot 2025-07-05 0841161778×888 205 KB

I assumed this may have been a hosting provider issue, but it seems to have been like this for a few months now.

Could you share a valid link to download the extension?

2) I can’t seem to upload any files on pinner.xyz, even under my storage limit of 999b.
Do I need to perform any additional configuration on my side to start using it?

Screenshot 2025-07-05 085053

The link to connect on discord https://discord.lumeweb.com/ is also broken.

3) How do I increase my monthly storage limit of 999b or pay for any of the services until web3portal.com becomes available?

4) I enabled 2FA on my account, but when I go to sign in it says I need to enter my code, yet there’s nowhere to enter it. After closing the tab and reopening in a new window, I was eventually prompted for the OTP.
I think I’ve tracked the issue down to the login URL param /login?to=%2Fuploads being present. When it’s not present, I’m correctly redirected and prompted for OTP.

Screenshot 2025-07-05 085501
Screenshot 2025-07-05 0855011920×713 32.3 KB

5) How do I access the Lume web browser like in this video?
https://www.youtube.com/watch?v=WkRJ4o1MyYI

6) Are the core portal features mentioned in the 2025 proposal available now on pinner.xyz?
(RBAC, storage/renter management, configuration management, etc.)

7) I was an early adopter of Skynet and built apps in that ecosystem, but often ran into data consistency issues (ie writes disappearing and suddenly reappearing).
Which parts of Skynet have been forked or reused for LumeWeb specifically, and has anything been done to address data consistency? Have you experienced any data consistency issues with LumeWeb?

8) I’ve noticed a concerning pattern over the last several years:

Mid–2022:
“Lume had to be nearly completely rebuilt over the last six months”
– Oct 2022 Grant Proposal

Late 2023:
“I see an opportunity to rewrite many internals while keeping the same high-level API”
– 2024 Grant Discussion

Late 2023:
“The portal: Needs a large refactor/rewrite to support many protocols”
– 2024 Grant Discussion

Late 2023:
“The relay: It needs a rewrite… ideally use a better JS runtime”
– 2024 Grant Discussion

November 2024:
“But the current JavaScript P2P code needs to be rewritten in Golang“
– Precursor to 2025 Grant Proposal

February 2024:
“Completely redesigned our protocol and storage architecture”
– 2025 Grant Proposal

February 2025:
“I am intending to do another refactor of the webapp code base and create a frontend framework/foundation with module federation”
– 2025 Grant Proposal

March 2025:
“Needed to recreate all infra”
– 2025 Grant Proposal

This recurring pattern of architectural changes, redo’s, and rewrites each year raises a few questions for me:

  • What lessons were learned from the past reworks that necessitated another overhaul in 2025?

  • Is there a stable, long-term technical direction the project is working toward, or are the rewrites symptomatic of shifting priorities or unclear planning?

  • Could this pattern of reworking indicate foundational design issues, and how is this being addressed going forward?

9) You mentioned the frontend was built using Refine.dev, Tailwind CSS, and React.
However, from my experience on the site (as shared above), I encountered multiple UX bugs (2FA loop, broken upload functionality, and download redirects).
Is there a QA plan in place for frontend usability and accessibility, especially with plans to launch a paid service?

10) A core part of the proposal discusses the “Sia Layer 2” functionality and integration with renter management, billing, etc.
Could you clarify which parts of that are currently live and usable today vs. still in development?

11) You mentioned:

“The current JavaScript P2P code needs to be rewritten in Golang”

Has any of that rewrite started? If not, do you anticipate any blockers or shifts in approach? What is the timeline and resource allocation for that effort? Why must it be rewritten?

12) The $244,000 in cumulative funding to date is substantial and reflects a multi-year commitment from the Foundation.
Could you share a breakdown or retrospective on what the community has received in terms of:

  • Stable production grade tools
  • Actual usage or adoption stats
  • Open source repos with tagged releases

13) Are there any internal metrics or KPIs you’re using to measure success and progress?
Something like upload volume, daily/monthly active users, or network bandwidth consumed?

14) Despite clear feedback to narrow the focus of this project

“Lume’s first year of funding was largely you working with dozens of different technology tools and networks and very little direct interaction with Sia software and APIs, that’s totally fine, but now I would like to see any follow up Lume grant present:”

  • “A drastically narrower scope”
  • “A drastically clearer focus on Sia”
    — @parox, November 2023, Lume Web 2024 Grant Discussion

The 2024 plan still spanned a broad set of infrastructure layers and technologies, many of which are not Sia exclusive:

  • Portal rewrite
  • Relay rewrite
  • P2P redesign in Golang
  • Kernel refactor
  • web3.news
  • Cloud hosting
  • Billing system
  • Dashboard
  • Terraform + DevOps
  • Bun.js migration
  • Module federation

All in all, my main concern is there’s this repeated narratives like:

  • “All software is MVP that needs more iterations.”
  • “Much work had to be rewritten or rebuilt.”
  • “Portal is in early beta.”
  • “Billing plugin not deployed.”
  • “Admin tools incomplete.”
  • “S5 plugin needs rewrite.”

And after three years and nearly a quarter million dollars, most components are still in alpha/beta or pending rewrite. The rate of rewrites vs production delivery appears disproportionate to the funding spent. You are one of the most active community members in Discord and on these forums and I can’t help but feel like you should be spending less time with the community and more time on completing this project.

I realize this is a lot to digest but so is the body of work, rewrites, and discussion around LumeWeb over the past three years. My intention here isnt to attack, but seek clarity and accountability as someone who cares deeply about the success of the Sia ecosystem

This is a highly ambitious project and you have been highly communicative but at this stage I think its reasonable for the community to ask for a more stable path forward, clearer deliverables, and better alignment between funding and outcomes. I am personally exhausted from reviewing your material and wading through all the technical jargon while trying to keep a focus on the meat and potatoes of LumeWeb

Looking forward to your responses and hoping these questions help sharpen focus and direction moving ahead. Also I recommending pinning this video to your grants https://www.youtube.com/watch?v=qMxoM1fQMJ4

Best regards
Jacob

2 Likes
24

Abuse Milestone Update

Per the request of the foundation I have fixed outstanding issues with the abuse plugin.