Standard Grant: Lume Web 2025

Grants Approved
2

Introduction

Project Name: Lume Web

Name of the organization or individual submitting the proposal: Hammer Technologies LLC

Describe your project.
Lume Web is a project with a long-term vision focused on web3, privacy, and data ownership. In the short to medium term, we are developing a web2.5 platform that prioritizes freedom and privacy in data storage.

Our 2025 goals include:

  1. Anti-Abuse Plugin

    • Develop a comprehensive anti-abuse system addressing spam, malware, and illegal content according to US law
    • Implement a block system with integration into community block lists such as IPFS badbits
    • Create a web application for content reporting

  2. Bluesky Integration Plugin

    • Develop an integration with “atproto” to store relational user data using SQLite in the reference Personal Data Server (PDS)
    • Back up data to Sia and provide an identity registry as needed
    • Handle IPLD (media) data storage via the portals framework

  3. Nostr Integration Plugin

  4. LBRY Integration Plugin

    • Develop an integration with the LBRY protocol/blockchain for pinning LBRY blob data, leveraging existing reflector code (GitHub - lbryio/reflector.go)
    • Enable bridging between Sia and LBRY from an L2 perspective
    • Facilitate cloud pinning/seeding of published LBRY content

  5. Cloud Import Integration Plugin

    • Enable file imports from cloud services like Google Drive and Dropbox

  6. IPNS Support

    • Enable IPNS record creation and management for content hosted on the portal
    • Provide users ability to create persistent names for their content

  7. Community Portal

    • Implement a public goods, community-run portal at web3portal.com
    • Provide free fair usage (5-10 GB per user) with daily limits.
    • This will be a service equivalent to what mike76 and redsolver are running in their grants.

Who benefits from your project?

  1. Sia Community/Users

    • Access to five new service integrations (four decentralized, one traditional)
    • Foundation-funded community portal as a public good
    • Enhanced evangelism of the Sia network’s decentralization benefits

  2. Mainstream Users

    • Access to diverse hosting services prioritizing freedom and privacy
    • Simplified user experience that abstracts technical complexity

How does the project serve the Foundation’s mission of user-owned data?
Our 2025 focus bridges multiple protocols, enabling users to consolidate their data on Sia. We use encryption-at-rest for data stored on Sia (via renterd), while any end-to-end encryption must be handled at the application level by the respective protocols or clients. We’re expanding to traditional cloud services for data importing and implementing anti-abuse measures to protect the network’s integrity.

Are you a resident of any jurisdiction on that list? No

Will your payment bank account be located in any jurisdiction on that list? No

Grant Specifics

Amount of money requested and justification with a reasonable breakdown of expenses:

Total Request: $73,000
Quarterly Payments: $18,250

Upfront Budget Request: $18,250

  1. Salary Expenses: $55,000

    • Allocated for Derrick Hammer
    • Q1 Salary: $13,750

  2. Annual Project Budget: $18,000

  • Design and Development: $5,000
    • Portal dashboard improvements
    • Cloud Import UX
    • Abuse report webapp and admin functions
  • R&D Infrastructure: $4,000 (subject to testing/scaling requirements)
  • Community Hosting Infrastructure: $4,000 (subject to user demand)
  • Community Hosting Storage Subsidy: $5,000 (may require future increases)

Timeline with measurable objectives and goals:

Q1: Anti-abuse Plugin ($18,250)

Deliverables

  • Report intake system
  • Automated report classification
  • Content scanning system
  • Response workflow management
  • Case tracking database

Validation Criteria

  1. Report Intake
    • Functional abuse@ email endpoint and web form
    • Documented API endpoint with authentication
  2. Report Processing
    • Automated classification and routing
    • Unique case ID generation
  3. Case Management
    • End-to-end workflow tracking
    • Basic analytics/reporting capability

Q2: Bluesky Integration ($18,250)

Deliverables

  • Account migration system
  • PDS-compliant data handler
  • Transfer status interface

Validation Criteria

  1. Data Transfer
    • Account migration functionality
    • Complete post/social graph transfer
  2. Platform Compliance
    • PDS specification conformance
    • Cross-client compatibility verification
  3. User Experience
    • Transfer status monitoring
    • Error handling and recovery

Q3: Nostr and LBRY Integrations ($18,250)

Nostr Deliverables

  • NIP-96 compliant file storage
  • TUS integration for NIP-96
  • Cloud import functionality

Nostr Validation

  1. Core Functionality
    • NIP-96/98 compliance
  2. Implementation
    • TUS protocol integration

LBRY Deliverables

  • Blob storage integration
  • Network participation system
  • User blob management interface

LBRY Validation

  1. Network Integration
    • Blob pinning and tracking
    • DHT participation verification

Q4: Cloud Import and IPNS Support ($18,250)

Cloud Import Deliverables

  • OAuth integration with major cloud providers
  • File import functionality

IPNS Support Deliverables

  • IPNS record creation and management

Cloud Import Validation

  1. Integration
    • OAuth authentication flow
    • Successful file imports

IPNS Support Validation

  1. Core Functionality
    • IPNS record management
    • Name resolution verification

Notes:

  • The progression and timing of milestones may be adjusted or reordered in light of emerging risks, unforeseeable events, or community input.

Potential risks that will affect the outcome of the project:

  1. Technical Risks:

    • Protocol specification changes could require integration rework
    • Performance challenges with large-scale data transfers
    • Community adoption of new protocol features

  2. Infrastructure Risks:

    • Unexpected scaling requirements
    • Storage cost variations based on usage
    • Anti-abuse system resource needs

  3. External Dependencies:

    • API changes from integrated services
    • Regulatory changes affecting content moderation
    • Protocol deprecations requiring redesign

Development Information

Will all of your project’s code be open-source?
Yes.

Leave a link where code will be accessible for review.

Do you agree to submit monthly progress reports?
Yes

Contact Info

Email: [email protected]

Any other preferred contact methods:
Discord: pcfreak30

Early submission

We are aware that we are submitting this about 1-1.5 months before our current grant is completed. Our justification is that we need to ensure continued funding in 2025 (Jan) and cannot afford to have a large gap, as we have no other significant active income at this time. Lume is our full-time mission.

3

Can you expand on these two budget items?

4

My intention is to operate web3portal.com as a community service. After talks with Steve in grant comms, legal made it clear that it needs to be on dedicated infra separate from anything paid (pinner.xyz) - this keeps it as a public good and avoids legal issues.

Basically want to recreate skynetfree.net as a free community service. Budget covers that infrastructure plus the Sia storage fees.

Just requesting to operate the same thing as mike76 and redsolver in their grants.

Thanks.

5

Out of curiosity: is the Upload button not working on https://account.testing.pinner.xyz, or was that the intent?

Regarding your new proposal, I am requesting you to create a long-term roadmap of your project, similar to what the Foundation has but looking into the years ahead. The reason I am asking this is that you keep submitting a new proposal for the coming year at the end of each year, and this is fine but it does leave a feeling that this is going to last forever. Please spare us all the frustration.

6

I just tested this in Brave (Version 1.70.119 Chromium: 129.0.6668.70) and was able to upload video files fine, even in incognito mode. I’d need to reproduce the reported issues to properly fix them if there’s actually a problem.

Regarding the roadmap, I’ve shifted my approach based on community reactions from last year and what I’ve learned from new data and experience this year. Instead of focusing on moonshot ideas about where I want things to go, I’m concentrating more on immediate and medium-term goals. This shift actually came from the community’s request to focus on the portal and bring more immediate value to Sia.

For the period after 2025, I do have other integration ideas from ecosystems I’ve researched. Some of these aren’t ready yet, and others would need significant time to port code to golang.

Ideas I’ve researched for potential integrations:

  • Iroh support (requires go port)
  • TBD Web5 (likely requires go port)
  • Syncthing support (act as a cloud peer, in golang already)
  • ActivityPub support
  • RemoteStorage.io
  • SolidProject.org
  • Hyper/Dat (requires go port)

And for application-level problems:

  • Docs
  • Email
  • Password manager
  • GitHub Pages/Vercel/Cloudflare Pages alternatives
  • Taking Siashare and making a golang version using S5/IPFS & other tech, either as an L3 or portal plugin (undecided) - basically run a simple end-user filesharing service like Wormhole
  • WordPress-related integrations (web 2.5)

But I’ve learned that nothing goes exactly as planned, and what seems like a good idea now might change later. Some of my thinking from 2023 evolved by mid-this year, and I ended up adapting what I built to make it more flexible, ensuring I could bring more features to Sia (which ultimately brings Lume more long-term funding through storage sales).

At this point, any future ideas I might share are just that - ideas. Things are very likely to change at the macro level which will impact the project. While Lume’s big picture goals will eventually return to the high-level web3 work I started with (which will honestly require a fair amount of experimentation), right now I’m following what the community requested: bringing demand to Sia, which obviously supports Lume as well. My focus going forward will be on bringing data to Sia as the community requested, while funding Lume’s bigger picture goals through paid services. The yearly grant proposals reflect this iterative approach - each focused on concrete deliverables that build toward sustainability rather than locking into long-term predictions that might not serve the community’s evolving needs.

I’m also planning to launch feedback.lumeweb.com to give the community a more direct role in guiding the project’s direction.

7

Okay, let me ask you a blunt question. For how many years are you planning to ask funding for development on Lume, and how much time shall pass before you deliver a working product?

8

Lume is already a working product, though we’re continuing to add polish and evolving features with my team. Our immediate plans include:

  • Soft launch EOY for Q1 2025 to work out issues
  • Implementing anti-abuse measures to address liability concerns
  • S5 integration either this year or potentially Q1, depending on redsolver’s development timeline

Regarding funding expectations:

  • 2025 will see us generating sales while continuing development
  • I may request 2026 funding if organic growth in 2025 isn’t sufficient for sustainability
  • There will be two portals:
    • The community portal will continue being grant-funded, and I may seek continued funding for it as a community service even after Lume itself no longer needs project-wide grants
    • The paid commercial portal through pinner.xyz

I understand the Foundation’s need to evaluate KPIs and see return on investment through storage demand. I don’t expect continued funding without demonstrating value - if we’re not generating ANY sales by the end of 2025, that’s concerning, and if we’re not successful within three years (2025-2027), that would indicate a fundamental issue with our execution or market fit. At that point, I would need to return to consulting work to pay bills while continuing the project part-time if the foundation decided to stop funding.

As Danger recently joked, I have ideas reaching toward 2030, and while these align with our original grant’s vision of broader ecosystem integration, I recognize that continued Foundation funding should be tied to delivering tangible value to Sia.

We’re committed to becoming self-sustaining through storage sales and paid services rather than relying indefinitely on grant funding.

9

Also, given you had issues regarding the upload button. Im posting this screen capture for the record to show that upload is working.

10

Thanks for your proposal to The Sia Foundation Grants Program.

After review, the committee has decided to approve your proposal. They’re excited to see the continuation of Lume and the integrations you’re building this year. Congratulations!

We’ll reach out to your provided email address for onboarding. This shouldn’t take long unless your info has changed from last time, but you may still need to adjust your timelines.

11

Progress Report for January

What progress was made on your grant this month?

  • Had to handle weird issues with TUS uploading on the pinner.xyz portal and double check uploading was working properly in general
  • renterd took a lot longer to sync then expected
  • Rest of the month has been spent on rebuilding the billing support.
    • The existing implementation after reviewing things turned out to be badly designed without a full understanding of killbills state machine system.
    • The justpay-written plugin needed a lot of changes to properly use the killbill apis.
    • This included significant rewrites to the hyperswitch java plugin, and dealing with upstream library issues. Most of these updates are still in WIP branches, but the hard parts of the billing backend systems should be done.
    • The billing plugin also has had significant changes and will have more in the future.
  • A design for the abuse report webapp has been created and ready to be ported for the Q1 anti-abuse priority.

Summarize any problems that you ran into this month and how you’ll be solving them.

While I currently consider the billing in a 90-95% ready state, I am intending to do another refactor of the webapp code base and create a frontend framework/foundation with module federation. Due to this, I am punting the billing support until after this is done with the goal of it being finished by the end of Q1.

This will also be used as a basis for the admin webapp/plugin as well.

I don’t wish to put effort into the web part of the billing further and need to scrap it. This also means further UX work will be punted until there is a better webapp foundation.

As a secondary concern, hyperswitch’s cloud pricing has changed, and went to target enterprise, which means I will have to self host a card vault and go through required compliance measures to bring billing online. I was considering it as a temporarily way to deploy faster, but thats no longer in the cards, it seems.

Links to repos worked on this month:

What will you be working on next?

  • I will be bringing the free portal online and doing any required devops work along the way.
    • I am also intend to be doing some server migration efforts and restructure things. Looking to adopt proxmox and put infra inside virtualization, which would allow use of another layer of redundancy/safety as well as using the proxmox sia plugin with an outside renterd instance.
  • Will work on the webapp foundation priority
  • Will work on S5 support assuming no blocking issues from redsolver. I have also made the decision that S5 will be an API-first service upfront with no UI support in the webapp given most planned app use cases will use the API. a UI will be added based on feedback. This is to reduce the eng time needed to bring S5 demand online while balancing other Q1 priorities.
  • Will do needed research on the anti-abuse and create a game plan for it. Will start those efforts if I have the time.
    • Know that anti-abuse will be a subsystem of the admin panel… which will require the new web foundation, so all of this is being prioritized on importance and is very inner-connected.
  • Will likely be doing some portal core design updates and possible plugin API changes, As stated in the past, these will continue as deemed needed for the evolution of the system, and is a large reason why no v1 in semver exists yet.
12

Progress Report for February

What progress was made on your grant this month?

  • Created the bulk of the portal webapp framework.
  • Start migration of webapp components and business logic to a core plugin
  • Started on refactor/restructuring of golang S5 codebase
  • Got the abuse report webapp UI ported to react
  • Started on the UI & code of the abuse admin
  • Did research on the requirements for the abuse service
  • Worked on core portal design changes needed to improve current data models and ensure an abuse service + content scanning can be supported.
  • Got pinner.xyz live. Will be able to get the free portal online this week.

Summarize any problems that you ran into this month and how you’ll be solving them.

The devops for the portal took much more effort then expected, as I had to effectively recreate all infra, and ran into new operational problems to be solved, due to both virtualization being used, and other black swans. Some of these are k8 related issues and others are more akash related issues.

Nothing right now is a further blocker for the portal infra, and the rest will be brought online soon.

Links to repos worked on this month:

What will you be working on next?

  • Getting the free portal infra online 1st week of march.
  • Getting the abuse plugin completed, which will also involve the admin plugin, which needs work too.
  • Working on S5 support further if there are no upstream blocking issues from redsolver.
  • Completing the billing support, ideally launching it if time permits.

My hope is to complete all needed priorities this quarter, but obviously the abuse comes 1st per my grant priorities. So there is a risk that S5 and billing will be worked on in Q2.

13

Progress Report for March

What progress was made on your grant this month?

  • Got lumeportal.com operational (https://github.com/LumeWeb/lumeportal.com-devops)
  • Got backend of abuse plugin completed
  • MVP mocks of both abuse report form and admin ui are completed
  • Further webapp framework architectural work
  • Further portal core design changes to meet the needs of the abuse plugin
  • A Go client library for NCMEC (CSAM Report API)
  • Worked on libraries that standardized rest API HTTP processing and filtering
  • Created unit testing packages for the core portal and mocks

Summarize any problems that you ran into this month and how you’ll be solving them.

  • Roughly 1 week of time was burnt on trying to create a unit test framework library to streamline testing of database queries in plugin code. This was stopped after a week as the requirements turned out to get more complex. As a result, no time has been left to get the webapp parts of the plugin completed.
  • I realized the NCMEC service only allows for reporting content, not for getting any hashes or being told if content is on a blocklist (content scanning). I will have to evaluate on how to integrate reporting CSAM content in the abuse reporting processes, and that may be de-prioritized if required while working on other tasks. ClamAV, however, has been integrated as a content scanner.

Links to repos worked on this month:

What will you be working on next?

  • Completing the webapp ui’s and testing
  • Refactoring anything needed in the abuse plugin
  • Evaluate integrating the NCMEC support
  • Getting the IPFS plugin synced with core portal changes as well as abuse support
  • Continue working on getting the dashboard ported to the new webapp framework as a frontend plugin.
  • Work on S5 and/or Bluesky planning if there is time available

The portals may be online but I expect them to undergo large upgrades, and so I will not be promoting the community portal until things are ready since many systems are in a transition regarding architectural or framework improvements. Specifically, the dashboard and the IPFS plugin will need testing after they get updated to the new core changes.

Q1 Milestone Report

A majority of the milestones has been completed.

  • Report Intake
    • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/service/email.go#L146
    • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/pkg/email/pipeline.go
    • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/pkg/email/imap_client.go
  • Documented API endpoint with authentication
    • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/api/abuse_api.go#L79
    • Swagger document & middleware integration is still TODO
    • Support includes ARF reports https://www.rfc-editor.org/rfc/rfc6692.html
    • Support includes parsing email threads as well
  • Report Processing
    • Automated classification and routing
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/pkg/email/classifier.go
    • Unique case ID generation
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/service/email.go#L491
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/api/dto/abuse_report.go#L28
  • Case Management
    • End-to-end workflow tracking
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/api/admin_extension_case.go
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/db/models/case.go
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/service/case.go
    • Basic analytics/reporting capability
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/api/admin_extension_case_analytics.go
    • Content scanning
      • https://github.com/LumeWeb/portal-plugin-abuse/blob/598e8d11bd46f28f45b07c03f39d3fb6c8ad8fe2/internal/pkg/scanner/clam_scanner.go

I am estimating needing a week to complete the milestone, which is mainly the webapp aspect. I will keep the committee updated on progress and if I end up under-estimating that ETA. I have largely been able to leverage AI to accelerate development on both design and development, So I do not believe that I will require an excessive more amount of time for this milestone.

14

Hi pcfreak30, I am checking out the status of Lume, looks like progress was made on the Q1 anti-abuse milestone, let me know the best way to check out these features on pinner.xyz once they are complete. Also, when I tried out pinner.xyz and ran into a few different issues. The app gives me many 400, 401, and 403 errors as I navigate around, try to create API keys, or upload files. Reviewing your upcoming plan, I would suggest figuring out how to keep the portal and existing functionality stable throughout all the development and refactoring cycles, and how to prioritize doing that community promotion sooner than later, I think you will a gain an invaluable source of continuous feedback and direction from a small group of active users.

15

Thanks for the detailed feedback on pinner.xyz and checking in on the anti-abuse work.

Regarding the errors (400s, 401s, 403s) you hit on pinner.xyz – you’re right, that definitely needs fixing. I’ll dig into those issues once I wrap up the current anti-abuse priority.

A lot of the instability you’re seeing stems from a major refactoring effort that’s actually mostly complete now. The core foundation (web framework updates, module federation, new abstractions, database migrations) is largely in place. The current phase involves updating the different application pieces – like IPFS support, Admin tools, and the Abuse system – to work as plugins within this new structure. This is where the remaining work and temporary breakage lies. Until these plugins are synced up, the APIs might be inconsistent.

Specifically for IPFS, it needs some dedicated effort (a few days probably) to fully integrate with the new internals because I changed how some things work. I did most of the heavy lifting for that back in February in a branch, but it needs revisiting and testing.

Be aware that you will be able to test things on lumeportal.com first before pinner.xyz. My immediate infra focus for the free, community instance will be lumeportal.com. That’s where I’ll concentrate efforts to provide a stable running service for wider use and feedback once these updates settle, before doing whats required on pinner.xyz.

So, the plan for the anti-abuse features is to release them as an updated Admin plugin once they’re aligned with the new core and the priority tasks are done. The abuse webapp side will have to get updated into a new web plugin when im done with it this month as well.

One thing to note is the dashboard frontend won’t be updated right away. Porting the React code and its IPFS/Uppy integrations into a plugin is a separate step that will happen after the backend stabilization. So, the web UI might feel a bit ‘legacy’ for a while even after the backend APIs are updated.

I agree with you on stability and community feedback. This whole refactoring effort, while causing short-term pain, is precisely to build that solid foundation needed for long-term stability on lumeportal.com. Once things are running smoothly there, it will be the right time to really push for community promotion and gather that continuous feedback you mentioned. The goal is to get past these major foundational changes soon so we don’t keep hitting these kinds of breakages and can focus on building out features on a stable base, particularly for the community on lumeportal.com.

Today I will be actively working on the admin webapp side of things and can give the committee an update at the end of the week on where I am at for the milestone.

16

Hello,

Thank you for your progress report!

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

17

A bit late, but enough progress has been made that ill summarize what has been done and what is left in terms of my milestone:

  • Have spent a decent more time on the webapp framework as several gaps were found that needed to be implemented.
  • Have been working on some needed UI components in a high level UI library thats being made that will also be foundational for future UI efforts to have a sort of design system.
  • Got most of the abuse dashboard ported into plugin-format.
  • Got a MVP mock abuse report webapp and a small user side dashboard for viewing a report by an access key created.

Remaining:

  • Porting the mock into plugin format
  • Testing both UI plugins
  • Testing APIs work correctly with the webapp
  • Do system testing with inbound emails
  • Creating swagger
  • Potentially other miscellaneous tasks

My timeline I have found to be off mainly b/c I have had remaining foundational work required.

I will provide another update on progress for this when it makes sense to or my next monthly report, whichever comes 1st.

18

Progress Report for April

What progress was made on your grant this month?

  • Continued chipping away at the webapp framework and the Go aspects for plugin loading.
  • Made headway on the UI component library – building those essential legos.
  • Spent some time wrangling miscellaneous JS tooling.
  • Knocked out about 50% of the abuse webapp frontend.
  • Did a bit of recon on the Bluesky ecosystem and the current state of PDSs.

Summarize any problems that you ran into this month and how you’ll be solving them.

This month’s main blockers were some unforeseen engineering timesinks, mostly tangled up in React and getting those UI legos just right. Here’s the breakdown:

  • React Wrangling:

    • Hit several low-level architecture snags while building the report webapp.
    • Tooling Headaches: Module federation tooling sent me on a few goose chases. Had to sink a fair bit of R&D time over the last month or two to get it cooperating. The hope is to eventually migrate to Rolldown, as they’re building native MF integration which should simplify things.
    • Routing Shenanigans: Ran into issues with react-router, navigation, and how they mesh (or don’t) with module federation. Specifically, index routes aren’t playing nice, forcing manual layout management in some spots :upside_down_face:. I’ve patched things up with workarounds for now, but this needs a proper fix down the line to use the router correctly.
    • Context Fun: Also navigated a variety of fun issues related to React contexts along the way.

  • UI Component Deep Dive:

    • As mentioned before, I’m building out reusable UI legos. The long-term goal is clear: need the ability to spin up tables, dialogs, and forms rapidly without repeating myself (DRY!). Future plugins, like file managers, will also build on these primitives (especially tables).
    • To achieve this, I’ve been investing time in creating solid components and React-based APIs (like useDialog) to make future UI development faster and more consistent.
    • Where the Time Went: Honestly, most of the energy this month poured into this lower-level UI foundation rather than getting stuck on specific application logic. Given I foresee potentially up to 10 different portal plugins, each potentially needing its own web UI, this foundational work is crucial. My experience building monolithic apps tells me this upfront effort saves massive amounts of time avoiding repetitive basic UI builds later.
    • Looking Ahead: Assuming no more surprise React ambushes (famous last words, especially since I’m kind of inventing the plugin system integration as I go), I don’t think the abuse app has much longer. After that, porting the other UIs (Dashboard, IPFS) to plugins should be quicker.
    • The Vision: Over time, this library will likely mature into a more formal design system for Lume, similar in spirit to what the Foundation is building for the Sia apps.

To give the committee a concrete look at the components that have been taking up this foundational effort, here are direct links to the code as of today:

On a related structural note: I’ve also consolidated the apps down. Now there’s primarily just the portal-app-shell (https://github.com/LumeWeb/web/tree/2f5e8b1ad5220dc735582317d3bf454750eab467/apps/portal-app-shell). This shell can be configured at build time (basically just needs a page title and app name/ID) to become any of the needed webapps, as the actual functionality gets loaded via web plugins.

I know this section got lengthy, but I wanted to properly communicate the technical details behind the project delays, the reasoning for the decisions (especially the UI component investment), and what exactly has been consuming time.

Links to repos worked on this month:

What will you be working on next?

  • Finish the remaining abuse milestones (frontend, admin, and testing email processing & routing).
  • Get the existing dashboard and IPFS webapp code migrated into plugins.
  • Circle back to IPFS functionality and fix anything broken during the current refactoring chaos.
  • Dive deeper into Bluesky research and testing:
    • Current plan is leaning towards using rsky (https://github.com/blacksky-algorithms/rsky), a Rust-based PDS. Why? Honestly, the thought of managing distributed SQLite, even with Litestream, gives me a bad gut feeling about DevOps pain… and my plate’s pretty full already :P.
    • Need to grok the atproto data structures to map out the plugin plan. This likely means running both rsky and maybe the reference PDS side-by-side to experiment and learn.
    • rsky uses Postgres and S3-compatible blob storage, which sounds like a decent foundation to integrate with the portal.
    • The approach will likely be an atproto portal plugin that communicates with a separate PDS server (like rsky) via an API. Embedding isn’t really feasible since there’s no mature Go PDS :cry:.
    • Heads up: rsky is actively developed, so some Bluesky features might be in flux or missing (e.g., I saw OAuth support is still being worked on). The good news is the Blacksky project seems to have solid community momentum and funding (https://opencollective.com/blacksky). I believe the critical PDS functions are mostly there, but I’ll re-evaluate this path if major roadblocks appear.
  • Take a look at the billing and S5 tasks list and figure out what’s feasible to tackle soon.
  • Gather user feedback – if I can get updated systems deployed in May. This might slip depending on progress.

As requested previously, here’s a very short, unedited video demonstrating some tangible progress. It shows the abuse report submission flow working. Next steps for this feature are implementing login to view submitted reports, plus adding commenting and file upload capabilities (likely using uppy).

Apologies again if this report feels long, but transparency about the work, the roadblocks, the reasoning, and the path forward felt important. Hoping the foundational work being laid now pays off in faster progress soon!

19

@pcfreak30 Just a quick note that we’re sending to all current grantees.

When submitting progress reports, especially ones that contain work towards your milestones, development work must be easy to access and review. Please directly link to work done, and if that report says “support added for feature x”, please link either to a branch or to a commit range showing the diff for that feature.

Moving forward, we will be assuming that progress that is not linked to directly was not made.

20

Progress Report for May

What progress was made on your grant this month?

  • Worked on query/sort/filter library
  • Given all of the planned plugins for the portal, I ended up going on a side quest and rewriting the entire http layer which uses the go echo router at the base. Took longer then I wanted, but the goal was:
    • To not use a unmaintained router (gorilla)
    • Create a framework for using the routing API to describe itself so swagger.yaml can be dynamically generated and served.
    • In the process of this several repos were worked on:
      • httputil library: Still mostly client side http utilities
      • gswagger fork: Middle layer that does a lot of the openapi work
      • portal-router: a new library
      • Created a new portal-middleware library that extracts out from portal and is a new near-complete rewrite of all the required HTTP needs.
  • Did further webapp framework engineering:
    • Found simpler solutions to past react issues
    • Created unit testing for many core libraries and some of the UI components
    • Started an effort on storybook
  • Took another side quest and ended creating a small testing framework for portal plugins. I have been working in unit test code with the abuse plugin and dashboard and found it was getting to be too much duplicated effort. Goal is to streamline that aspect of development as much as possible.
  • Worked on the abuse plugin. Mainly upgrading it to use new libraries, new routing, verify tests work. Still needs to use the new testing library.
  • Worked on the dashboard plugin. Mainly upgrading it to use new libraries, new routing. Still has a bit of work left.
  • Worked on the abuse plugin. Mainly upgrading it to use new libraries, new routing.

Summarize any problems that you ran into this month and how you’ll be solving

No issues this month other then I hit a lot more foundation work being required, unexpectedly, in the process of trying to build the admin ui & abuse admin.

Links to repos worked on this month:

What will you be working on next?

  • Finish the abuse (I honestly am hoping this takes under a week, the react parts on this are not complicated as I see it)
  • Focus on atproto and figure out its requirements
  • Get the dashboard webapp onto the new framework (my account, etc)
  • Get IPFS back in sync with core, and get a web plugin for it ported over from the legacy webapp.
  • Get the community server operational for user testing.
  • Work on S5

As of right now I am definitely viewing my milestones as being pushed back overall and my expectations are that I will primarily be doing a lot of plugins/integrations in a short period, so I foresee the next 3-4 months as being very compressed once I am able to get everything back online again. That is why I am trying to make that effort as minimal as possible so if im creating a ton of plugins they aren’t hard to make… be it protocol, informational or misc.

Lastly for a visual on milestone progress here is a screenshot of the abuse admin dash, no real data yet.

Selection_2399
Selection_23992257×1000 44 KB

21

Quick Update: Abuse System MVP & Framework Progress

This is not my monthly report for June

Here’s a short video demoing a very rough MVP for the admin side of the abuse system.

The good news is that I’ve gotten unit tests passing for both the DB and API code, all running on the testing framework I built. The core web framework also has its unit tests from May.

On the tech side, two key things:

  1. DB Testing: I learned a hard lesson from a previous side quest: Don’t try to mock the DB. The testing framework now uses temporary SQLite DBs, runs all migrations, and tests against a real structure. It just works.
  2. UI Rewrite: In the process of building this, I rewrote the layout for the admin/dashboard UI (the shared GeneralLayout component). This includes a new menu and profile menu UI.

All major webapp framework challenges are now largely solved. The tooling will likely evolve to a rolldown plugin for module federation, but the path forward is clear.

While there’s a lot more that could be done here, and the TS codebase could use some love, these are low-priority tasks against everything else required this year.

I will be focusing next on:

  • Doing more internal refactoring on subsystems like events, config, cron, db, and possibly more. I want to get to something I will be more happy with before I view it as as final design for a while (as many plugins will depend on it)
  • Creating an abstraction so I can more rapidly create plugins and handle the most common actions (upload, pin, dl). This may include adding to the helper libraries.
  • Get the IPFS plugin using this abstraction and get full unit testing on it, ideally also get it in compliance with the IPFS pinner spec test suite.
  • Get the portal dashboard ported over to the new framework.

I have also found that for the atproto integration, community tooling has been made in several languages and there is both a golang tool at indigo/cmd/goat at fa0e9d0066c10c010f28053a4bd0fa49337a6562 · bluesky-social/indigo · GitHub and a rust tool at GitHub - NorthskySocial/pds-migration: Backend Application for user PDS migration. So leveraging these will largely solve the required backend code for migrating user accounts over, and that then becomes more of a question of the a webapp plugin for a wizard, etc.

I may or may not get to any significant progress on atproto this month. I will be largely focusing on getting IPFS back online and usable again.

That said, I do expect to get my milestones back on track based on the effort done already and what I plan on doing to improve the speed of upcoming integrations.

If the committee has any questions or concerns about my milestones, please let me know.