Standard Grant: Native prometheus endpoints for Sia services

Grants Rejected
1

Introduction

Project Name: Native Prometheus for Sia

Name of the organization or individual submitting the proposal: Bustedware LLC

Describe your project.

Sia data sources Grafana was presented to the community in early February 2024 based on forks of Sia’s main services, hostd, renterd, and walletd. The forked version of these services contain native Prometheus endpoints for use with the dashboards available from the sia grafana repo. Since early February work has already been performed for refactoring and merging into the official Sia repositories, having already completed hostd. Related PRs here:

  1. https://github.com/SiaFoundation/hostd/pull/300 - completed
  2. https://github.com/SiaFoundation/renterd/pull/1068 - in progress
  3. https://github.com/SiaFoundation/walletd/pull/56 - in progress

How does the projected outcome serve the Foundation’s mission of user-owned data?

This effort serves to supplement the Sia Grafana grant by making user-owned data more accessible. Having Prometheus endpoints natively integrated into hostd, renterd, and walletd services is more desirable since forks tend to lag a little behind features and bug fixes from the main repositories.

Grant Specifics

Amount of money requested and justification with a reasonable breakdown of expenses:

  • $24,000 for part time salary work over the next 6 months of both the Sia Grafana and refactor of forked code bases.

  • Q1 - Monitor for community feedback in Discord with questions related to Prometheus or Sia Grafana, fielding any questions along the way as well as integrating improvements based on the type of feedback being received.

    • golang learning curve, sia code base learning curve for following proposed updates to sia core
    • internal/prometheus.go is in every codebase hostd, renterd, walletd. this could potentially be moved to core? I don’t know anything about that codebase, so adding to core and updating hostd, renterd, walletd could be useful. Especially since there are many issues identified with the renterd PR already that need addressed which is present in hostd.
    • similar for the writeResponse() method. move that into core and update hostd, renterd, walletd codebase
    • PRs should be submitted for each service by the end of this quarter to start receiving feedback from the development team.

  • Q2 - Monitor for community feedback in Discord with questions related to Prometheus or Sia Grafana, fielding any questions along the way as well as integrating improvements based on the type of feedback being received.

    • continue integrating internal/prometheus.go and writeResponse() into hostd, renterd, and walletd services. working through any issues identified by the development team in the PR
    • PRs should be merged for each service hostd, renterd, and walletd. All issues have been worked through for each repository

What are the goals of this small grant?

  • Changes to any of the hostd, renterd, and walletd code base which may require an update with its Prometheus endpoint implementation and/or corresponding Sia Grafana dashboard panel

  • Bake in community feedback to the dashboard. Like we saw with hostd once the refactor work made it to the official hostd repository, various changes and improvements were made to the hostd dashboard as a result of community driven feedback. Sia Grafana also improved as a result as more community members started using it I found better ways for users to get started, such as changing the siagrafana.json schema to make it easier to configure all the required values before running any automation scripts

  • I will also gain valuable knowledge from the various Sia code bases as well as learning to manage more complex Go lang projects like ones with external libraries such as Sia Core.

Potential risks that will affect the outcome of the project:

  • If the community is not engaged with the dashboards or interested with Prometheus endpoints in general that will significantly hinder any ability to drive progress using insights gained from their perspectives.
  • The ability to move the proposed implementation into sia core in time, get through any issues identified by dev team from subsequent PRs, is a lot to squeeze in for 6 months and 3 services accounting for my small golang learning curve and sia core learning curve.

Development Information

Will all of your project’s code be open-source?

Yes

Leave a link where code will be accessible for review.

https://github.com/SiaFoundation/hostd
https://github.com/SiaFoundation/renterd
https://github.com/SiaFoundation/walletd

Do you agree to submit monthly progress reports?

Yes

Contact info

Email: jbills@bustedware.com

2

Hi, thanks for this renewed proposal. I have a couple of comments on it.

Risk assessment is an integral part of a grant proposal, and it is as important as the project budget and the project milestones (the latter are also missing in your proposal). Would you include them, please.

To avoid any misinterpretation: for the purpose of evaluating, this is a standard grant, even though it is smaller than your original proposal. Small grants are limited to $10,000 in budget.

3

Hey thanks @mike76 . I have incorporated the requested items into the proposal and updated the proposal title to reflect its status as being a Standard Grant

Looking forward to hearing back from the foundation/committee

4

Thanks for your new proposal to the Sia Foundation Grants Program!

This proposal was submitted after our Wednesday cutoff last week, and unfortunately we didn’t have time to review it during this week’s meeting. It will be reviewed during the committee’s next meeting on April 2nd. Thanks!

5

Thanks again for your new grant idea. After review, the committee has decided to reject this proposal. There were a couple of noted concerns, but they primarily cited the following.

  • Your last grant required significant team work hours to merge into the code base following completion. This type of work is probably better handled by our team so we can avoid that distinct hand off in the future.
  • Your primary milestones listed for both quarters involve monitoring community feedback, which is difficult to measure.

We’ll move this to the Rejected section of the forum. If you need any other feedback, please let us know! And if you feel you can address the committee’s concerns with a new grant request, of course feel free to resubmit.

6

Your last grant required significant team work hours to merge into the code base following completion. This type of work is probably better handled by our team so we can avoid that distinct hand off in the future

Only because someone caused a huge cluster f* w/ it all ;)

And then I have to deal with shit like this: Prometheus: remove id label on hostd_consensus_state_index by bustedware · Pull Request #312 · SiaFoundation/hostd · GitHub that makes absolutely no sense and then receive no feedback.