Standard Grant Proposal: HTLC Upgrade for Sia for use in Atomic Swaps

Grants Approved
1

Project Name: HTLC Upgrade for Sia for use in Atomic Swaps

Name of the organization : Komodo Platform

Project Description:
Based off of the work in a prior SIA grant; Grant Proposal: Sia Atomic Swap with Adaptor Signatures Feasibility Study - #10 , we are submitting a proposal to add HTLC functionality and integration into our Atomicdex protocol of Sia to facilitate atomic swaps.

We have received feedback from the Sia developers regarding our previously delivered work regarding Sia DEX integrations. The Sia team has begun work on implementing our https://github.com/KomodoPlatform/sia_adaptor_sig_study/blob/main/hltc_emulator_protocol/option2_spend_policies.md

here: types: Add PolicyAfter and PolicyHash · SiaFoundation/core@c2d9fd9 · GitHub

The current timeline for when this will be deployed into mainnet consensus is unclear. Komodo Platform developers would like to develop the necessary tools for real world DEX trading in parallel to this ongoing development from the Sia developers.

Objectives:

  • Integrate wallet support for SIA into Komodo Wallet

  • Integrate atomic swap support for SIA within Komodo Wallet DEX. Provide support across a multitude of blockchains. “BTC-like” blockchains and EVM-based coins and tokens at a minimum

  • Provide the Sia team the necessary tools and knowledge base to deploy a white label version of this software if desired. Eg, “SiaDEX”

Who benefits from your project?

The SIA community benefits from extending HTLC atomic swaps, providing a method to engage in P2P exchange and reducing reliance on centralized third-parties for the acquisition or liquidation of SIA coin.

How does the project serve the Foundation’s mission of user-owned data?

“Coins” on a fundamental level are data, ledger entries on blockchain. This project serves the foundations mission of user-owned data by allowing users of SIA coin to non-custodially hold and trustlessly exchange data with other peers .

Budget Estimate:

The total budget estimation for the project stands at $100,000. This is based on an estimate of 1000 hours at a rate of $100/hour. Our rate is justified by our extensive experience and the specialized skills required for this project.

Timeline / Milestones :

Milestone #1 Scope of Work Completion
Description:
The initial scope of work phase is to determine exact work needed to complete the project, including assigning of personnel and tasks. SoW report delivered when complete.

Estimated time: 4-6 weeks

Milestone #2 Development Phase Completion

Description: Development Work on HTLC upgrade and integration into the Atomic Swap Protocol . A summary report will be provided when complete along with monthly updates.

Estimated Time: 8-12 weeks

Milestone #3 QA Completion

Description: Extensive Quality Assurance on the new swap protocol ensuring its safety and reliability.

A test report will be handed over at the completion of the milestone

Estimated Time: 12-16 weeks

Milestone # 4 App Release

Description: Finalizing SIA coin integration and release in Komodo Wallet DEX; A Demo will be provided of the integration into the Komodo Wallet and atomic swap exchange of SIA coin upon completion.

Estimated Time: 4-6 weeks

Total Estimated Time 8-10 Months
Payment Schedule: Monthly, if completed before month 10 , the remaining balance will be due upon completion of the final milestone.

Development Information:

The project will be open source and will be hosted on the Komodo Platform Github

The project will post monthly updates

Contact Information:

E-mail: jason@komodoplatform.com

2

Just want to put in my opinion regarding a white-label version of atomicdex. If regulations in the US are going where I think they are, the future of accessing DEX’s are via web3 itself and centralized frontends will not be allowed. That would put a KYC burden on the foundation.

Komodo could maintain a branch for Sia on GitHub, but the foundation should not have the burden of hosting it.

3

Hello @kmd,
Thanks for your proposal!

The committee is definitely intrigued by this grant idea, and appreciated your work on the feasibility study. They are requesting a few items for further evaluation.

  • Your budget breakdown is lacking, requesting $100,000 without citing destination of funds. We understand that you had an expense breakdown in your report from the feasibility study, but that should be present here, and ideally expanded upon in more detail.
  • Your milestones are very high level. Please provide more detail here.
  • Finally, you cited no risks for your project.

Should you decide to make these updates to your proposal, the committee will re-review.

Thank you,
Kino on behalf of the Sia Foundation and Grants Committee

4

Thank you for the feedback

We will work on getting the missing information updated promptly.

5

Project Name: HTLC Upgrade for Sia for use in Atomic Swaps

Name of the organization: Komodo Platform and Van Ketting BV

Project Description:

Based off of the work in a prior SIA grant; Grant Proposal: Sia Atomic Swap with Adaptor Signatures Feasibility Study - #10 3 , we are submitting a proposal to add HTLC functionality and integration into our Komodo DeFi Framework (DEX) protocol of Sia to facilitate atomic swaps.

We have received feedback from the Sia developers regarding our previously delivered work regarding Sia DEX integrations.

The Sia team has begun work on implementing our https://github.com/KomodoPlatform/sia_adaptor_sig_study/blob/main/hltc_emulator_protocol/option2_spend_policies.md

Here:

https://github.com/SiaFoundation/core/commit/c2d9fd9da64658e1cadbafffc0e5d85dae80b1dd

The current timeline for when this will be deployed into mainnet consensus is unclear. Komodo Platform developers would like to develop the necessary tools for real world DEX trading in parallel to this ongoing development from the Sia developers.

Objectives:

Integrate wallet support for SIA into Komodo Wallet

Integrate atomic swap support for SIA within Komodo Wallet DEX. Provide support across a multitude of blockchains. “BTC-like” blockchains and EVM-based coins and tokens at a minimum

Provide the Sia team the necessary tools and knowledge base to deploy a white label version of this software if desired. Eg, “SiaDEX”

Who benefits from your project?

The SIA community benefits from extending HTLC atomic swaps, providing a method to engage in P2P exchange and reducing reliance on centralized third-parties for the acquisition or liquidation of SIA coins.

How does the project serve the Foundation’s mission of user-owned data?

“Coins’ ’ on a fundamental level are data, ledger entries on blockchain. This project serves the foundation’s mission of user-owned data by allowing users of SIA coin to non-custodially hold and trustlessly exchange data with other peers .

Budget Estimate:

The total budget estimation for the project stands at $100,000. This is based on an estimate of 1000 hours at a rate of $100/hour. Our rate is justified by our extensive experience and the specialized skills required for this project.

The $100,000 will be spent on personnel including

Full-Time Developers

QA/Testers

Additionally we will leverage Komodo developer resources to complete the task at hand as needed

Budget & Milestones

Summary:

Total Estimated Time: 8-10 Months

Total Budget: $100,000

Total Estimated Hours: 1000 hours

Detailed Allocation:

Milestone #1 (Scope of Work): $10,000 (100 hours)

Milestone #2 (Development): $40,000 (400 hours)

Milestone #3 (QA): $40,000 (400 hours)

Milestone #4 (App Release): $10,000 (100 hours)

Milestone #1: Scope of Work Completion

Estimated Time: 4-6 weeks

Budget Allocation: 100 hours x $100/hour = $10,000

The initial scope of work phase is to determine exact work needed to complete the project, including assigning of personnel and tasks. During this phase we will investigate deeper into all the steps necessary to complete integration into Komodo Wallet and dexp2p network, further investigation into the HTLC integration form a detailed plan to execute development, including giving more firm estimation of man-hours involved, SoW report delivered when complete.

Milestone #2: Development Phase Completion

Estimated Time: 8-12 weeks

Being a core phase, a significant portion of the budget should be allocated here.

Budget Allocation: 400 hours x $100/hour = $40,000

Development Work on HTLC upgrade and integration into the Atomic Swap Protocol . Development work will consist of protocol integration of Siacoin into the Komodo Wallet and dexp2p network; development of atomic swap sequence using Sia’s pending SpendPolicy upgrade, A summary report will be provided when complete along with monthly updates.

Milestone #3: QA Completion

Estimated Time: 12-16 weeks

QA is crucial, especially in blockchain projects, for ensuring security and reliability.

Budget Allocation: 400 hours x $100/hour = $40,000

Extensive Quality Assurance on the new swap protocol ensuring its safety and reliability. QA is done alongside documentation and postman collection updates for each feature branch. We have automated tests which run on each pull request after CI/CD binaries are built. Further testing is done in a variety of environments to confirm functionality on win/mac/linux and across tcp/wss. For the apps, we use testgrid.io for automated tests, but also have manual sessions on a variety of devices with the intention of trying to break things, find edge cases with non-standard sequences and ensure no regression.

Milestone #4: App Release

Estimated Time: 4-6 weeks

Finalizing SIA coin integration and release in Komodo Wallet DEX; A Demo will be provided of the integration into the Komodo Wallet and atomic swap exchange of SIA coin upon completion.

Budget Allocation: 100 hours x $100/hour = $10,000

A test report will be handed over at the completion of the milestone.

Risks:

Possible risks that could delay completion of the project in the allotted time include

SpendPolicy adoption delay to mainnet

Unforeseen issues in HTLC emulation leading to inaccurate estimation

Developer Catastrophe

These risks will be mitigated as much as they are in our power to do so, to ensure that we complete the grant in the allotted time.

If delays are to occur for one reason or another, we will immediately inform the Sia Grant Committee upon our discovery and plan with the how to move forward accordingly

Development Information:

The project will be open source and will be hosted on the Komodo Platform Github

The project will post monthly updates

Contact Information:

E-mail: partners@komodoplatform.com

6

@Kinomora here’s the updated proposal. We’re looking forward to your feedback.

7

Hello @kmd,

Thanks for providing us with the requested updates to your proposal. The better budget breakdown, more detailed milestones, and inclusion of risks were very helpful during our assessment. After review, the committee has approved this grant!

We’ll reach out via email to get you set up. Onboarding typically takes a week or two, adjust your internal timelines as appropriate.

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

1 Like
8

That’s great news Kinomora! Thank you

9

Here is the Scope of Work for Milestone #1

1 Like
10

The first progress report has been uploaded to github:

No major hiccups. Progress is within the expectations of the Scope of Work document.

Please direct any questions to “alrighttt” on Discord.

1 Like
11

Thanks for your progress report. Please create a new post with the content directly in the forum as opposed to linking out to Github. We prefer this for archival purposes. Thanks!

12

Progress Report #1

This report was initially published on github on February 23, 2024.

Sia Skeleton Protocol

A minimal implementation of the Sia protocol was pushed to the sia-dev branch of GitHub - KomodoPlatform/komodo-defi-framework: This is the official Komodo DeFi Framework repository

This represents the foundation for the Sia protocol within the Komodo DeFi Framework and serves as a template for all future Sia related work.

This enables the integration of a Sia-based coin within the Komodo DeFi Framework API via the newly created task::enable_sia::init API endpoint.

Usage:

Start the Defi Framework API with the following within coins file:

  {
    "coin": "TSIA",
    "asset": "TSIA",
    "fname": "TSIA",
    "rpcport": 59842,
    "mm2": 0,
    "protocol": {
      "type": "SIA",
      "protocol_data": {
      }
    }
  }

Send a POST request with the following body:

{
	"userpass": "{{userpass}}",
    "mmrpc": "2.0",
    "method": "task::enable_sia::init",
    "params": {
        "ticker": "TSIA",
        "activation_params": {"http_url" : "http://localhost:9980/", "http_auth": "password"}
    }
}

This enables further commands such as my_balance. All endpoints currently provide mocked data or intentionally cause a thread to crash via the unimplemented!() Rust macro.

The relevant code for this “skeleton implementation” can be viewed in this git comparsion: Comparing af571608c34f38d82e3c421f9bd61c5265db0cf8...sia-dev · KomodoPlatform/komodo-defi-framework · GitHub

Docker Testnet Framework

The basis for the Docker unit test framework has also been established within the Komodo DeFi Framework. This is based on the Dockerfile provided by the Sia team within the walletd repo. It has been adapted to establish a “komodo” testnet instead of the anagami testnet.

The foundation address of this testnet has been changed to one controlled by the Komodo API team. This testnet has its POW difficulty lowered to allow blocks to be created on demand.

Moving forward, another testnet will be established in a similar fashion. One testnet will work as a persistent internal testnet for the Komodo API team. The other testnet implementation will facilitate unit tests within the Komodo DeFi Framework and will not be persistent.

The relevant code can be found within this commit: init basic sia docker node · KomodoPlatform/komodo-defi-framework@32fdb85 · GitHub

Usage:

cargo test --features "run-docker-tests" --package mm2_main --test docker_tests_main docker_tests::dummy -- --exact

Walletd CPU miner changes

The need to create an arbitrary number of blocks with the walletd CPU miner was identified. A new parameter, blocks, was added to the mine command of walletd. This new parameter allows the user to specify how many blocks the CPU miner should mine before exiting. This is crucial for future unit tests within the Komodo DeFi Framework.

A pull request adding this parameter was opened: add blocksToMine to testnet miner by Alrighttt · Pull Request #53 · SiaFoundation/walletd · GitHub

Walletd /util/generateaddress Endpoint

A new walletd endpoint has been created to enable address generation from an arbitrary SpendPolicy. This was identified as a necessary first step towards testing the newly introduced consensus mechanisms that will allow for HLTC-like consensus mechanisms.

The code is simple and relies on already established mechanisms for serializing and deserailizing the Go types within walletd. This endpoint is unlikely to be merged upstream or be made available to users unless requested.

The code for this endpoint can be found in this commit: add generateaddress util method; change initial POW on anagami · SiaFoundation/walletd@0b53f40 · GitHub

Usage:

curl -u :password --location 'http://localhost:9980/api/utils/generateaddress/' \
--header 'Content-Type: text/plain' \
--data '"thresh(1,[thresh(2,[after(1708697003),pk(0x0102030000000000000000000000000000000000000000000000000000000000)]),thresh(2,[h(0x0102030000000000000000000000000000000000000000000000000000000000),pk(0x0405060000000000000000000000000000000000000000000000000000000000)])])"'

Next Steps

The Komodo API team will commence with manual testing of the newly introduced Sia consensus mechanisms.

ed25519 signature creation within the Komodo DeFi Framework has been identified as a useful first step in regards to this testing. Given arbitrary ed25519 signing using the DeFi Framework’s private key handler, the API team can begin to produce complex HLTC transactions within walletd.

As this consensus testing goes forward, various endpoints will be added to both the Komodo DeFi Framework and walletd APIs.

The interface between both APIs will be established as needed in order to facilitate this manual testing.

Unit tests utilizing the newly established walletd Docker framework within the Komodo DeFi Framework will be established in tandem with any new code for the Sia protocol integration.

13

Progress Report #2

As per the Scope of Work document, a test report regarding the manual testing processes of Sia’s new “v2” consensus mechanisms is expected as of today. Much of the time in the past weeks was dedicated to establishing the “Sia Rust” library. This task was identified as a necessary step towards further in-depth consensus mechanism testing. A test report will be produced in the coming weeks.

This report has also been published to github: https://github.com/KomodoPlatform/sia-grant/blob/main/ProgressReport2.md

Komodo Testnet walletd Rebase

The Komodo testnet branch has been rebased to SiaFoundation’s latest master branch after previously being some weeks behind due to using the “its-happening” branch as upstream.

This was identified as a critical step forward as the DeFi Framework will require use of walletd API’s newly introduced “indexer mode” and its new endpoints such as api/addresses/{address}/balance.

ref: GitHub - Alrighttt/walletd at komodo-testnet-rebase

Komodo DeFi Framework Draft Pull Request

The initial pull request with all Sia related changes thus far has been opened within the Komodo DeFi Framework github repository. This is a draft pull request. It will be finalized and will be reviewed by the API team before being merged into the dev branch.

As of today, this pull request helps easily visualize all newly added Sia functionality to the DeFi Framework.

ref: feat(sia): Initial Sia integration by Alrighttt · Pull Request #2086 · KomodoPlatform/komodo-defi-framework · GitHub

Sia API Client Interface within DeFi Framework

The initial interface between the Komodo DeFi Framework API and Sia’s walletd API has been established. This is currently only serves as an example on how to implement walletd API endpoints within the DeFi Framework.

Two endpoints have been integrated, /api/consensus/tip and api/addresses/{address}/balance.

ref: https://github.com/KomodoPlatform/komodo-defi-framework/blob/d180505b43f8167bd733263e73804ea60d4c1632/mm2src/coins/sia/http_client.rs#L102

walletd API WASM Research

Some research was performed to assess the feasibility of using the WASM object produced by Sia’s core/walletd within the DeFi Framework API.

The assessment is that it is not feasible to use this WASM object in any capacity. No further related work will be performed.

A bare bones example of how to build and utilize this WASM object was published.

ref: GitHub - Alrighttt/sia-wasm-web-example: A simple demonstration on how to utilize the WASM object outputted by Sia's walletd

Sia Rust

The beginnings of the Sia Rust library have been established.

All functionality necessary for generating addresses from arbitrary UnlockConditions or SpendPolicy objects has been established.

A series of unit tests have been established in both the Go and Rust codebases to easily identify any potential future discrepancies.

The tests found in the above “rust_port_test.go” will immediately identify any changes that must be incorporated into Sia Rust if the Sia developers change any fundamental data types or hashing or encoding functions for these data types.

Next Steps

The testnet setup will be reorganized in the coming weeks as manual testing of the consensus mechanisms continue. The Komodo persistent testnet will be re-established in the coming weeks.

Further unit tests will be established within both the Go and Rust codebases as additional data types are ported to Sia Rust. The current goal is to port all data structures necessary to create and serialize Sia transactions.

The draft pull request within the DeFi Framework will be finalized, reviewed and merged into the dev branch.

The initially planned “Sia Consensus Model Testing Report” will be delivered as the necessary tooling is developed to produce Sia transactions from with the DeFi Framework. The SpendPolicy and UnlockConditions data structures being ported are a major step towards this.

14

Hi @alrighttt, please shift your progress report deliverables to the 2nd of each month, covering the work done in the prior month. Also, while we welcome additional info, please follow the report format outlined here: About the Grants category - #8

Make sure you use that template, and if you’d like to provide additional info you can do that after those reporting requirements are met.

15

Hello @alrighttt,

This is a reminder that your April progress report is now late.

Progress reports are due by the end of the 2nd day of the month, in this instance your April progress report was due on May 2nd.

Please be sure to follow the Progress Report Template and submit your progress report so we may continue funding your grant.

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

16

Apologies, I am preparing the report for later today. Will not happen again.

17

Progress Report Form

What progress was made on your grant this month?

The first pull request to the Komodo DeFi Framework was created, reviewed and merged into the dev branch.

The API client within the Komodo DeFi Framework has been refactored significantly since this pull request.

Further progress has been made on Sia Rust. Work has begun on implementing serialization and encoding of the Sia v1 transaction format.

A docker test suite was established to allow running a custom walletd instance in the DeFi Framework CI. This will be heavily utilized to closely mimic the real world Sia consensus model.

Summarize any problems that you ran into this month and how you’ll be solving them.

Please summarize your issues into a few sentences or bullet points:

We had anticipated further integration of the Sia API endpoints within the Komodo DeFi Framework. However, the “index mode” was not made available until recently. We instead shifted our focus to Sia Rust.

What will you be working on next?

Please summarize your development goals into a few sentences or bullet points:

  • Next code sprint pull request has been drafted. This will be reviewed and iterated upon next week.

  • Test Sia walletd pull request #112 to ensure the wallet balance and UTXO fetching mechanisms are sufficient for our needs.

  • Integrating new index mode API endpoints to the DeFi Framework.

  • Complete Sia v1 transaction support.

Links to repos worked on this month:

18

Hello @alrighttt

Thank you for your progress report!

Regards,
Kino on behalf of the Sia Foundation and Grants Committee