Small Grant: Sia SMB Mounts

Introduction

Project Name: Sia SMB Mounts

Name of the organization or individual submitting the proposal: Michael Bulanov

Project Description

SMB (Server Message Block) is a communication protocol used to share files, printers, etc. between the network nodes (Server Message Block - Wikipedia). Even though other file sharing protocols exist (NFS, AFP, etc.), SMB is one of the few protocols natively supported by Windows.

This project includes developing a standalone service compatible with all popular platforms (Windows, Linux, and MacOS), which uses renterd as the storage backend to mount network shares and then access them as the user would normally do with conventional network shares.

How does the projected outcome serve the Foundation’s mission of user-owned data?

The project aims at expanding the user base, especially to those who have to stick to Windows, either due to personal preferences, or because their employer uses Windows as the corporate OS.

Grant Specifics

Budget and Its Justification

The project owner requests an amount of 9.000 USD, which shall constitute the part-time salary of the project developer (who is the same person as the project owner). This amount shall be spread evenly across the project timeline, which is 3 months.

Project Goals

The outcome of the project shall be a fully-functional standalone service (siasmb), which uses renterd as the storage backend to mount network shares. Although the main focus shall be on Windows, other popular platforms (Linux and MacOS) shall be supported, too.

Potential Risks

The only risk that the project owner could identify involves any yet undiscovered technical roadblocks that could prevent or significantly delay the implementation of the SMB protocol on one or more platforms. However, based on the preliminary research, this risk appears to be rather small.

Development Information

Will all of your project’s code be open-source? Yes

Leave a link where code will be accessible for review.

Do you agree to submit monthly progress reports? Yes

Contact Info

The contact info of the project owner is already known to the Grants Committee and the Sia team.

1 Like

What advantage would having a seperate binary for SMB bring that mounting the S3 API via rclone wouldn’t solve?
And what moved you to making it a standalone service instead of implementing a native Renterd rclone endpoint for example?

I’m more of a Linux person, so I might be missing something obvious :slight_smile:

1 Like

The high-level objective is to let a Windows user right-click an icon in their File Explorer and attach a network drive. AFAIK SMB allows that (almost) natively.

True, a lot can be done already by combining rclone and renterd. I just want to make the UX as seamless as possible.

2 Likes

It will be so great !

1 Like

Thanks for your latest proposal to The Sia Foundation.

The committee has voted to approve your proposal! This grant request was succinct and detailed the useful tool you intend to build. With your history of completed projects it was an easy decision for the committee.

We’ll reach out via our established channels to get you started. Congrats!

1 Like

Thanks Steve and The Committee!

Hello @mike76

This is a reminder that your progress report is due.

Please be sure to follow the Progress Report Template and submit your progress report by the 2nd of each month. Thanks!

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

Hi Kino,

According to the agreement, my first progress report is due on September 2nd as the first grant payment is due on August 15th.

Regards,
Mike

Progress Report August

What progress was made on your grant this month?

  • Set up my workplace (this has actually been quite a challenge, because I’ve just moved to another country)
  • Studied the SMB and SMB2/3 protocol specifications
  • Defined basic structures
  • Started working on the negotiation and session setup code

Links to repos worked on this month:

What will you be working on next month?

  • Define requests and responses relevant for the renterd integration
  • Implement SMB2 support in addition to SMB

Hello @mike76

Thank you for your perfect Progress Report!

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

Progress Report September

What progress was made on your grant this month?

  • Made a design decision to stick to SMB2 dialect 2.0.2 for simplicity
  • Implemented NTLM authentication
  • Implemented and debugged dialect negotiation and session setup

Links to repos worked on this month:

What will you be working on next month?

  • Debugging session setup issues on different platforms took me quite some time. Next month I am going to do some cleanup in the codebase and move on with the renterd integration.

Progress Report October

What progress was made on your grant this month?

  • I can now use my integration to connect to a renterd instance from an SMB client on Windows, Linux, or MacOS, and retrieve the tree structure.
  • Multiple renterd instances are supported.
  • Multiple buckets within a single renterd instance are supported, too.

Links to repos worked on this month:

What will you be working on next month?

  • Next month I’ll be working on reading from and writing to renterd-SMB shares.

Hello @mike76

Thank you for your progress report!

Regards,
Kino on behalf of the Sia Foundation and Grants Committee

Progress Report November

What progress was made on your grant this month?

  • Implemented reads, writes, deletes, renames, and directory creations.
  • Tested the software with several different clients on all major platforms (Windows, MacOS, and Linux).
  • Spun up a public server to test the integration.

Running the public server provided some valuable insights:

  1. It is of the key importance to protect the infrastructure from bots, because opening the SMB port (445) seems to attract tons of them within seconds.
  2. Some Linux clients do not give the server any information about the size of the file being uploaded. This creates a challenge because of the way multipart uploads work.
  3. A few bugs were discovered.

Links to repos worked on this month:

What will you be working on next month?

  • Within the next few days I shall:
  1. Iron out the remaining bugs.
  2. Write up the README file.
  3. Release the v1.0 of the software.

After that, the planned scope of the grant shall be completed. For the future, some more work may be foreseen, like:

  1. Rework the renterd client as soon as the API is finalized (with some breaking changes).
  2. Implement newer protocol dialects including SMB3, because SMB 2.0.2 turned out to be limited to a chunk size of 64kiB (max. 256kiB if the client supports large MTU), which is far less than the sector size of 4MiB on Sia.