February 2016 Roadmap
-
1. Improved Networking Code
We've gotten a lot of complaints about now much bandwidth Sia is consuming, and we've tracked most of the bandwidth issues to inefficient transaction propagation, inefficient block propagation, and inefficient blockchain downloading. We've spent some time getting to the heart of these inefficiencies, and believe that our next client will be somewhere around 10x as efficient when it comes to using network resources. Not all of the gains will be available immediately, but as people continue to upgrade their clients, more clients will be taking advantage of the improved protocols.
2. Batched File Contracts - Giving Sia Scalability
Currently, Sia makes somewhere between 48 and 200 transaction for every single file that gets uploaded to the network. This won't scale, and as we're seeing in our congested network, is already failing to scale. We've known that we would need to batch file contracts for a long time now, and laid much of the groundwork to do so even before June 2015 prototype launch. We can fix this by reducing the number of transactions to about 200 per person, which still means we have limited amount of scaling, but it will go substantially farther than it would have previously. There's a second, less secure method that can be used to batch groups of people, but we're not going to be pursuing that soon. At 200 transactions per person, Sia has enough room for about 100,000 users. Beyond that, we will need to use multi-person batching, but the theory is still being confirmed/developed and we've got time before we need to cross that bridge. (other alternatives include making multiple chains and having some smaller 'master chain' that ties everything together)
There's going to be a pretty big usability change that comes with creating batched file contracts. Instead of pay-as-you-upload, Sia is going to switch to a pay-in-advance mode, which is pretty much how the modern world works right now. IE, with DropBox, you pay $10 and then you get a month of service. You'll open large contracts (you decide how large - 1GB, 5GB, 200GB, etc.) with a bunch of hosts at the same time, putting enough money to cover the cost into the transaction. While you'll have to put enough money into the contract in advance, you can actually have the contract mostly pay out to you, instead of to the host. The file contract will be open, but the host will not be getting any of the money. Then, as you upload files, you revise the file contract to pay the host.
So, you'll have to put like 50,000 SC up front (2,000 per host for 25 hosts), but then you'll get back any that you don't use after 3 months. For larger uploading, you might choose to pay for 500,000 up front. You lose 3.9% to the siafund fee, but otherwise you don't spend any of it until you've actually made uploads that spend the money. You'll get refunds for anything you don't use. Then, we can put all of your files into the same file contract. File contract revisions only get broadcast right before the file contract expires. You can make 1,000,000 file contract revisions, and only the last one will make it onto the blockchain.
3. Sia soft-fork, supporting transaction with time deadlines and supporting Schnorr Secp256k1 signatures (faster, ~more secure)
Sia is going to soft-fork to support 2 new features, and they are going to come at the same time. The first is Schnorr Secp256k1 signatures, which are faster and more secure. They also allow for the use of HD wallets, which our current signature scheme does not allow. And they also for compact multisig, which Sia makes heavy use of (host and renter both do multisig). While we won't be adding support for the multisig soon, eventually it will mean that we can reduce the size of our file contracts and file contract revisions by about 30%.
We're also going to be adding a timeout for transactions. Currently, after you broadcast a transaction the only way to nullify it is to double-spend it and hope that a miner accepts the double-spend. This is both annoying and actually creates a small vulnerability (will disclose later) in the host-renter protocol. But, when you have timeouts, you know for sure that your transaction will not be valid if it is still unconfirmed after a certain height. We'll be doing the softfork before we take advantage of any of the features added by the soft fork, the miner's will need to have >51% mining the new code in order for it to be secure. Miners that don't upgrade will still be protected, and won't be at risk of mining bad transactions.
Following the soft-fork there will be some long-awaited upgrades to the wallet, including faster startup time, faster unlock time, and better backup management.
4. Multi-drive Host w/cached Merkle roots
The host is also getting another round of upgrades. There will now be support for adding multiple drives, and the host will be more efficient when creating storage proofs on large files. We are also finally adding support for Sybil resistance, and setting the stage to support dynamic IP addresses as well. Dynamic IP addresses are not going to be ready this month, but they will be closer after this next round of upgrades.
We're looking for feedback on this roadmap, but overall feel that this month should be a significant step forward in making Sia stable and ready for production. We're hoping to reach 1.0 by the end of March 2016. We do believe in placing software quality first, and will not release 1.0 until it is ready to be called that. In the meantime, I'd like to thank everyone for using the beta and tolerating the bumpiness that comes with beta software.
-
Very nice to see transparency on current and projected issues.
Could you include a timeline for #2 ?
- Auxiliary comments:
As a side effect of this morning's coffee I offer the following loose idea:
While adding a secondary block chain to simply manage what might cripple a single BC leading to your 100,000 person limit might be new territory for BC technologies, I think it's a valid direction to explore. Imagine current block chain models as a single strand of a double helix of DNA, now imagine what you get when you increase dimensionality and have two intertwined block chains, interdependent. Without digressing I would like to see better planning such that blockchains can merge into composite chains.
Hats off SIA team, file systems are tricky.
I have been entertaining a model to bring containers to block chain models, simply because I REALLY like the spoon.net hat they sent me - just kidding, but containers could be stored as files proxy of SIA as it stands, I am interested in a block chain file system - sure I like Microsoft's new Protogon (Resilient File System (ReFS), but I foresee an ability (while SLOW) to allow for a truly decentralized file system, if not in whole, in part- permissions and all. I am picturing a container, but INSIDE the container? it's 100% decentralized.
I'm settling for SIA and STORJ and QORA and various other solutions that are allowing for nice tidy files to be put into decentralized storage.
My take ? SIA and the likes are hands down my POV - the first and only REAL block chain technologies the 'average' consumer can make use of (beyond ledger/transaction tracking services such as bitcoin, [insert 750+ alt coins here]) hands on, no hastles - NO command line- just nice storage solutions comparable to the very centralized ones that placate the airwaves of most AM radio stations these days 'For $30 a month, user XYZZY - and we back up blah blah blah'. So GET THIS RIGHT SIA heh... the world is counting on you.
What will be MOST interested to me is if smaller companies start repackaging SIA services as front end storage services... nothing wrong with that, everyone wins. I see it. Who knows, maybe one day Amazon says 'yeah, let's just outsource another 30 petabytes to SIA this month, they can get us a lower cost than our datacenters'.
While you list some pretty serious issues above, I patiently await this offering. Such exciting times.
Tim Miltz
