Consensus DB downloads
The Sia blockchain is now too large to host on pixeldrain, I've written a guide on the wiki explaining how to bootstrap new Sia clients: https://siawiki.tech/daemon/bootstrapping_the_blockchain. I'll keep the rest of the post intact for historic purposes.
The blockchain of the Sia client can take a very long time to download. So if you're in a hurry to try the new release and don't want to wait an entire day (or more) for the client to sync you can use one of these download links:
1.2.1: https://pixeldra.in/u/GRp8fn (Height 103937, 2017-05-09)
1.1.1: https://pixeldra.in/u/ERtTMk (Height 95904, 2017-03-16)
1.0.0: https://pixeldra.in/u/QsxziQ (Height 57738, 2016-06-29)
0.6.0: Expired (Height 51661, 2016-05-23)
0.6.0-RC2: Expired (2016-05-05)
0.5.2: Expired (2016-05-03)
Here's a guide to installing one of these databases:
- Shut down the sia client. Look in your task manager to see if there's no child siad processes running.
- Navigate to the
consensusdirectory. With the UI release it's located in
Sia/consensus/in the application data directory, see this page for the location on your system: https://siawiki.tech/daemon. And for the commandline version it's just in
- Download the consensus.db file from pixeldrain and use it to replace the file in the consensus directory.
- Start the sia client again and enjoy a fully synced blockchain.
After bootstrapping the client may take a few hours to start, please be patient and don't kill the process while starting. It may look like it froze, but it's actually indexing the blockchain to look for transactions related to the wallet.
If anyone asks I can upload updated ones later.
In addition to date, could you note the bloch height of each consensus file? Also, will you be updating these frequently?
By "long time", my experience is that the IBD takes about 4 - 5 hours to complete, which isn't all to bad. Still, starting with one of these consensus.db files will speed things up.
I don't really know at which height these were when I uploaded them, that's why I added the dates instead.
For me downloading the 0.6.0 RC2 blockchain took 9 hours, I wish I had one of these when I started syncing :P .
I will be updating the links when people ask for it, or every few months or so.
For me, neither worked - it seems that the software checks every single TX before getting started (step 3/7 loading transaction pool) - which would be the equivalent of downloading the blockchain from scratch...
@creds The 0.6.0 download shown here is for a release candidate, a lot has changed since these were uploaded and they're no longer compatible. I will update the list with a db for the 0.6.0 release.
The rescan for the tpool can take a while, perhaps an hour or so, but it still should be faster than downloading the blockchain from scratch. If you load the blockchain before starting
siad, it should speed things up a lot and not need to do any heavy downloading.
I'd like to note that sia consensus DB may download significantly slower than Bitcoin's blockchain. The latter took my system about 3 days to get from scratch to actual state (and it's more than 70 GB) while with sia i'm waiting 5 days already and got only to block 46875. The bottleneck, I guess, is the storage random writes speed. Here's what it looks like in task manager:
Had I known that the DB download is that disk-intensive, I'd put it on SSD... Or should I expect the pressure to persist after the download finishes and move it anyway?
@Veber The intense I/O will stop after the blockchain download completes. I don't think it should be taking 5 days though, even without an SSD. Do you have a slow drive?
A combination of random writes and calls to FSync() are what's causing most of the stress.
@Taek Thanks for the reply. The drive is Microsoft storage space with parity (sort of software RAID5 with extremely large stripes) It's indeed somewhat slow, but this kind of load for it is the worst I can think of performance-wise. But it's almost synced now, and as it's a one-time operation and the configuration is probably rare, I guess it's not a serious issue.
Upd: after the download finished, the storage is indeed mostly idle.
Oh yeah, doing RAID on top of the random writes that we use in the database would destroy performance, that completely explains why it was taking so long.
After 1.0 is released we're going to be spending some energy on optimization, and also on finding people to help us with optimization. Though I'm not sure there's much we can do about the slowdowns associated with RAID. We're storing a bunch of hashes on disk in sorted order, there's not a whole lot we can do to improve that performance (other than batch things better so it's not nearly as random).
Hey, I just want to say that it seems like the 0.6 consensus.db is in fact compatible with 1.0. But I will still update it anyway, next time I restart my wallet.
Link is there now!
A few updates:
- It still makes sense not to keep the blockchain on a slow drive as the wallet unlocking time can get ridiculously high (more than an hour, no kidding!)
- I had no success bootstrapping download with your blockchain snapshot, @fornax. Did you verify that it's usable? Maybe there are some caveats?
Should be usable, I use it for bootstrapping new nodes too. It just takes a while to load the consensus set for the first time, but it's still faster than completely downloading it from scratch.
It has come to my attention that the consensus database has grown over 5 GB since I last uploaded it.. Pixeldrain has a file size limit of 5 GB, so I can no longer update this list through traditional methods. The last version I uploaded still works fine with the latest version of Sia, so you can use that for now.
For the next database version I may set up a little webserver that automatically releases the latest consensus db to the public every day or so. Or I need to increase pixeldrain's file size limit, this is also an option since it's my own website. But that would require modifying the code, I'd rather not do that.
Hi, Could you please add the db file for 1.3.0?
Hey @pashapanther, this thread has been abandoned since a long time. I have written a guide over on the wiki to replace it. Check it out here: https://siawiki.tech/daemon/bootstrapping_the_blockchain