DIY Sia storage farm
-
And just like that Backblaze announces a new storage pod: https://www.backblaze.com/blog/open-source-data-storage-server
-
@Fornax said:
And just like that Backblaze announces a new storage pod
at a cost of $36.86/TB, which is still higher than 8 TB drives in a regular full-tower chassis.
The backblaze pods have very high density, however, which is required for housing 500 of these pods.
Bottom line seems to be a home-user could actually be competitive with enterprise storage providers.
-
@Taek said:
Do you need 1000W power supply?
No, with an estimate of 20W/drive 600 W total should be sufficient.
It's worth pointing out that in your rig above you've only got 6 SATA connectors, but 12 drives, so that's not going to work.
I also put in some 4-port SATA PCIe boards. These are cheap ($16/ea).
-
Any thoughts on using external USB hard drive enclosures?
There are cheap USB hard drive enclosures. Here's one for ~$13, with free shipping. The cost for 30 is ~$400:
http://www.amazon.com/inch-Silver-External-Drive-Enclosure/dp/B00S0UCEF6/You would also need powered USB hubs. Here's a 13-port hub for ~$20, also free shipping if you buy enough. You'd need 3 of them for 30 ports, which costs about $60:
http://www.amazon.com/dp/B00HL7Z46K/ref=psdc_281413_t1_B0051PGX2IThat's ~$460 for the hardware that can host 30 hard drives & connect them all to a computer. So it's cheaper than one of the enterprise-class storage enclosures. With the $215/drive cost from in-cred-u-lous you end up at $6910 for a farm fully loaded with drives. That's a cost of ~$29/TB before RAIDing. And a system like this can be added to gradually, as slowly as 1 hard drive at a time. So you can start out very small & grow at any speed.
What I don't know:
- How much processing does a farm controller need to do, say per TB in a fully loaded farm? What kind of CPU is needed to handle it?
- How many drives can reasonably be farmed before saturating the bandwidth of a single USB port?
- Is there any cooling needed if you stack 30 USB hard drive enclosures in close proximity? It's surely not an anticipated use.
- What's the total power usage of the USB hubs in a setup like this? This impacts monthly cost of the farm.
- Would you need hubs that provide a certain amount of power? Or do all powered USB hubs provide the same amount of power per port?
- USB drives would need to be software RAIDed; would this hurt performance too much?
- Would it be better/cheaper to use a USB->SATA adapter? Are a set of towers purely used to mount drives cheaper than 30 USB hard drive enclosures?
So tempting to just get a Raspberry Pi farming with an array of USB drives connected to it. Not sure if it's possible. I see from Taek the memory cost to host a farm is high; would it work if most of it was in virtual memory? You could set aside a sufficient portion of a large SD card as swap space.
-
Consider remanufactured brand shit, like this, spit cheap
http://www.serverhome.nl/storage/nas-server/hp-proliant-dl180g6-14.html
If you want to order in bulk, let me know
-
HP ProLiant DL180G6
Total: € 692,90
Incl tax:€ 838,42
HP ProLiant DL180G6 mainboard / chassis
HP ProLiant DL180G6 rackserver
CPU
1 x Heatsink HP ProLiant DL180G6 P/N: 507247-001
1 x 2.26GHz / Quad Core / QPI 5.86 GTs / Cache 8M / TDP 60WXeon L5520
Memory
6 x 8GB 2Rx4 PC3-10600R DDR3-1333 ECC, Samsung
Remote Acces - iLO
No iLO
Raid / Storage Controller
No raid. Attention: this server has no Harddisk / RAID controller.
HP SmartArray Memory
No raid memory
HP SmartArray BBWC Batery
No raid battery
Harddisk
No harddisk
Bracket / Caddy / Tray
12 x Harddisk Bracket 3.5" SAS / SATA Type HP ProLiant G1 - G7 : ML110G7, ML150G5, DL320G4, DL360G5, DL360G7, DL380G6, DL380G7, etc. incl. 4 screws P/N : 373211-001, 373211-002, 335537-001 To mount your own harddisk. Delivery time: 7 days
PSU / Power Supply Unit
2 x HP HSTNS-PL18 Power Supply, 750W, P/N: 506822-201, 506821-001, 511778-001
-
Looks like the enclosure I linked actually powers itself. First one I looked at was powered just by the USB port. So with powered enclosures maybe you could use unpowered USB hubs. Then you'd need power strips to give you enough electric outlets to feed all the enclosures.
-
@coinmonkey Buying refurbished equipment is a good option, though potential buyers should know that the maximum storage/drive capacity of "legacy" equipment might be limited (up to 42 TB, or just 5 TB drives, for the model series you link to).
PS: Could you please remove your second post, however, as it just adds noise to this thread. A link is sufficient. Thanks!
-
@HoteiLife said:
Any thoughts on using external USB hard drive enclosures?
For convenience, I would personally prefer a single enclosure, like a full-size tower or something. If you need to expand what a tower or rack module can hold, however, external enclosures is a good idea. Are there enclosures that fit multiple drives but does not force on your RAID? i.e. 1 volume per drive.
-
@in-cred-u-lous said:
For convenience, I would personally prefer a single enclosure, like a full-size tower or something. If you need to expand what a tower or rack module can hold, however, external enclosures is a good idea. Are there enclosures that fit multiple drives but does not force on your RAID? i.e. 1 volume per drive.
There are, but they all seem to be more expensive per drive. Even the USB 2.0 enclosures without RAID. Here's a 4 drive enclosure for $380, which is $95/drive:
http://www.newegg.com/Product/Product.aspx?Item=9SIA0AJ35C4582There's an 8 drive enclosure for $300, which makes $37.50/drive, still over the $13/drive for single drive enclosures:
http://www.newegg.com/Product/Product.aspx?Item=9SIA8T93RU6064For whatever reason the single drive enclosures are seemingly the cheapest option by a large margin.
-
@HoteiLife said:
For whatever reason the single drive enclosures are seemingly the cheapest option by a large margin.
An alternative option to USB enclosures is external SATA enclosures which is a lot faster:
http://smile.amazon.com/Vantec-Inches-Aluminum-Mobile-MRK-M2512T/dp/B00IAUP3OK ($9 / drive)
http://smile.amazon.com/Sans-Digital-HDDRACK5-5-Bay-Organizing/dp/B001LF40KE ($7.50 / drive)Edit: I read the reviews of these and the first option sounds really bad and probably best avoided. Maybe there are better options but I could find none.
The first option is for 2.5" drives, which can be useful if you have a bunch of these around (from old laptops, external drives etc). You can't daisychain SATA drives though so you still need controller boards with additional eSATA/SATA breakouts.
-
@in-cred-u-lous said:
An alternative option to USB enclosures is external SATA enclosures which is a lot faster:
http://smile.amazon.com/Vantec-Inches-Aluminum-Mobile-MRK-M2512T/dp/B00IAUP3OK ($9 / drive)
http://smile.amazon.com/Sans-Digital-HDDRACK5-5-Bay-Organizing/dp/B001LF40KE ($7.50 / drive)The first option is for 2.5" drives, which can be useful if you have a bunch of these around (from old laptops, external drives etc). You can't daisychain SATA drives though so you still need controller boards with additional eSATA/SATA breakouts.
Nice, with cooling & designed to be stackable. I like it. I think you can get USB to SATA adapters. But what they would cost & how many you could get connected total I don't know.
-
Guess you can also get splitters to share single SATA ports among multiple drives. Again at what point the SATA channel becomes saturated is a consideration. Here's a 1-to-4 splitter:
http://smile.amazon.com/Cable-Matters-Internal-Mini-SAS-Breakout/dp/B012BPLYJC/
-
@HoteiLife said:
I think you can get USB to SATA adapters. But what they would cost & how many you could get connected total I don't know.
The ones I've seen are all 1-to-1 and I'm not sure they're bidirectional either... In any case, you loose the speed advantage of SATA.
-
@HoteiLife said:
Guess you can also get splitters to share single SATA ports among multiple drives. Again at what point the SATA channel becomes saturated is a consideration. Here's a 1-to-4 splitter:
http://smile.amazon.com/Cable-Matters-Internal-Mini-SAS-Breakout/dp/B012BPLYJC/Guess that's not right. It's a SCSI-to-SATA splitter.
-
@HoteiLife said:
@HoteiLife said:
Guess you can also get splitters to share single SATA ports among multiple drives. Again at what point the SATA channel becomes saturated is a consideration. Here's a 1-to-4 splitter:
http://smile.amazon.com/Cable-Matters-Internal-Mini-SAS-Breakout/dp/B012BPLYJC/Guess that's not right. It's a SCSI-to-SATA splitter.
Correct, SATA can't be split/daisychained. You need 1 port on the controller per drive. The cables you linked to is SAS to SATA. I think Intel makes different 6 port SAS cards which you could use these cables with to get 4 x 6 = 24 SATA channels on a single board. These boards are ~$250. These are probably ideal for RAID arrangements. Otherwise, the 4-channel SATA controllers I mention in my first post would be cheaper.
-
@in-cred-u-lous said:
Correct, SATA can't be split/daisychained. You need 1 port on the controller per drive. The cables you linked to is SAS to SATA. I think Intel makes different 6 port SAS cards which you could use these cables with to get 4 x 6 = 24 SATA channels on a single board. These boards are ~$250. These are probably ideal for RAID arrangements. Otherwise, the 4-channel SATA controllers I mention in my first post would be cheaper.
I've seen port multipliers for SATA. This thread discusses using a 1-to-5 multiplier & says you can go 4 multipliers deep:
http://raspberrypi.stackexchange.com/questions/360/can-i-attach-a-sata-controllerThere's one for sale there:
http://www.newegg.com/Product/Product.aspx?Item=9SIA1JM2FM0733But I'm inclined to go USB. It's designed to connect large numbers of devices.
-
@HoteiLife said:
I've seen port multipliers for SATA.
But I'm inclined to go USB. It's designed to connect large numbers of devices.For a stable system you want to reduce potential points of failure, meaning eliminating things such as adapters and external multiplier cards IMO.
-
At $1-$2/T/month there's a lot of risk involved trying to be a host. First of all it takes a lot of time for ROI if you compare to mining the right crypto for example. If after 6 months one of your big disks fails it's gonna take you even longer to get ROI, let alone make some money. Initial cost of hardware is pretty high as well if you want a decent rack/server and compete like a boss. With that comes a higher electricity cost and you're not even guaranteed that all your space will be rent out through smart contracts. Most likely you'll be happy that 1/3 of your space is used all the time the first year since release. About 3,8% of your potential profit goes to Sia as well? Seems fair and I have no problem with it, but it's another little chunk that gets eaten out of your potential profit.
I'm a system specialist and obviously I'm interested in building a storage server as well using SIA because I like the concept, but man the ROI seems long and the returns minimal. :(
-
@SteamboatG Until the network grows, I think you will be competitive at $3 - $4/TB/mo. If you want real profit, you would want to minimize risk through averaging by scaling. But yeah, still high risk as no-one really knows how big Sia will grow.
