Yo, this is for all your techno nerds out there in the spring community...
The company I work for are looking at setting up a NAS like system. What we require is:
- Plenty of storage (4TB+)
- RAID redundancy (minimal 1 mirrored pair)
- Expandability to add more hard drives if needed
Optionally it will be able to run a source control server (Git/SVN) and a web server/database
On the server we will be reading/writing large project file (Photoshop design files 500MB+ and the like) regularly, to about 10-20 clients.
It will need to be accessible by Windows, Linux and OS X. We require files to be locked down based on user permissions.
We are looking at purchasing 2 identical systems, one as a complete offset backup which is communicated to with a short point to point wireless network.
We have looked at 2 possible options:
- A dedicated NAS box which costs about $5,000 each NZD (4k USD, 2.7K Euro)
- Build a custom Linux box sourcing parts directly ($1.5k - $3.5K NZD) each
What would you recommended?
- Hardware?
- Software?
- OS?
- Buy prebuilt or make our own?
- RAID?
- SATA?
- Minimum specs?
- Anything Im missing?
(To re-iterate, the most important things is OS X compatibility, plenty of storage, cost effective)
Buy a NAS box or build a Linux Server
Moderator: Moderators
Re: Buy a NAS box or build a Linux Server
i personally would probably go for a single- or dual-core Atom box with Linux software RAID & LVM. You can then setup FTP, Samba, NFS, ... whatever. It is likely that custom built stuff will create more work for the system-/network- admin(s). If you are one of them, then you should consider that too. Maybe it helps in preventing your position from being annihilated, or it will create more work while you already have too much.
Re: Buy a NAS box or build a Linux Server
I'm using NAS:es from QNAP at home, and I'm very satisfied with them :)
Just plug in the disks, create the array and then everything basically works...
Here is my current setup:
TS-859 Pro with 8x2TB SATA disks in a RAID-6 array (10.6TiB)
I basically use this as a movie archive and backup disk
TS-459 Pro+ with 4x1.5TB SATA disks in a RAID-6 array (2.8TiB)
This one I use as a NFS disk for my VMware server
What I really like about the QNAPs is the RAID upgrade features, which means that if I need to upgrade my 4x1.5TB array, I can just replace one disk at the time for a larger one and when all four disks has been replaced (and rebuilt, takes about 12 hours per disk) then I can just extend the disk and have more empty space... without having to shutdown or empty the array
Getting a e-mail or SMS whenever a disk crashes is also a nice feature
I've only tried accessing the files though NFS and Samba but those two works great :)
Also, it has a really nice web front-end for management
Just plug in the disks, create the array and then everything basically works...
Here is my current setup:
TS-859 Pro with 8x2TB SATA disks in a RAID-6 array (10.6TiB)
I basically use this as a movie archive and backup disk
TS-459 Pro+ with 4x1.5TB SATA disks in a RAID-6 array (2.8TiB)
This one I use as a NFS disk for my VMware server
What I really like about the QNAPs is the RAID upgrade features, which means that if I need to upgrade my 4x1.5TB array, I can just replace one disk at the time for a larger one and when all four disks has been replaced (and rebuilt, takes about 12 hours per disk) then I can just extend the disk and have more empty space... without having to shutdown or empty the array
Getting a e-mail or SMS whenever a disk crashes is also a nice feature
I've only tried accessing the files though NFS and Samba but those two works great :)
Also, it has a really nice web front-end for management
Re: Buy a NAS box or build a Linux Server
I had the same question and decided to go with the cheapest possible solution:
Use an existing dead laptop and set some external HDDs. Instead of RAID it just mirrors the main drive to the backup drive nightly. The fact that it's a lappy means I save on power since the little 60W psu is much cheaper to run than the 400W of a regular box.
Use an existing dead laptop and set some external HDDs. Instead of RAID it just mirrors the main drive to the backup drive nightly. The fact that it's a lappy means I save on power since the little 60W psu is much cheaper to run than the 400W of a regular box.
Re: Buy a NAS box or build a Linux Server
As probably expected my answer is "depends" :)
When I read the requirements I see two usage patterns:
* photoshop + revision control system = high volume, high latency OK, low to medium thoughput
* database = low volume, low latency important, low throughput
So it depends on how fast your DB must be, if you have to go with expansive 10k SAS disks or if SATA is ok (DB for "normal" website).
I assume from here on SATA is OK, because "cost effective" is important, and if you need a very fast database then you need professional services anyway, because DB speed depends on a lot of factors.
If you put 8 good 1TB SATA disks in a case you'll get like 400 MB/s sustainable gross read-throughput. Ofc CIFS will eat up 35% of that
The latency is just a little bit higher than the latency of each disk. If you use RAID-5 (+1 hotspare) or RAID-6 you get 6TB capacity.
If 2 ppl want to save their 500MB Photoshop files and 2 ppl open 500MB Photoshop files at the same time they will be happy if you have connected your fileserver with 2x1Gbit. Check your switch(es) for link aggregation. They will probably support only certain NICs. Then ask the vendors of the NAS/server-NICs if they function properly with your switches.
Let's assume 1TB data to read and 1TB data to write and 2Gbit/s: with CIFS you will get like 130 MB/s in both directions... disks should be able to handle this... 4 seconds for everyone to wait... that is a very optimistic maximum, but it can be achieved. What is the requirement?
The question if hardware or software RAID: Linux sw RAID outperforms most RAID controllers for reading as long as the system is not under high CPU load. (If it's a dedicated fileserver that never happens.) Writing seems to go a little faster with hardware RAID controllers. BTW: any h/w RAID controller below 400 U$ is not worth taking into consideration.
But there is an important feature of h/w RAID controllers: hot-swap! If you cannot shutdown the server for maintenance then you have no choice. (Though you can do hot-swap with a controller that supports it, use the disks solo and use s/w RAID. But if you have a h/w solution, it's much more convenient to use it :)
If s/w solution: which Linux distro doesn't matter. The involved components will be just the kernel and Samba (CIFS fileserver, also used by MacOS X). It's important to install a 64 bit kernel, not only for memory, but also for speed.
One thing that has to be taken into consideration is authentication: do you have central user management? These user accounts need to be known to the fileserver. You probably have an Active Directory (MS Windows server?) or a OpenLDAP (Mac OS X or Linux server?). The NAS product you want to buy should support your user database.
CPU: For s/w RAID more than 2 cores are not needed, what is needed is a high frequency (device access is mostly serial - like spring even in a RAID setup.
RAM: more is better - it helps caching. More that 8 GB doesn't make sense in most cases.
Prebuilt/selfmade: If you can afford it, it's good to buy from HP/IBM/Dell/etc because you get server hardware that is well tested and supported in its combination and it should always have redundant power supplies and an out-of-bands management board which is really helpful for administrators.
Example: HP ProLiant DL380
http://h10010.www1.hp.com/wwpc/us/en/sm ... 84082.html
Important thing about RAID controllers: If they have cache memory then it _must_ be flash backed or have a battery backup unit (BBU), or you risk loosing as much data as fits into the cache!
There are also combinations possible: there are NAS-software-distribution available as OSS or for purchase. Their strengths are support, UI, optimizations, their limitations are the same as with NAS boxes: low flexibility and low extensibility.
And last but not least: is there an employee that can do Linux system administration? :)
If flexibility is not needed and the product has the needed features: go for NAS, because someone else already did the work to make it work well. Buy a cheap 2nd server and put the DB on that.
If you have a Linux system administrator and you think there will be movement in the requirements (wishes arise for a new mail server, non-expensive backup solution, groupware, etc) then I'd definitely go for the custom made Linux system (if possible on HP/IBM hardware).
When I read the requirements I see two usage patterns:
* photoshop + revision control system = high volume, high latency OK, low to medium thoughput
* database = low volume, low latency important, low throughput
So it depends on how fast your DB must be, if you have to go with expansive 10k SAS disks or if SATA is ok (DB for "normal" website).
I assume from here on SATA is OK, because "cost effective" is important, and if you need a very fast database then you need professional services anyway, because DB speed depends on a lot of factors.
If you put 8 good 1TB SATA disks in a case you'll get like 400 MB/s sustainable gross read-throughput. Ofc CIFS will eat up 35% of that
The latency is just a little bit higher than the latency of each disk. If you use RAID-5 (+1 hotspare) or RAID-6 you get 6TB capacity.
If 2 ppl want to save their 500MB Photoshop files and 2 ppl open 500MB Photoshop files at the same time they will be happy if you have connected your fileserver with 2x1Gbit. Check your switch(es) for link aggregation. They will probably support only certain NICs. Then ask the vendors of the NAS/server-NICs if they function properly with your switches.
Let's assume 1TB data to read and 1TB data to write and 2Gbit/s: with CIFS you will get like 130 MB/s in both directions... disks should be able to handle this... 4 seconds for everyone to wait... that is a very optimistic maximum, but it can be achieved. What is the requirement?
The question if hardware or software RAID: Linux sw RAID outperforms most RAID controllers for reading as long as the system is not under high CPU load. (If it's a dedicated fileserver that never happens.) Writing seems to go a little faster with hardware RAID controllers. BTW: any h/w RAID controller below 400 U$ is not worth taking into consideration.
But there is an important feature of h/w RAID controllers: hot-swap! If you cannot shutdown the server for maintenance then you have no choice. (Though you can do hot-swap with a controller that supports it, use the disks solo and use s/w RAID. But if you have a h/w solution, it's much more convenient to use it :)
If s/w solution: which Linux distro doesn't matter. The involved components will be just the kernel and Samba (CIFS fileserver, also used by MacOS X). It's important to install a 64 bit kernel, not only for memory, but also for speed.
One thing that has to be taken into consideration is authentication: do you have central user management? These user accounts need to be known to the fileserver. You probably have an Active Directory (MS Windows server?) or a OpenLDAP (Mac OS X or Linux server?). The NAS product you want to buy should support your user database.
CPU: For s/w RAID more than 2 cores are not needed, what is needed is a high frequency (device access is mostly serial - like spring even in a RAID setup.
RAM: more is better - it helps caching. More that 8 GB doesn't make sense in most cases.
Prebuilt/selfmade: If you can afford it, it's good to buy from HP/IBM/Dell/etc because you get server hardware that is well tested and supported in its combination and it should always have redundant power supplies and an out-of-bands management board which is really helpful for administrators.
Example: HP ProLiant DL380
http://h10010.www1.hp.com/wwpc/us/en/sm ... 84082.html
Important thing about RAID controllers: If they have cache memory then it _must_ be flash backed or have a battery backup unit (BBU), or you risk loosing as much data as fits into the cache!
There are also combinations possible: there are NAS-software-distribution available as OSS or for purchase. Their strengths are support, UI, optimizations, their limitations are the same as with NAS boxes: low flexibility and low extensibility.
And last but not least: is there an employee that can do Linux system administration? :)
If flexibility is not needed and the product has the needed features: go for NAS, because someone else already did the work to make it work well. Buy a cheap 2nd server and put the DB on that.
If you have a Linux system administrator and you think there will be movement in the requirements (wishes arise for a new mail server, non-expensive backup solution, groupware, etc) then I'd definitely go for the custom made Linux system (if possible on HP/IBM hardware).
- very_bad_soldier
- Posts: 1397
- Joined: 20 Feb 2007, 01:10
Re: Buy a NAS box or build a Linux Server
I think it will be cheaper for your company in the long run to buy a professional NAS system in the first place instead of hand-crafting your own.
Re: Buy a NAS box or build a Linux Server
I'm not the guy to ask but this is my opinion:
If you have at least 3 geeks capable of establishing and maintaining this type of thing then build it yourself... otherwise you are probably better to go with a dedicated NAS box.
there is no question that the "do it yourself" option would be cheaper, more flexible, easier to upgrade and probably just plain better... but unless you have the guys to manage it your going to be bleeding that added value to training, downtime and other headaches.
If you have at least 3 geeks capable of establishing and maintaining this type of thing then build it yourself... otherwise you are probably better to go with a dedicated NAS box.
there is no question that the "do it yourself" option would be cheaper, more flexible, easier to upgrade and probably just plain better... but unless you have the guys to manage it your going to be bleeding that added value to training, downtime and other headaches.
Re: Buy a NAS box or build a Linux Server
Well... the Atom CPU's in NAS:es don't use much power either... my 4bay QNAP uses around 60w with 4 SATA disksPxtl wrote:The fact that it's a lappy means I save on power since the little 60W psu is much cheaper to run than the 400W of a regular box.
My regular box at home, i5 760 with 3 SATA disks uses around 100w... as long as you use integrated graphics in a regular box it doesn't consume to much power either...
I kinda doubt ec2 can replace a NAS for anything but archiving... anyways... I wouldn't want to have that monthly cost... 4TB costs around 450$ at month, plus network traffic?
Re: Buy a NAS box or build a Linux Server
Well, I'm actually a software dev, and yea I don't really want to get stuck managing the server.hoijui wrote:If you are one of them, then you should consider that too. Maybe it helps in preventing your position from being annihilated, or it will create more work while you already have too much.
That acctually looks alright, not too pricy either. (only $3k here)Zydox wrote:QNAP TS-859 Pro
This sounds a bit haphazard.Pxtl wrote:Use an existing dead laptop and set some external HDDs.
Good info here, thanks.dansan wrote:Stuff..
PS: We don't use centralized user management, and we are mostly Mac based (I have the only Windows PC).
That's a joke:
- Its too expensive
- It would have high latency compared to LAN
- It would have low bandwidth compared to LAN
- It just had major downtime