Anyone with a Freenas server have advice?

Submitted by teldar on

So I've got my computer built and have installed freenas but I don't have enough hard drives yet, I think, so I'm not ready to build my vdev. I though vdev expansion was here in 11.2, but it doesn't look like it yet. 

 

Anyone have any got anything to tell me to avoid doing? I already found one thing. I bought a small SSD for my system drive and after and now I'm not sure I shouldn't have just bought a pair of 32Gb flash drives.

 

 

karpodiem

August 16th, 2018 at 3:15 PM ^

Ha! I'm surprised you're asking this on MGoBoard but you've found the right guy to help you. I was very close to doing a FreeNAS build and then I thought - "wait, why don't I use Ubuntu  + ZFS on Linux?", which is very well supported (tons of questions on stack overflow) and is Linux based (again, better supported than FreeBSD). So that's what I did - I installed Ubuntu 18.04 on a Lenovo TS440 server and have six out of eight drive bays full.

Shoot me an e-mail - karpodiem at gmail dot com.

BlueMars24

August 16th, 2018 at 3:25 PM ^

I know this doesn't answer your question, but I looked around at building my own NAS and decided just to buy a Synology box. It's awesome. I've had it for about 6 months and don't regret the decision. 

UMDWolve

August 16th, 2018 at 4:09 PM ^

My best advice for the z file system is to use 2^n + P disks where P is the level of parity raid.  For example, a raidz2 array could have 6 disks (4+2), 10 disks (8+2) or maybe even 18 disks (16+2).

 

Also touching on what karpodiem said earlier - zfs on linux runs fairly well, I've been using it on an 18.04 system (not vanilla ubuntu) for a while with some cheap Dell SAS HBAs.  Running ZFS on linux lets you run your other software on the same box, no VMs or additional physical servers needed.

 

EDIT: the version of zfs on *buntu repos is 0.7.5.  If you want the latest version, you'll have to install from a third party repo or from source.  Changelogs are at https://zfsonlinux.org/ if you want to see if any of the fixes apply to your use cases.

karpodiem

August 17th, 2018 at 9:22 AM ^

RAIDZ2 is a pain is more or less impossible to expand, when that time comes. Based on the striping/parity of the file system, you can't swap a drive out and expand the pool.

I very much recommend a RAID1 ZFS configuration. It's mirrored drives of the same capacity. 2TB drives are reasonably cheap these days. 4TB is obviously better. I have a thing for capacity in the power of 2.

With a ZFS box, you want a computer case that has 1) at least four 3.5 drive bays 2) preferable six or eight drive 3.5 bays. I ended up buying a Lenovo TS440 on sale years ago and it's one of the best decisions I've ever made; it's a workhorse. It has six SATA ports - I have a cheap SSD for the server OS, and four SATA ports with 4TB drives.

I also have 2x2TB drives in drive bays 5 and 6. I found out you can flash the firmware of a specific LSI raid card which will disable the RAID on the card itself (you don't need RAID on the hardware level, because ZFS is doing it for you) and it will pass the disks to the operating system as if they were connected via SATA - https://www.ebay.com/itm/LSI-SAS-9211-8i-8-port-6Gb-s-PCI-E-Internal-HBA-Both-Brackets-IT-MODE/152937435505?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2060353.m1438.l2649

Anyways, let me know if anyone else has questions - I could talk about this all day. It took me two or years to finally do a build and pull the trigger after doing all this research, but I'm glad I waited and did Ubuntu + ZFS on Linux (ZoL).

bronxblue

August 16th, 2018 at 9:06 PM ^

I've never enjoyed working with FreeBSD as an OS, so I was always gun-shy about playing around with FreeNAS.  I haven't set up a NAS, but I'd second the Ubuntu + ZFS design.  You'll have access to more resources, with Ubuntu plus ZFS is well-supported by it.

Good luck.

teldar

August 17th, 2018 at 6:39 PM ^

I think I'm convinced. Karpodiem is going to email me back this coming week. I've already worked out that limitations on freenas are limitations of zfs. My only real questions are about system failures and upgrades.

 

What happens if your system drive fails? on freenas everything important is stored on the data drives apparently. I don't believe that's the case with Linux. How do you tell Linux to use a ZFS array that's already there? And how do you update to a newer version of ZFS? Can you update to a newer version of ZFS if it's just run as a storage file system on Linux?

 

Also, can you share Linux on a primarily Windows net worth? Do they support the same protocol for sharing?

Qseverus

August 17th, 2018 at 2:16 PM ^

I’ve been running a FreeNAS server for about 4 years.  I initially used USB drives as my boot drive. I say “drives” because I quickly found out that FreeNAS will happily and repeatedly find something to complain about when using a USB boot drive.

I have had USB failures during boot-up. USB failures during boot scrubs. I have even had FreeNAS complain that a newly partitioned, formatted and otherwise empty USB drive was too full to install the operating system (It wasn't). In short, it didn’t matter what USB drive I tried FreeNAS found a reason to complain. Cheap (PNY, SanDisk) or expensive (Corsair, Mushkin) it didn’t matter.

I finally switched to an old Intel SSD drive I wasn’t using and haven’t had a boot problem since. Save yourself a lot of trouble and expense. Use a decent SSD boot drive.

Qseverus

August 17th, 2018 at 8:24 PM ^

In FreeNAS the operating system resides on its own drive be it an SSD or USB drive. If push comes to shove you can easily reinstall the OS on that separate drive, reload your configuration and you’re back in business. Your data is untouched. I frequently backup my configuration so I will have it if needed. 

Another reason to use a good SSD boot drive is that FreeNAS can keep previous boot environments. If a new update screws something up you can easily revert to a previous working boot version/environment.

I would also strongly suggest you buy a motherboard that supports IPMI. This will allow you to access your FreeNAS server from another computer. Very handy for accessing the FreeNAS server BIOS, reverting to previous versions, or just keeping tabs on server health without connecting a monitor & keyboard to the server.