I use ZFS in my homelab, and I wish these 5 myths would go away

I use ZFS in my homelab, and I wish these 5 myths would go away

If you have ever considered using ZFS for your home server, you have probably encountered some rather intimidating advice. Apparently, you need enterprise grade hardware, enormous amounts of RAM, and a server motherboard before you should even think about creating your first pool. Reading some discussions, you would think ZFS was designed exclusively for people who have a server rack in their basement. I have been using ZFS in my homelab, and I can understand why people are particularly cautious about storage. After all, nobody wants to lose years of photos, documents, and backups because of a configuration mistake. However, some of the advice surrounding ZFS has been repeated so often that recommendations have gradually turned into supposed requirements. There are legitimate reasons to be careful with ZFS, but I think these five myths have discouraged enough people from trying an otherwise excellent filesystem. ZFS absolutely requires ECC RAM Your regular desktop memory is perfectly capable of running ZFS Credit: Patrick Campanale / How-To Geek One of the first things you will encounter when researching ZFS is the insistence that ECC memory is practically mandatory. Someone asks whether they can turn an old desktop into a NAS, and before long, the discussion turns into recommendations for server motherboards and expensive memory upgrades. The original question about storage somehow became a shopping list for an entirely new computer. ECC stands for Error-Correcting Code, and for once, the name does a pretty good job of explaining what it does. Sometimes, the bits stored in memory can change unexpectedly because of hardware faults or other problems. ECC memory can detect and correct certain memory errors before they affect the data being processed. This is particularly useful for ZFS because one of its biggest advantages is data integrity. ZFS calculates checksums when writing data, allowing it to detect corruption when that data is read back. However, if data becomes corrupted in memory before ZFS calculates its checksum, the filesystem might write that corrupted data along with a perfectly valid checksum. That sounds concerning, and it certainly is. However, there is something important that gets overlooked in these discussions. This problem is hardly exclusive to ZFS. If faulty RAM corrupts a file before ext4 or another filesystem writes it, you can end up with corrupted data there as well. ECC memory reduces that risk, and I would certainly prefer it for a server storing irreplaceable data. However, treating it as a mandatory requirement creates the impression that ZFS is somehow unusually dangerous without it. If you are building a homelab from an old desktop with ordinary RAM, there is no reason to abandon your ZFS plans simply because the machine does not support ECC. You are accepting some additional risk, as you would with any filesystem running on non-ECC memory. Personally, I would be more concerned about someone running ZFS with ECC memory but without a proper backup strategy than someone using ordinary RAM and maintaining reliable backups. You need 1GB of RAM for every TB of storage Your storage capacity does not determine how much memory ZFS needs This is probably my favorite ZFS myth because it makes expanding your storage sound unnecessarily expensive. Want to add another 8TB hard drive to your home server? According to this rule, you should probably order another 8GB of RAM while you are at it. The famous 1GB-per-TB recommendation has been repeated so many times that people sometimes treat it as a requirement of ZFS. In reality, ZFS has no fixed relationship between storage capacity and memory requirements; how much RAM you need depends almost entirely on your workload. This recommendation is often associated with deduplication, a feature that allows ZFS to identify identical blocks of data and store them only once. It sounds particularly attractive when you have several copies of the same files, but keeping track of all those blocks can require considerable amounts of memory. The requirements depend on the number of unique blocks and the deduplication configuration, rather than simply the total capacity of your drives, but deduplication has also improved, and it is not the same as it used to be. Much of the advice you find online comes from older documentation written for earlier versions of ZFS. Since OpenZFS 2.3, Fast Dedup adds a quota that limits how large the deduplication table can grow, along with a log that batches updates to reduce random writes. It makes deduplication more predictable, but it does not eliminate the memory cost, because the table still needs to fit in ARC or be supported by fast dedup vdev. Fortunately, most homelabs have little reason to enable deduplication in the first place. Without deduplication, ZFS can work on systems with relatively modest memory. For a small dedicated home server, I would generally aim for 4GB to 8GB, although simpler installations can work with less, I wish I could say the same about TrueNAS. It has its own ideas about how much RAM a home server should have and requires at least 8GB for basic operation, and you may need more depending on your workload. Part of the confusion also comes from the Adaptive Replacement Cache, or ARC. ZFS uses available RAM to cache frequently accessed data, which can make subsequent reads considerably faster. If you check memory usage on a ZFS server, you might find that the ARC has occupied a surprisingly large portion of it. That is generally a good thing, unlike Chrome eating up your RAM. ZFS is putting otherwise available memory to use, and the ARC can shrink when applications need more RAM. The effectiveness of that memory reclamation depends on your operating system and configuration, but a large ARC is not automatically evidence that your server is running out of memory. Of course, if your Proxmox server runs virtual machines, databases, and several containers, you will need enough memory for those applications. However, adding another hard drive does not suddenly create a corresponding RAM requirement. Compression makes ZFS slower Sometimes, asking your CPU to do more work actually makes storage faster Compression sounds like something that should come with a performance penalty. After all, the CPU has to compress data before writing it and decompress it when reading it. Surely, skipping these extra steps would make everything faster? Surprisingly, that is often not how it works. Consider a home server with a few mechanical hard drives. Your CPU can process data considerably faster than those drives can write it. If ZFS manages to compress 100MB of data down to 60MB, your disks have 40MB less data to write. When reading that data back, ZFS also has less information to retrieve from storage. The CPU performs some additional work, but the time saved on disk I/O can more than compensate for it. This is particularly true with lz4, a lightweight compression algorithm designed for speed. On modern processors, its overhead is generally small enough that enabling compression can improve performance for many workloads. There is another interesting detail. Suppose you are storing movies, JPEG images, or ZIP archives that have already been compressed. ZFS can recognize when compressing a block does not produce sufficient savings and store that block uncompressed instead. It still spends some CPU time attempting compression, but it avoids making the stored data larger. This is why I generally consider compression a sensible setting for homelab storage unless you are working with an especially CPU-constrained machine or an unusual workload. You only need to scrub a pool when something goes wrong The whole point of scrubbing is to find problems before they become serious One thing I have learned from running a homelab is that often storage problems arrive at the worst possible time. A hard drive might appear to work perfectly well, only for you to discover that a file you have not accessed in months has become unreadable. That is precisely why ZFS has scrubbing, and why I think treating it as an emergency troubleshooting tool misses much of its purpose. Normally, ZFS verifies a block’s checksum when that block is read. If the checksum does not match, it knows something has gone wrong. However, consider a photo archive that you copied to your server two years ago and have barely touched since. If a portion of that data becomes corrupted on disk, you might have no idea until the next time you try to open it. Unfortunately, that might also be the moment you discover that another disk has failed or that your backup does not contain the file you need. A scrub addresses this problem by systematically reading the allocated data in the pool and verifying its checksums, including blocks that ordinary workloads might not access for months or years. If ZFS discovers corrupted data and your pool has sufficient redundancy, such as a healthy mirror or RAID-Z configuration, it can use another copy or parity information to reconstruct the correct data and repair the damaged copy. Of course, if no valid copy is available, ZFS can identify and report the corruption, but it cannot magically recreate the missing information. Checksums are useful, but they are not a substitute for redundancy or backups. I like to think of scrubbing as routine maintenance, much like checking whether your backup system actually works before you need to restore anything. For many homelabs, running a scrub approximately once a month is a reasonable starting point. The appropriate frequency depends on the size of your pool, its workload, and how long a scrub takes to complete. Hardware RAID works just as well underneath ZFS Letting ZFS manage the individual disks makes recovery much easier If you have ever considered repurposing an old enterprise server for your homelab, you might have come across machines with dedicated hardware RAID controllers. They were expensive pieces of equipment when new, and the idea of putting one to use with ZFS sounds perfectly reasonable. After all, the RAID controller is already handling redundancy, so why should ZFS care about what happens underneath? Quite a lot, actually. One of the biggest advantages of ZFS is that it manages the filesystem and storage redundancy together. When ZFS encounters corrupted data in a properly configured mirror or RAID-Z pool, it can identify the affected block and use another valid copy or parity information to recover it. A hardware RAID controller can interfere with this process because it generally presents an entire array to the operating system as a single logical disk. ZFS might have no idea how many physical drives exist underneath that volume. Imagine one disk in your RAID array starts returning corrupted data. The controller might handle the issue internally, but ZFS has limited visibility into what is happening. If corrupted data eventually reaches ZFS and fails checksum verification, it may be unable to repair the block because the redundancy is controlled by the RAID controller rather than ZFS itself. There is also the issue of hardware failure, which I think is particularly relevant when dealing with used enterprise equipment. Suppose you build your storage setup around a hardware RAID controller that is already ten years old. If that controller dies, you may need a compatible replacement to access the array. Finding one could become an unpleasant little project of its own, especially if you are dealing with proprietary RAID metadata. For ZFS, I prefer giving the filesystem direct access to the individual drives. An ordinary SATA controller can accomplish this, while servers with SAS backplanes commonly use an HBA operating in IT mode. This passes the disks through to the operating system, allowing ZFS to manage redundancy and monitor errors more effectively. It also makes the pool less dependent on a particular RAID controller. You can generally move the disks to another compatible system and import the pool, provided the new system can access the drives correctly. This does not mean that ZFS will immediately corrupt your data if you create a pool on top of a hardware RAID array. It can work, but you lose some of the visibility and recovery capabilities that made ZFS attractive in the first place. ZFS is more approachable than its reputation suggests ZFS has a reputation for being complicated, and some of that reputation is deserved. You do need to think carefully about pool layouts, redundancy, backups, and the consequences of a disk failure. Certain decisions can be difficult to reverse once you have filled a pool with data. However, I think the community sometimes makes ZFS sound far more demanding than it needs to be. A modest home server with ordinary RAM and a few hard drives can be a perfectly capable ZFS machine. If anything, I would rather see newcomers spend more time understanding how snapshots, scrubs, and backups work than worrying about whether their hardware satisfies an arbitrary memory requirement they encountered in a forum discussion. Those are the things that will actually matter when something eventually goes wrong.

Original Source

Read the full article at Howtogeek →

KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.