Unraid® is an embedded operating system that is designed to provide youwith the ultimate control over your hardware. In addition to performingthe duties of a robust NAS (network-attached storage), Unraid is alsocapable of acting as an application server and virtual machine host.Unraid installs to and boots from a USB flash device and loads into aroot RAM file system. By using a modern Linux kernel with up-to-datehardware drivers, Unraid can operate on nearly any 64 bit system(x86_64) with minimal consumption of system memory. All configurationdata relating to the operating system is stored on the flash device andloaded at the same time as the operating system itself. Management ofyour Unraid system is accomplished through an intuitive web interfacethat offers basic controls for common tasks as well as advanced tuningoptions for the more savvy user. Unraid automatically chooses defaultsettings that should work for most people's needs, but also allows youto tweak settings to your liking. This makes Unraid intuitive where youwant it, and tunable where you need it. By combining the benefits ofboth hardware and software agnosticism into a single OS, Unraid providesa wide variety of ways to store, protect, serve, and play the contentyou download or create.
The capabilities of Unraid are separated into three core parts:software-defined NAS, application server, and localized virtualization
Network Attached Storage
At its core, Unraid is a hardware-agnostic solution that can turn almostany 64-bit capable system into a NAS. Unraid can manage an array ofdrives (connected via IDE, SATA, or SAS) that vary in size, speed,brand, and filesystem. In addition, by eliminating the use oftraditional RAID-based technologies, we can scale on-demand by addingmore drives and without needing to rebalance existing data. Unraid'sNAS functionality consists of a parity-protected array, user shares, andan optional cache pool.
Parity-Protected Array
The primary purpose of an Unraid array is to manage and protect the dataof any group of drives (JBOD) by adding a dedicated parity drive. Aparity drive provides a way to reconstruct all of the data from a faileddrive onto a replacement. Amazing as it seems, a single parity drive canadd protection for all of the others! In Unraid parity is alwaysmaintained in real-time and a bit more detail on the access pattern tothe drives is described under Manual/StorageManagement with their performanceimplications.
The contents of a hard drive can be thought of as a very long stream ofbits, each of which can only be a zero or a one. If you sum the valuesof the nth bit on every drive and determine whether that sum is even orodd, then you can force the corresponding nth parity bit to also be evenor odd (zero or one). If a data drive fails, that parity information cannow be used to deduce the exact bit values of the failed drive, andperfectly rebuild it on a replacement drive. Here's an example:
In the picture above, we have three drives and each has a stream of bitsthat vary in count based on the device size. By themselves, thesedevices are unprotected and should any of them fail, data will be lost.To protect ourselves from failure, we must add a fourth disk to serve asparity. The parity disk must be of equal or greater size than thelargest data disk. To calculate the value of each bit on the paritydisk, we only need to know the sum total for each column. If the sum ofa column is an even number, the parity bit should be a 0. If the sum ofa column is an odd number, the parity bit should be a 1. Here's thesame image as before, but with parity calculated per frame:
Now let's pretend that drive 2 in our example has suffered a failureand a new drive has been purchased to replace it:
To rebuild the data on the newly replaced disk, we use the same methodas before, but instead of solving for the parity bit, we solve for themissing bit. For column 1, the sum would be 0, an even number, so themissing bit must be a 0 as well. For column 6, the sum would be 1, anodd number, so therefore the missing bit must also be a 1.
The ability to rebuild a disk using parity provides protection from dataloss. Parity protection also provides fault-tolerance by allowing fullusage of the system while keeping all data accessible, even when a drivehas failed.
Unlike most RAID systems, Unraid saves data to individual drives. Tosimplify manageability, users can create shares that allow fileswritten to them to be spread across multiple drives. Each share can bethought of as a top-level folder on a drive. When browsing through ashare, all data from all drives that participate in that share will bedisplayed together. Users do not need to know which disk a file is on inorder to access it under a share. Shares can be tuned to include/excludespecific disks and to use various methods for determining how files areallocated across those disks. In addition to controlling how data isdistributed across drives, users can also control what network protocolsthe share is visible through as well as define user-level securitypolicy. When accessing your Unraid server over a network protocol, allshares exported through that protocol will be visible, but you cantoggle protocols for both individual shares as well as at a globalsetting level. Should you have private data on your system that you wishto protect from anonymous access, user accounts can be created andpolicies defined to limit access to only trusted individuals.
Cache
The cache-drive feature of Unraid provides faster data capture.Generally speaking, by using a cache alongside an array of three or moredrives, you can achieve up to 3x write performance. When data is writtento a user share that has been configured to use the cache drive, all ofthat data is initially written directly to the dedicated cache drive.Because this drive is not a part of the array, the write speed isunimpeded by parity updates. Then an Unraid process called "the mover"copies the data from the cache to the array at a time and frequency ofyour choosing (typically in the middle of the night). Once the movercompletes, the space consumed previously on the cache drive is freed up.
With a single cache drive, data captured there is at risk, as a paritydrive only protects the array, not the cache. However, you can build acache with multiple drives both to increase your cache capacity as wellas to add protection for that data. The grouping of multiple drives in acache is referred to as building a cache pool. The Unraid cache poolis created through a unique twist on traditional RAID 1, using a BTRFSfeature that provides both the data redundancy of RAID 1 plus thecapacity expansion of RAID 0.
Application Server
Traditional NAS solutions to application support come with three primarylimitations:
- They cannot support applications written for other operatingsystems.
- They can be cumbersome to install and even more difficult to remove.
- They don't always "play nice" with other applications in the sameOS.
Docker addresses these problems in a number of key ways:
- It allows for the use of any Linux operating system to empower agiven application (no longer limited by the operating system of thehost itself).
- It removes the "installation" process that applications have to gothrough by providing pre-installed images that ensure a consistentrun-time experience for the user and making them easier to removewhen the user is done with them.
- It enables applications that would normally have issues withcoexistence to live in harmony in the same operating environment.
Docker is made up of three primary components: the Engine, the Hub, andContainers.
The Engine
The Docker Engine represents the management component that is built intoUnraid 6. Using the engine, we can control application access to vitalsystem resources, interact with the Docker Hub, and isolate applicationsfrom conflicts.
From a storage perspective, the engine leverages the copy-on-writecapabilities of the BTRFS filesystem combined with Docker imagesprovided through the hub. The images are essentially tar files with ahierarchy so that other images which depend upon a common layer don'tneed to replicate storage for the layer they share. The shared layersare put in a read-only state, while changes made to them are reflectedonly in the instance for the application that changed it. In simplerterms, this means that applications can be efficient in their use ofboth system performance and storage capacity.
This differs from virtual machines where the entirety of the guestoperating system is emulated. The overhead required to support a virtualmachine is therefore much higher than the equivalent overhead for acontainer. Docker containers that feature common libraries and binariesdo not replicate those shared resources and instead, leverage theefficiency of Linux and a copy-on-write filesystem to minimize theoverhead.
The Hub
One of biggest advantages Docker provides over both traditional Linuxcontainers (LXCs) and virtual machines (VMs) is in its applicationrepository: the Docker Hub. Many traditional Linux operating systemsnowadays come with a component in their framework known as a packagemanager. The job of the package manager is to let people easily installapplications written for a particular operating system from catalogsthat are known as repositories. While package managers do their jobfairly well, they come with all the limitations mentioned earlier. Linuxcontainers and virtual machines, while competent at providing a way tocontrol resources allocated to an application, still rely on traditionalpackage managers for software retrieval and installation into theirrun-time environments.
In contrast, the Docker Hub provides all the benefits without thelimitations of a traditional package manager. Using the Docker engine,pre-built applications can be downloaded automatically and, thanks tothe copy-on-write benefits we've already covered, the only data that isactually downloaded is data not already present on your system. The hubcontains over 14,000 Dockerized apps, so finding what you're looking forshouldn't be a problem. In addition, thanks to some of our loyalcommunity members, users can quickly add many of the most popularcontainers through the use of templates in Unraid 6. These forum membershave taken it upon themselves to build and maintain these templates andthe list of available templates continues to grow.
The Docker Hub can be accessed at http://registry.hub.docker.com.
Containers
The cornerstone of Docker is in its ability to use Linux control groups,namespace isolation, and images to create isolated executionenvironments in the form of Docker containers. Docker controls theresources allocated to the Containers and isolates them from conflictingwith other applications on the same system. This provides all thebenefits of traditional virtual machines, but with none of the overheadassociated with emulating hardware, making containers ridiculouslyefficient and in some studies, barely distinguishable from bare-metalequivalents.
Docker works by allowing applications access to the system resources ofthe host operating system, such as CPU, memory, disk, and network, butisolates them into their own run-time environments. Unlike virtualmachines, containers do not require hardware emulation, which eliminatesoverhead, hardware requirements, and provides near bare-metalperformance.
Virtualization Host
Virtualization technology has advanced greatly since it was firstintroduced and provides a wealth of benefits to users. By supporting theuse of virtual machines on Unraid 6, we can run an even wider array ofapplications in isolated environments. While Docker containers are thepreferred method for running Linux-based headless applications, virtualmachines offer these unique benefits:
- Run non-Linux operating systems (e.g. Windows).
- Support drivers for physical devices independently of Unraid OS.
- Customize and tune the guest operating systems.
Unraid Server OS is designed to run as a virtualization host, leveraginga hypervisor to partition resources to virtualized guests in a secureand isolated manner. To simplify, virtual machines can be assigned awider array of resources than Docker containers but still offer the samebenefits of isolated access to those resources. This enables Unraidservers to handle a variety of other tasks, more than justnetwork-attached storage.
Assignable Devices
Our implementation of KVM includes modern versions of QEMU, libvirt,VFIO*, VirtIO, and VirtFS. We also support Open Virtual MachineFirmware (OVMF) which enables UEFI support for virtual machines (addingSecureBoot support as well as simplified GPU pass through support). Thisallows for a wide array of resources to be assigned to virtual machinesranging from the basics (storage, compute, network, and memory) to theadvanced (full PCI / USB devices). We can emulate multiple machine types(i440fx and Q35), support CPU pinning, optimize for SSDs, and much more.Best of all, these virtualization technologies prevent their use fromimpacting the reliability of the host operating system.
Simplified Management
Management of your Unraid system is accomplished through an intuitiveweb interface that offers basic controls for common tasks as well asadvanced tuning options for the more savvy user. Unraid automaticallychooses default settings that should work for most people's needs, butalso allows you to tweak settings to your liking. This makes Unraidintuitive where you want it, and tunable where you need it.
- Dashboard View. With indicators for disk health, temperatures,resource utilization, and application states, the dashboard providesa 50,000 foot view of what's happening on your system.
- Array Operation. Assign devices for use in either the array orcache, spin up and down individual disks, start and stop the array,and even perform an on-the-fly parity check, all from a single page.
- Share Management. Setting up shares on Unraid is easy. Give theshare a name, optionally apply policies to access and distributioncontrols, and click create!
- Disk Tuning. Control how and when devices spin down, the defaultfile system format, and other advanced settings.
- Network Controls. Enable NIC bonding and bridging, set timeservers, and more.
- APC UPS Safe Shutdown. When connected to an APC UPS, Unraid cansafely shut down the system in the event of a power loss.
- System Notifications. Unraid can alert you to important eventshappening on your system through the web management console as wellas e-mail and other notification systems.
- Task Scheduler. Choose if and when you want to have an automaticparity check occur as well how often the mover script shouldtransfer files from the cache to the array.
- Docker Containers. Manage application controls from a singlepane of glass. Add applications with minimal effort usingcommunity-provided templates.
- Virtual Machines. Choose between pre-created virtual machineimages or create your own custom VM from scratch.
