Linux 7 Storage Concepts: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
Line 52: Line 52:


  /dev/vdb1                      /support-nfs-storage  xfs defaults 0 0
  /dev/vdb1                      /support-nfs-storage  xfs defaults 0 0
'''Quota per Directory''' https://scriptthe.net/2014/08/06/setting-up-a-hard-quota-with-a-directory-on-xfs/


=<span id='devicemapper'></span>Device Mapper=
=<span id='devicemapper'></span>Device Mapper=

Revision as of 18:04, 18 February 2018

Internal

Block

A block is a fixed-size chunk of data. The size is determined by the kernel, and depends on the system's architecture and the filesystem being used. The block size the kernel uses to access a specific device can be obtained with:

blockdev --getbsz /dev/sda2

Sector

A sector is a small block whose size is usually determined by the underlying hardware.

Block Devices

Unlike a character device, a block device provides random access to fixed-size blocks of data. On Linux, the block devices can be either "mapped", offering access to a logical volume in a volume group (/dev/mapper/VolGroup00-LogVol01), or "static", which is a traditional storage volume (/dev/sdba).

Block Driver

A block driver provides access to block devices - devices that transfer randomly accessible data in fixed-size blocks. These are primarily hard drives. Aside providing user space processes read and writing access to block storage, the block drivers act as conduit between the core memory of the system and secondary storage, therefore they can be seen as making part of the virtual memory subsystem.

Much of the design of the block layer is centered on performance, as the entire system cannot run well if the block I/O subsystem is not well tuned.

Alos see:

NFS Driver

File System

ext4

An ext4 filesystem is created with mkfs.ext4.

Journal Recovery is done with e2fsck in userspace at boot time.

Metadata Error Behavior. When metadata errors are encountered, the behavior is configurable. The default is to continue.

Resize

XFS

An XFS filesystem is created with mkfs.xfs.

Journal Recovery is done in kernel space at mount time. An fsck.xfs command exists, but it does not perform any useful action. If the journal needs repairing, unmount and mount the filesystem

Metadata Error Behavior. When an unrecoverable metadata error is encountered, the filesystem will be shut down.

Resize. The filesystem can be extended while online with xfs_growfs. It cannot be shrunk.

Speculative allocation. XFS uses speculative preallocation to allocate blocks past EOF as files are written. This avoids file fragmentation due to concurrent streaming writes on NFS servers. This temporarily increases the size of the file, but if the preallocated space is not used for five minutes, the preallocation will be discarded. Because of this, fragmentation is rarely a significant issue on XFS filesystems.

Mounting an XFS fileystem in /etc/fstab:

/dev/vdb1                       /support-nfs-storage  xfs defaults 0 0

Quota per Directory https://scriptthe.net/2014/08/06/setting-up-a-hard-quota-with-a-directory-on-xfs/

Device Mapper

Device Mapper is a kernel-based framework for volume management. It has thin provisioning and snapshotting capabilities. Used as one of Docker storage backend options.

Loopback Device

A loopback device is an O/S mechanism that allows exposing a file as a block device. The loopback devices are usually named /dev/loop0, /dev/loop1, etc.

Loopback devices are managed with losetup, which can associate a loopback device with a file:

losetup /dev/loop0 ./lvm0.img

Once setup, the loopback devices should be reported as block devices by lsblk.

Loopback devices are used, among other things, to set up storage with Docker devicemapper driver.

External resources:

Subjects