Back to the main page

RedHat Logical Volume Manager (LV mirror example)

Volume management creates a layer of abstraction over physical storage, so logical storage volumes can be created. To use the disk for an LVM logical volume it must be initialized as a physical volume (PV). Initializing a block device as a physical volume places a label near the start of the device.

Physical Volumes

Let's create physical volumes on disks sdc and sdd (command pvcreate):
# pvcreate -v /dev/sdc /dev/sdd
    Wiping cache of LVM-capable devices
    Set up physical volume for "/dev/sdc" with 286739329 available sectors
    Zeroing start of device /dev/sdc
  Writing physical volume data to disk "/dev/sdc"
  Physical volume "/dev/sdc" successfully created
    Wiping cache of LVM-capable devices
    Set up physical volume for "/dev/sdd" with 286739329 available sectors
    Zeroing start of device /dev/sdd
  Writing physical volume data to disk "/dev/sdd"
  Physical volume "/dev/sdd" successfully created

See the physical volume information with command pvs
# pvs
  PV         VG   Fmt  Attr PSize   PFree
  /dev/sdc        lvm2 a--  136.73G 136.73G
  /dev/sdd        lvm2 a--  136.73G 136.73G

Show more info with command pvdisplay
# pvdisplay
  "/dev/sdc" is a new physical volume of "136.73 GB"
  --- NEW Physical volume ---
  PV Name               /dev/sdc
  VG Name
  PV Size               136.73 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               SaSfXm-996Y-YToO-Ogfm-EzFM-Nmm3-Sfh6Pu

  "/dev/sdd" is a new physical volume of "136.73 GB"
  --- NEW Physical volume ---
  PV Name               /dev/sdd
  VG Name
  PV Size               136.73 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               YmuTUH-5VHn-bGAe-dIAY-5G2m-wJJE-XHaG7L

Scan all supported LVM block devices:
# pvscan
  PV /dev/sdc                      lvm2 [136.73 GB]
  PV /dev/sdd                      lvm2 [136.73 GB]
  Total: 2 [273.46 GB] / in use: 0 [0   ] / in no VG: 2 [273.46 GB]

If you want to remove disk from the physical volume (zeros LVM metadata on empty physical volume)
# pvremove /dev/sdd
  Labels on physical volume "/dev/sdd" successfully wiped

Reverse previous action of removing PV and continue with creating Volume Group.

Volume Group

# vgcreate volgroup1 /dev/sdc /dev/sdd
  Volume group "volgroup1" successfully created

Display info about Volume Group
#vgs 
  VG        #PV #LV #SN Attr   VSize   VFree
  volgroup1   2   0   0 wz--n- 273.45G 273.45G

# vgdisplay
  --- Volume group ---
  VG Name               volgroup1
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               273.45 GB
  PE Size               4.00 MB
  Total PE              70004
  Alloc PE / Size       0 / 0
  Free  PE / Size       70004 / 273.45 GB
  VG UUID               oOdAA7-w1re-x6BD-i2vB-keeL-aPbz-UlubIa

If you want to remove physical volume from volume group ...
# vgreduce  volgroup1 /dev/sdd 
  Removed "/dev/sdd" from volume group "volgroup1"

# vgs
  VG        #PV #LV #SN Attr   VSize   VFree
  volgroup1   1   0   0 wz--n- 136.73G 136.73G

To add physical volume to volume group...
# vgextend  volgroup1 /dev/sdd
  Volume group "volgroup1" successfully extended

#pvs
  PV         VG        Fmt  Attr PSize   PFree
  /dev/sdc   volgroup1 lvm2 a--  136.73G 136.73G
  /dev/sdd   volgroup1 lvm2 a--  136.73G 136.73G

To scan for block devices that may be used as physical volumes:
#lvmdiskscan
  /dev/ramdisk [       16.00 MB]
  /dev/ram     [       16.00 MB]
  /dev/root    [        4.40 GB]
  /dev/ram2    [       16.00 MB]
  /dev/sda2    [        4.40 GB]
  /dev/ram3    [       16.00 MB]
  /dev/sda3    [        4.40 GB]
  /dev/ram4    [       16.00 MB]
  /dev/sda4    [       55.17 GB]
  /dev/ram5    [       16.00 MB]
  /dev/ram6    [       16.00 MB]
  /dev/ram7    [       16.00 MB]
  /dev/ram8    [       16.00 MB]
  /dev/ram9    [       16.00 MB]
  /dev/ram10   [       16.00 MB]
  /dev/ram11   [       16.00 MB]
  /dev/ram12   [       16.00 MB]
  /dev/ram13   [       16.00 MB]
  /dev/ram14   [       16.00 MB]
  /dev/ram15   [       16.00 MB]
  /dev/sdb1    [       68.36 GB]
  /dev/sdc     [      136.73 GB] LVM physical volume
  /dev/sdd     [      136.73 GB] LVM physical volume
  3 disks
  18 partitions
  2 LVM physical volume whole disks
  0 LVM physical volumes

(Mirrored) Logical Volume

To create mirrored logical volumes...
Use m1 to create one mirror, which creates 2 copies of filesystem. The 2 legs of mirror are on disks sdc and sdd. There is no third disk in volume group so mirror log is placed on one of 2 disks.
#lvcreate -L 10G -m1 -n mirr_log_vol_1 volgroup1 /dev/sdc /dev/sdd
  Logical volume "mirr_log_vol_1" created

#lvs
  LV             VG        Attr   LSize  Origin Snap%  Move Log                 Copy%  Convert
  mirr_log_vol_1 volgroup1 mwi-a- 10.00G                    mirr_log_vol_1_mlog   1.17

#lvdisplay
  --- Logical volume ---
  LV Name                /dev/volgroup1/mirr_log_vol_1
  VG Name                volgroup1
  LV UUID                nEtEhz-1qyR-rCHT-pYru-4YdP-N8vd-D5iibn
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                10.00 GB
  Current LE             2560
  Mirrored volumes       2
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:3

#lvscan
  ACTIVE       '/dev/volgroup1/mirr_log_vol_1' [10.00 GB] inherit

Create ext3 file system and mount it:
# mkfs.ext3 /dev/volgroup1/mirr_log_vol_1
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1310720 inodes, 2621440 blocks
131072 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Have line "/dev/volgroup1/mirr_log_vol_1 /lv-mirror ext3 defaults 1 2" in the file /etc/fstab and mount /lv-mirror
#df -h /lv-mirror
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/volgroup1-mirr_log_vol_1
                      9.9G  151M  9.2G   2% /lv-mirror

Let's create second LV and mount it, also see "copy%" tells once mirror is fully ready:
#lvs
  LV             VG        Attr   LSize  Origin Snap%  Move Log                 Copy%  Convert
  mirr_log_vol_1 volgroup1 mwi-ao 10.00G                    mirr_log_vol_1_mlog 100.00
  mirr_log_vol_2 volgroup1 mwi-a-  5.00G                    mirr_log_vol_2_mlog   1.95

Let's now for example physically remove a disk from the server. The syslog messages are:
kernel: scsi 0:0:3:0: rejecting I/O to dead device
kernel: device-mapper: raid1: A read failure occurred on a mirror device.
lvm[5624]: Secondary mirror device 253:2 read failed.
kernel: device-mapper: raid1: Trying different device.
lvm[5624]: volgroup1-mirr_log_vol_1 is now in-sync.
lvm[5624]: Device failure in volgroup1-mirr_log_vol_2.
lvm[5624]: Couldn't find device with uuid vTabhs-mrX7-bZhG-E5yr-DNZ0-ffhL-n2zRAE.
smartd[3459]: Device: /dev/sdd, No such device, open() failed

And file systems are still okay.
#lvscan -v
    Finding all logical volumes
    Wiping cache of LVM-capable devices
  Couldn't find device with uuid vTabhs-mrX7-bZhG-E5yr-DNZ0-ffhL-n2zRAE.
    There are 1 physical volumes missing.
  ACTIVE            '/dev/volgroup1/mirr_log_vol_1' [10.00 GB] inherit
  ACTIVE            '/dev/volgroup1/mirr_log_vol_2' [5.00 GB] inherit

#pvs -v
    Scanning for physical volume names
    Wiping cache of LVM-capable devices
  Couldn't find device with uuid vTabhs-mrX7-bZhG-E5yr-DNZ0-ffhL-n2zRAE.
    There are 1 physical volumes missing.
    There are 1 physical volumes missing.
  PV             VG        Fmt  Attr PSize   PFree   DevSize PV UUID
  /dev/sdc       volgroup1 lvm2 a--  136.73G 121.73G 136.73G SaSfXm-996Y-YToO-Ogfm-EzFM-Nmm3-Sfh6Pu
  unknown device volgroup1 lvm2 a-m  136.73G 136.73G      0  vTabhs-mrX7-bZhG-E5yr-DNZ0-ffhL-n2zRAE


Back to the main page