Back to the main page
Configure disks in Hardware RAID5 on OL

Configure disks in Hardware RAID5 on OL/RHEL

Prep work

What RAID controller card is in the system?

These are useful commands: 
#  lspci -nn | grep -i raid 
23:00.0 RAID bus controller [0104]: LSI Logic / Symbios Logic MegaRAID SAS-3 3108 [Invader] [1000:005d] (rev 02)

#  lshw -c storage 
  *-storage
       description: RAID bus controller
       product: MegaRAID SAS-3 3108 [Invader]
       vendor: LSI Logic / Symbios Logic
       physical id: 0
       bus info: pci@0000:23:00.0
       logical name: scsi0
       capabilities: storage pm pciexpress vpd msi msix bus_master cap_list rom
       configuration: driver=megaraid_sas latency=0

What hard disks are in the system? 

#  lshw -c disk | egrep -e disk -e size\: -e name -e physical -e "logical name" 
  *-disk:0
       physical id: 2.0.0
       logical name: /dev/sda
       size: 557GiB (598GB)
  *-disk:1
       physical id: 2.1.0
       logical name: /dev/sdb
       size: 1116GiB (1198GB)
  *-disk:2
       physical id: 2.3.0
       logical name: /dev/sdc
       size: 1116GiB (1198GB)

#  lsblk 
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 557.9G  0 disk
  |-sda1   8:1    0     2G  0 part /boot
  |-sda2   8:2    0    50G  0 part /
  |-sda3   8:3    0     8G  0 part [SWAP]
  |-sda4   8:4    0     1K  0 part
  |-sda5   8:5    0     8G  0 part /var
sdb      8:16   0   1.1T  0 disk
  |-sdb1   8:17   0   1.1T  0 part
sdc      8:32   0   1.1T  0 disk
  |-sdc1   8:33   0   1.1T  0 part


To see hard disk parameters:
#  hdparm  -I /dev/ 

Also look the file /etc/fstab and use " df -h" to see what disks are used and mounted.

OL kernel drivers for RAID card 

Ref URL: https://hwraid.le-vert.net/wiki/LSIMegaRAIDSAS

The megaraid_sas is part of OL kernel, hence it should be present on the system. 

# modinfo megaraid_sas
filename:       /lib/modules/4.1.12-61.1.9.el6uek.x86_64/kernel/drivers/scsi/megaraid/megaraid_sas.ko
description:    Avago MegaRAID SAS Driver
version:        06.810.09.00-rc1
license:        GPL
...shortened ...

Management and reporting tools

The megacli is a proprietary tool by LSI, download location : https://www.broadcom.com/support/oem/oracle/6gb/sg_x_sas6-r-int-z 
MegaCLI Version
 /opt/MegaRAID/MegaCli/MegaCli64 -v 

HW RAID card info
 /opt/MegaRAID/MegaCli/MegaCli64 -AdpAllInfo -aALL 

Enclosure info
 /opt/MegaRAID/MegaCli/MegaCli64 -EncInfo -a0 

Get number of Physical Drives on Adapter
 /opt/MegaRAID/MegaCli/MegaCli64 -PDGetNum -aALL 

List Hard Disks
 /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL 

List Virtual Drive info
 /opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -Lall -aALL 

Display Configuration
 /opt/MegaRAID/MegaCli/MegaCli64 -CfgDsply -aALL 

Show summary
 /opt/MegaRAID/MegaCli/MegaCli64 -ShowSummary  -a0 

Show physical drives inside virtual drives
 /opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aALL | grep -E "Virtual Drive:|Slot Number:" | xargs | sed -r 's/(Slot Number:)(\s[0-9]+)/\2,/g' | sed 's/(Target Id: .)/Physical Drives ids:/g' | sed 's/Virtual Drive:/\nVirtual Drive:/g' 

Example of output:
 Virtual Drive: 0 Physical Drives ids:  0,
 Virtual Drive: 1 Physical Drives ids:  2,
 Virtual Drive: 3 Physical Drives ids:  1,

Implementation

Command alias

Use command alias, add this line into the file /root/.bash_profile
alias megacli='/opt/MegaRAID/MegaCli/MegaCli64'

Starting point

As per prep work:

Delete legacy logical drive

Disk in slot 1 (logical/virtual disk 3) is configured as RAID0, hence we'll need to delete this logical disk
megacli -CfgLdDel -L3 -a0

Create RAID5

Will use this command to create RAID5 
 megacli -CfgLdAdd -rX[enclosure_id:physical_id,enclosure_id:physical_id] -aN 

  X = 5 (-r5 for RAID5)
  enclosure_id=252
  physical_id=
  N=0 is adapter 0
Examle of creating RAID5 
 megacli -CfgLdAdd -r5[252:1,252:3,252:4,252:5,252:6] -a0 
Adapter 0: Created VD 1
Adapter 0: Configured the Adapter!!
Exit Code: 0x00
Physical disks inside virtual ones are: 
        Virtual Drive: 0 Physical Drives ids:  0,
        Virtual Drive: 1 Physical Drives ids:  1,  3,  4,  5,  6,
        Virtual Drive: 2 Physical Drives ids:  2,

Disk labeling

Make disk label, the new logical volume 1 has size 4.4T, so I use parted command to manipulate it, first I create the label. 
(parted)  mklabel 
New disk label type?  gpt

Disk partitioning

Then make the partition 
(parted)  mkpart primary 
 File system type?  [ext2]?
 Start?  0% 
 End?  100% 
 (parted)  p 
 Model: LSI MR9361-8i (scsi)
 Disk /dev/sdb: 4796GB

File system creation

I wasn't able to format partition using mkfs.ext4 command, and parted allowed me to format only ext2, so this is what I did. 
(parted)  mkfs 
Yes/No?  Yes 
Partition number?  1 
File system type?  [ext2]?  ext2  
(parted)  p 
Number    Start   End     Size    File system  Name     Flags
 1      1049kB  4796GB  4796GB    ext2         primary
Then convert from ext2 to ext4. Since disk is under multipath control, use "multipath -ll" to find disk name. 
#  tune2fs -O extents,uninit_bg,dir_index,has_journal /dev/mapper/3600605b00ba10960206c1641632c2c0fp1 
tune2fs 1.43-WIP (20-Jun-2013)
Creating journal inode: done
This filesystem will be automatically checked every 30 mounts or
0 days, whichever comes first.  Use tune2fs -c or -i to override.

File system check

Run fsck. 
#  e2fsck -pf /dev/mapper/3600605b00ba10960206c1641632c2c0fp1 
/dev/mapper/3600605b00ba10960206c1641632c2c0fp1: Backing up journal inode block information.
/dev/mapper/3600605b00ba10960206c1641632c2c0fp1: Moving journal from /.journal to hidden inode.
/dev/mapper/3600605b00ba10960206c1641632c2c0fp1: Adding dirhash hint to filesystem.
/dev/mapper/3600605b00ba10960206c1641632c2c0fp1: 11/585449472 files (0.0% non-contiguous), 18405758/1170897408 blocks

Mounting

And mount by adding to /etc/fstab 
/dev/mapper/3600605b00ba10960206c1641632c2c0fp1 /raidpool ext4 defaults 1 2



Go back to the top of the page




Back to the main page