SAN zoning

Back to the main page

SAN zoning

Here is the doc in order to better understand SAN zoning with FC switch. Players are ...

Two StorEdge 6120 in Hight availability array

SunFire X4200 server (name: unixlab-7, OS: Solaris 10)

SunFire T2000 server (name: unixlab-2, OS: Solaris 10)

FC Switch SANbox2-16

... connected as per picture.

First volumes and slices are created on StorEdge 6120. Example of creating volume of 6 disks, RAID 1, with one standby disk ...

6120 # vol add vol9-1 data u1d01-06 raid 1 standby u1d07

The volume has to be initialized, it takes some time ...

6120 # vol init vol9-1 data

Mount volume ...

6120 # vol mount vol9-1

Then create some slices on this volume:

6120 # volslice create slice0 -z 100GB vol9-1
6120 # volslice create slice1 -z 150GB vol9-1
6120 # volslice create slice2 -z 200GB vol9-1

6120 # volslice list
Slice         Slice Num     Start Blk     Size Blks     Capacity      Volume
slice0        0             0             209723904      100.003 GB   vol9-1
slice1        1             209723904     314585856      150.005 GB   vol9-1
slice2        2             524309760     419447808      200.007 GB   vol9-1
slice3        3             943757568     524289024      250.000 GB   vol9-1
-             -             1468046592    288532224      137.582 GB   vol9-1

And assing the RW access to some slices from any host:

6120 # lun default all_lun rw
6120 # lun default lun 3 none

6120 # lun perm list
6120 # lun     slice   WWN                     Group Name      Group Perm      WWN Perm        Effective Perm     
--------------------------------------------------------------------------------------------------------
0       0       default                 --              --              rw              rw
1       1       default                 --              --              rw              rw
2       2       default                 --              --              rw              rw
3       3       default                 --              --              none            none
--------------------------------------------------------------------------------------------------------

No zoning is configured on the FC switch.

SANbox2-16 # show topology
  Port   Local Local                   Remote Remote                  Unique
  Number Type  PortWWN                 Type   NodeWWN                 ID
  ------ ----- -------                 ------ -------                 ------
  2      F     20:02:00:c0:dd:01:b0:65 N      10:00:00:03:ba:cc:c7:50 020200   P
  3      F     20:03:00:c0:dd:01:b0:65 N      20:00:00:00:c9:4a:c2:8f 020300   P
  6      F     20:06:00:c0:dd:01:b0:65 N      10:00:00:03:ba:cc:c7:38 020600   P
  7      F     20:07:00:c0:dd:01:b0:65 N      20:00:00:00:c9:4a:be:38 020700   P
  11     F     20:0b:00:c0:dd:01:b0:65 N      20:00:00:e0:8b:10:13:e4 020b00   P
  15     F     20:0f:00:c0:dd:01:b0:65 N      20:01:00:e0:8b:30:13:e4 020f00   P

And servers can see LUNs and create zpool.

SunFire X4200:
{unixlab-7}/# format </dev/null
AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <DEFAULT cyl 8921 alt 2 hd 255 sec 63>
          /pci@0,0/pci1022,7450@2/pci1000,3060@3/sd@0,0
       1. c0t1d0 <FUJITSU-MAY2073RCSUN72G-0501-68.37GB>
          /pci@0,0/pci1022,7450@2/pci1000,3060@3/sd@1,0
       2. c6t60003BACCC75000050C67AA40007EC7Bd0 <SUN-T4-0302-100.00GB>
          /scsi_vhci/disk@g60003baccc75000050c67aa40007ec7b
       3. c6t60003BACCC75000050C67AB3000358E2d0 <SUN-T4-0302-150.01GB>
          /scsi_vhci/disk@g60003baccc75000050c67ab3000358e2
       4. c6t60003BACCC75000050C67ABF000D60BFd0 <DEFAULT cyl 26106 alt 2 hd 255 sec 63>
          /scsi_vhci/disk@g60003baccc75000050c67abf000d60bf

{unixlab-7}/# zpool list test-100
NAME       SIZE  ALLOC   FREE    CAP  HEALTH  ALTROOT
test-100  99.5G  98.5K  99.5G     0%  ONLINE  -

{unixlab-7}/# zpool status test-100
  pool: test-100
 state: ONLINE
 scan: none requested
config:
        NAME                                     STATE     READ WRITE CKSUM
        test-100                                 ONLINE       0     0     0
          c6t60003BACCC75000050C67AA40007EC7Bd0  ONLINE       0     0     0

SunFire T2000:
{unixlab-2}/# format 
AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /pci@780/pci@0/pci@9/scsi@0/sd@0,0
       1. c0t1d0 <SEAGATE-ST914602SSUN146G-0400-136.73GB>
          /pci@780/pci@0/pci@9/scsi@0/sd@1,0
       2. c4t60003BACCC75000050C67AA40007EC7Bd0 <SUN-T4-0302-100.00GB>
          /scsi_vhci/ssd@g60003baccc75000050c67aa40007ec7b
       3. c4t60003BACCC75000050C67AB3000358E2d0 <SUN-T4-0302-150.01GB>
          /scsi_vhci/ssd@g60003baccc75000050c67ab3000358e2
       4. c4t60003BACCC75000050C67ABF000D60BFd0 <SUN-T4-0302 cyl 34132 alt 2 hd 96 sec 128>
          /scsi_vhci/ssd@g60003baccc75000050c67abf000d60bf

{unixlab-2}/# zpool list test-150
NAME       SIZE  ALLOC   FREE    CAP  HEALTH  ALTROOT
test-150   149G  83.5K   149G     0%  ONLINE  -

{unixlab-2}/# zpool status test-150
  pool: test-150
 state: ONLINE
 scrub: none requested
config:
        NAME                                     STATE     READ WRITE CKSUM
        test-150                                 ONLINE       0     0     0
          c4t60003BACCC75000050C67AB3000358E2d0  ONLINE       0     0     0

But there is no security here, since both servers see all LUNs and one server can just create zpool with already used LUN by other server, and here is unpredictable situation with most likly data corruption and other problems.

This can be quickly fixed with LUN permissions for only one server's WWN ...
The LUN masking hides (or mask) LUNs so server can see LUNs you want to see.

{unixlab-7}/# fcinfo hba-port | grep "HBA Port WWN"
HBA Port WWN: 10000000c94ac28f
HBA Port WWN: 10000000c94abe38

6120 # lun perm list
lun     slice   WWN                     Group Name      Group Perm      WWN Perm        Effective Perm
--------------------------------------------------------------------------------------------------------
0       0       default                 --              --              none            none
0       0       10000000c94ac28f        --              --              rw              rw
0       0       10000000c94abe38        --              --              rw              rw
1       1       default                 --              --              rw              rw
2       2       default                 --              --              rw              rw
3       3       default                 --              --              none            none
--------------------------------------------------------------------------------------------------------

This server sees LUN 0:

{unixlab-7}/# format
AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <DEFAULT cyl 8921 alt 2 hd 255 sec 63>
          /pci@0,0/pci1022,7450@2/pci1000,3060@3/sd@0,0
       1. c0t1d0 <FUJITSU-MAY2073RCSUN72G-0501-68.37GB>
          /pci@0,0/pci1022,7450@2/pci1000,3060@3/sd@1,0
       2. c6t60003BACCC75000050C67AA40007EC7Bd0 <SUN-T4-0302-100.00GB>
          /scsi_vhci/disk@g60003baccc75000050c67aa40007ec7b
       3. c6t60003BACCC75000050C67AB3000358E2d0 <SUN-T4-0302-150.01GB>
          /scsi_vhci/disk@g60003baccc75000050c67ab3000358e2
       4. c6t60003BACCC75000050C67ABF000D60BFd0 <DEFAULT cyl 26106 alt 2 hd 255 sec 63>
          /scsi_vhci/disk@g60003baccc75000050c67abf000d60bf

But this one cannot see LUN 0:

{unixlab-2}/# format
AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /pci@780/pci@0/pci@9/scsi@0/sd@0,0
       1. c0t1d0 <SEAGATE-ST914602SSUN146G-0400-136.73GB>
          /pci@780/pci@0/pci@9/scsi@0/sd@1,0
       2. c4t60003BACCC75000050C67AB3000358E2d0 <SUN-T4-0302-150.01GB>
          /scsi_vhci/ssd@g60003baccc75000050c67ab3000358e2
       3. c4t60003BACCC75000050C67ABF000D60BFd0 <SUN-T4-0302 cyl 34132 alt 2 hd 96 sec 128>
          /scsi_vhci/ssd@g60003baccc75000050c67abf000d60bf

And how about zoning?
A zone is a named group of devices that can communicate with each other. Membership in a zone can be defined by switch port number, port Fibre Channel address, or by device worldwide name (WWN). Devices can communicate only with devices that are members of the same zone. A zone can be a member of more than one zone set. Several zone sets can be defined for a fabric, but only one zone set can be active at one time. The active zone set determines the current fabric zoning.

The SANbox2-16 used here supports next three zone types:

Soft zone (It divides the fabric for purposes of controlling discovery. Members of the same soft zone automatically discover and communicate freely with all other members of the same zone. The soft zone boundary is not secure; traffic across soft zones can occur if addressed correctly. Soft zones can overlap, a port can be a member of more than one soft zone. Membership can be defined by Fibre Channel address, port ID and domain ID, or worldwide name.)
Access Control List (ACL) - hard zone (It divides the fabric for purposes of controlling discovery and inbound traffic. ACL zoning is a type of hard zoning that is hardware enforced. This type of zoning is useful for controlling access to certain devices without totally isolating them from the fabric. Members can communicate with each other and transmit outside the ACL zone, but cannot receive inbound traffic from outside the zone. ACL zones can overlap, a port can be a member of more than one ACL zone. Membership can be defined only by domain ID and port ID.)
Virtual Private Fabric (VPF) - hard zone (It divides the fabric for purposes of controlling discovery and both inbound and outbound traffic. This type of zoning is useful for providing security and reserving paths between devices to guarantee bandwidth. VPF zoning is a type of hard zoning that is hardware enforced. Members can only transmit to and receive from members of the same VPF zone. The VPF zone boundary is secure against both inbound and outbound traffic. VPF zones cannot overlap, a port can be a member of only one VPF zone. Membership can be defined only by domain ID and port ID.)

See next commands to create a hard zone (ACL one)

SANbox2-16 (admin-zoning) # zone create unixlab-2
SANbox2-16 (admin-zoning) # zone type unixlab-2 HardACL
SANbox2-16 (admin-zoning) # zone type unixlab-2
  Zone type value: HardACL

Find switch's Domain ID with the command:

SANbox2-16 (admin) # show switch
  Switch Information
  ------------------
  SymbolicName                      fc3
  SwitchWWN                         10:00:00:c0:dd:01:b0:65
  SwitchType                        SANbox2-16
  PROMVersion                       V0.4-0-0   (Fri Jan 10 15:32:35 2003)
  DomainID                          2 (0x2)
  ...

And add "domainID,portID" to the zone (see the picture, only server unixlab-2 can access the storage)

SANbox2-16 (admin-zoning) # zone add unixlab-2 2,2
SANbox2-16 (admin-zoning) # zone add unixlab-2 2,6
SANbox2-16 (admin-zoning) # zone add unixlab-2 2,11
SANbox2-16 (admin-zoning) # zone add unixlab-2 2,15
SANbox2-16 (admin-zoning) # zone members unixlab-2
  Current list of members for Zone: unixlab-2
  ---------------------------------
  2,2
  2,6
  2,11
  2,15

Create a zoneset, add the zone to it and activate the zoneset:

SANbox2-16 (admin-zoning) # zoneset create zoneset_hard
SANbox2-16 (admin-zoning) # zoneset add zoneset_hard unixlab-2
SANbox2-16 (admin) # zoneset activate zoneset_hard
SANbox2-16 (admin) # zoning list
  Active ZoneSet Information
  ZoneSet     Zone      ZoneMember
  --------------------------------
  zoneset_hard
              unixlab-2 (HardACL)
                        2,2
                        2,6
                        2,11
                        2,15
  Configured Zoning Information
  ZoneSet     Zone      ZoneMember
  --------------------------------
  zoneset_hard
              unixlab-2 (HardACL)
                        2,2
                        2,6
                        2,11
                        2,15

Now the hard zone is active and all 4 LUNs have rw permissions, so only unixlab-2 can see LUNs, and we can create zpool from LUN4.

6120 # lun perm list
lun     slice   WWN                     Group Name      Group Perm      WWN Perm        Effective Perm
--------------------------------------------------------------------------------------------------------
0       0       default                 --              --              rw              rw 
1       1       default                 --              --              rw              rw
2       2       default                 --              --              rw              rw
3       3       default                 --              --              rw              rw
--------------------------------------------------------------------------------------------------------

{unixlab-7}/# format 
       0. c0t0d0 <DEFAULT cyl 8921 alt 2 hd 255 sec 63>
          /pci@0,0/pci1022,7450@2/pci1000,3060@3/sd@0,0
       1. c0t1d0 <FUJITSU-MAY2073RCSUN72G-0501-68.37GB>
          /pci@0,0/pci1022,7450@2/pci1000,3060@3/sd@1,0

{unixlab-2}/# format
AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /pci@780/pci@0/pci@9/scsi@0/sd@0,0
       1. c0t1d0 <SEAGATE-ST914602SSUN146G-0400-136.73GB>
          /pci@780/pci@0/pci@9/scsi@0/sd@1,0
       2. c4t60003BACCC75000050C67AA40007EC7Bd0 <SUN-T4-0302-100.00GB>
          /scsi_vhci/ssd@g60003baccc75000050c67aa40007ec7b
       3. c4t60003BACCC75000050C67AB3000358E2d0 <SUN-T4-0302-150.01GB>
          /scsi_vhci/ssd@g60003baccc75000050c67ab3000358e2
       4. c4t60003BACCC75000050C67ABF000D60BFd0 <SUN-T4-0302-200.00GB>
          /scsi_vhci/ssd@g60003baccc75000050c67abf000d60bf
       5. c4t60003BACCC75000050C67AE400058B23d0 <SUN-T4-0302-250.00GB>
          /scsi_vhci/ssd@g60003baccc75000050c67ae400058b23

{unixlab-2}/# zpool create space3 c4t60003BACCC75000050C67AE400058B23d0
{unixlab-2}/# zpool list space3
NAME     SIZE  ALLOC   FREE    CAP  HEALTH  ALTROOT
space3   248G  95.5K   248G     0%  ONLINE  -
{unixlab-2}/# zpool status space3
   pool: space3
 state: ONLINE
 scrub: none requested
config:
        NAME                                     STATE     READ WRITE CKSUM
        space3                                   ONLINE       0     0     0
          c4t60003BACCC75000050C67AE400058B23d0  ONLINE       0     0     0

The LUN masking works on top of zoning.
A zone allow/deny access to a port of storage, and LUN masking allow/deny one server access to specific LUNs on that storage. So let's now have hard zone active and use LUN masking to deny access to LUN 0.

6120 # lun default lun 0 none
6120 # lun perm list
lun     slice   WWN                     Group Name      Group Perm      WWN Perm        Effective Perm
--------------------------------------------------------------------------------------------------------
0       0       default                 --              --              none            none 
1       1       default                 --              --              rw              rw
2       2       default                 --              --              rw              rw
3       3       default                 --              --              rw              rw
--------------------------------------------------------------------------------------------------------

The Server unixlab-2 can't see the LUN 0

{unixlab-2}/# format
       0. c0t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /pci@780/pci@0/pci@9/scsi@0/sd@0,0
       1. c0t1d0 <SEAGATE-ST914602SSUN146G-0400-136.73GB>
          /pci@780/pci@0/pci@9/scsi@0/sd@1,0
       2. c4t60003BACCC75000050C67AB3000358E2d0 <SUN-T4-0302-150.01GB>
          /scsi_vhci/ssd@g60003baccc75000050c67ab3000358e2
       3. c4t60003BACCC75000050C67ABF000D60BFd0 <SUN-T4-0302-200.00GB>
          /scsi_vhci/ssd@g60003baccc75000050c67abf000d60bf
       4. c4t60003BACCC75000050C67AE400058B23d0 <SUN-T4-0302-250.00GB>
          /scsi_vhci/ssd@g60003baccc75000050c67ae400058b23

Back to the main page