6.3 Alternate PV Links

     

In the world of high-availability, we are always concerned with a Single Point Of Failure (SPOF). We need to consider our interface cards as one such SPOF. The way we alleviate this issue is to employ Alternate PV Links . This is simply an additional interface card and additional cable connected to the disk. The hardware setup of an Alternate PV Link is crucial as well. If your interface is a SCSI interface, you will need to ensure that the SCSI target address of the interface card is unique. Having two devices on the SCSI bus with the same target address will cause no end of trouble (lots of SCSI lbolt / reset messages in syslog for one thing!). The problem with Alternate PV Links is the ability to spot them. From the perspective of HP-UX, it simply sees another disk drive: another entry from ioscan . The skill is being able to identify an Alternate PV Link in order to configure them into a volume group . Take this ioscan output from my system, for example:

 

 root@hpeos003[]  ioscan -fknC disk  more  Class     I  H/W Path      Driver S/W State   H/W Type     Description ====================================================================== disk      0  0/0/1/0.0.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c0t0d0   /dev/rdsk/c0t0d0 disk      1  0/0/1/0.1.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c0t1d0   /dev/rdsk/c0t1d0 disk      2  0/0/1/0.2.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c0t2d0   /dev/rdsk/c0t2d0 disk      3  0/0/1/0.3.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c0t3d0   /dev/rdsk/c0t3d0 disk      4  0/0/1/0.4.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c0t4d0   /dev/rdsk/c0t4d0 disk      5  0/0/1/0.5.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c0t5d0   /dev/rdsk/c0t5d0 disk      6  0/0/1/0.6.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c0t6d0   /dev/rdsk/c0t6d0 disk      7  0/0/1/1.15.0  sdisk CLAIMED     DEVICE       HP 36.4GST336753LC                           /dev/dsk/c1t15d0   /dev/rdsk/c1t15d0 disk      8  0/0/2/1.15.0  sdisk CLAIMED     DEVICE       HP 36.4GST336753LC                           /dev/dsk/c3t15d0   /dev/rdsk/c3t15d0 disk      9  0/6/0/0.8.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c4t8d0   /dev/rdsk/c4t8d0 disk     10  0/6/0/0.9.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c4t9d0   /dev/rdsk/c4t9d0 disk     11  0/6/0/0.10.0  sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c4t10d0   /dev/rdsk/c4t10d0 disk     12  0/6/0/0.11.0  sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c4t11d0   /dev/rdsk/c4t11d0 disk     13  0/6/0/0.12.0  sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c4t12d0   /dev/rdsk/c4t12d0 disk     14  0/6/0/0.13.0  sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c4t13d0   /dev/rdsk/c4t13d0 disk     15  0/6/0/0.14.0  sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c4t14d0   /dev/rdsk/c4t14d0 disk     16  0/6/0/1.0.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c5t0d0   /dev/rdsk/c5t0d0 disk     17  0/6/0/1.1.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c5t1d0   /dev/rdsk/c5t1d0 disk     18  0/6/0/1.2.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c5t2d0   /dev/rdsk/c5t2d0 disk     19  0/6/0/1.3.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c5t3d0   /dev/rdsk/c5t3d0 disk     20  0/6/0/1.4.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c5t4d0   /dev/rdsk/c5t4d0 disk     21  0/6/0/1.5.0   sdisk CLAIMED     DEVICE       HP 73.4GST373307LC                           /dev/dsk/c5t5d0   /dev/rdsk/c5t5d0 root@hpeos003[] 

It would appear that I have a host of disks connected to interface 0/6/0/1. The device addresses of 1, 2, 3, 4, 5, and 6 look suspiciously similar to the devices connected to interface 0/0/1/0. If you know your hardware configuration well and it is well documented, then it may be obvious to you that the disks c5t1d0 , and so on, are actually Alternate PV Links to disk c0t1d0 , and so on. You need to be sure of your configuration because it wouldn't be the first time I had a customer who ran a pvcreate “f command against a device file that turned out to be an Alternate PV Link. You should be suspicious the first time you run pvcreate against such a device because it will warn you that the device already belongs to a volume group:

 

 root@hpeos003[]  pvcreate /dev/rdsk/c5t1d0  pvcreate: The physical volume already belongs to a volume group root@hpeos003[] 

This should immediately raise our suspicions because this device is not listed in /etc/lvmtab :

 

 root@hpeos003[]  strings /etc/lvmtab  /dev/vg00 /dev/dsk/c1t15d0 /dev/dsk/c3t15d0 /dev/vgora1 /dev/dsk/c0t1d0 /dev/dsk/c0t2d0 /dev/dsk/c0t3d0 /dev/dsk/c4t9d0 /dev/dsk/c4t10d0 root@hpeos003[] 

The disk in question could have been an old disk, and pvcreate is simply saying that it has found a valid LVM header on the disk. We need to be absolutely sure that this is a separate disk before using pvcreate “f . Before even contemplating using pvcreate “f , we can at least read the LVM header of the disk using this and the existing device file. If the LVM header is different, then the disks are separate disks. If the LVM header is exactly the same, then the new device file is simply an Alternate PV Link to the same disk. Reading the LVM header off of the disk requires a little knowledge of the LVM structures at the beginning of the disk. That's where you'll find the 8KB header used to point to the boot area. We then have the PVRA . The first element of the PVRA is a LVM Record. That identifies this disk as an LVM disk. The LVM Record is 8 bytes in size . That's 8KB + 8 bytes = 8200(2008 in hex) bytes. If we read from that location, we will find four interesting numbers in the PVRA ; the CPU-ID , the PV-ID , the CPU-ID (again), and the VG-ID . If we were to read this information from two distinct device files and see exactly the same information, we are looking at an Alternate PV Link:

 

 root@hpeos003[]  echo "0x2008?4D"  adb /dev/dsk/c0t1d0  2008:           894960601       1067429315      894960601       1067430042 root@hpeos003[]  echo "0x2008?4D"  adb /dev/dsk/c5t1d0  2008:           894960601       1067429315      894960601       1067430042 root@hpeos003[] 

As we can see, here the information from both device files is exactly the same. I can confirm this with the other disks in the volume group:

 

 root@hpeos003[]  echo "0x2008?4D"  adb /dev/dsk/c0t2d0  2008:           894960601       1067429321      894960601       1067430042 root@hpeos003[]  echo "0x2008?4D"  adb /dev/dsk/c5t2d0  2008:           894960601       1067429321      894960601       1067430042 root@hpeos003[]  echo "0x2008?4D"  adb /dev/dsk/c0t3d0  2008:           894960601       1067429631      894960601       1067430042 root@hpeos003[]  echo "0x2008?4D"  adb /dev/dsk/c5t3d0  2008:           894960601       1067429631      894960601       1067430042 root@hpeos003[] 

We can safely conclude that the device c5t[123]d0 are Alternate PV Links for the corresponding device c0t[123]d0 . We can add the Alternate PV Links into the volume group:

 

 root@hpeos003[]  vgextend /dev/vgora1 /dev/dsk/c5t[123]d0  Volume group "/dev/vgora1" has been successfully extended. Volume Group configuration for /dev/vgora1 has been saved in /etc/lvmconf/vgora1.conf root@hpeos003[] root@hpeos003[]  vgdisplay -v vgora1  more  --- Volume groups --- VG Name                     /dev/vgora1 VG Write Access             read/write VG Status                   available Max LV                      255 Cur LV                      1 Open LV                     1 Max PV                      16 Cur PV                      6 Act PV                      6 Max PE per PV               17501 VGDA                        12 PE Size (Mbytes)            4 Total PE                    87495 Alloc PE                    500 Free PE                     86995 Total PVG                   2 Total Spare PVs             1 Total Spare PVs in use      0    --- Logical volumes ---    LV Name                     /dev/vgora1/db    LV Status                   available/syncd    LV Status                   available/syncd    LV Size (Mbytes)            1000    Current LE                  250    Allocated PE                500    Used PV                     2    --- Physical volumes ---    PV Name                     /dev/dsk/c0t1d0   PV Name                     /dev/dsk/c5t1d0  Alternate Link   PV Status                   available    Total PE                    17499    Free PE                     17249    Autoswitch                  On    PV Name                     /dev/dsk/c0t2d0   PV Name                     /dev/dsk/c5t2d0  Alternate Link   PV Status                   available    Total PE                    17499    Free PE                     17499    Autoswitch                  On    PV Name                     /dev/dsk/c0t3d0   PV Name                     /dev/dsk/c5t3d0  Alternate Link   PV Status                   available    Total PE                    17499    Free PE                     17499    Autoswitch                  On    PV Name                     /dev/dsk/c4t9d0    PV Status                   available    Total PE                    17499    Free PE                     17249    Autoswitch                  On    PV Name                     /dev/dsk/c4t10d0    PV Status                   available/standby spare    Total PE                    17499    Free PE                     17499    Autoswitch                  On    PV Name                     /dev/dsk/c4t11d0    PV Status                   available    Total PE                    17499    Free PE                     17499    Autoswitch                  On    --- Physical volume groups ---    PVG Name                    PVG0    PV Name                     /dev/dsk/c0t1d0    PV Name                     /dev/dsk/c0t2d0    PV Name                     /dev/dsk/c0t3d0    PVG Name                    PVG1    PV Name                     /dev/dsk/c4t9d0    PV Name                     /dev/dsk/c4t10d0    PV Name                     /dev/dsk/c4t11d0 root@hpeos003[] 

If something should happen to the Primary Link, e.g., the interface card fails, we should see LVM automatically move over to the Alternate Link .

 

 root@hpeos003[]  more /var/adm/syslog/syslog.log  ... Oct 30 17:28:57 hpeos003 vmunix: LVM: Path (device   0x1f001000   ) to PV 1 in VG 1 Failed! Oct 30 17:28:57 hpeos003 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = graphics/ccc.gif 0x0000000042096080), from raw device   0x1f001000   (with priority: 0, and current flags: graphics/ccc.gif 0x40) to raw device   0x1f051000   (with priority: 1, and current flags: 0x0). Oct 30 17:28:57 hpeos003 vmunix: LVM: Path (device   0x1f002000   ) to PV 1 in VG 1 Failed! Oct 30 17:28:57 hpeos003 vmunix: LVM: Path (device   0x1f003000   ) to PV 2 in VG 1 Failed! Oct 30 17:28:57 hpeos003 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = graphics/ccc.gif 0x00000000420a1040), from raw device   0x1f003000   (with priority: 0, and current flags: graphics/ccc.gif 0x40) to raw device   0x1f053000   (with priority: 1, and current flags: 0x0). Oct 30 17:28:58 hpeos003 vmunix: LVM: Performed a switch for Lun ID = 0 (pv = graphics/ccc.gif 0x0000000042096840), from raw device   0x1f002000   (with priority: 0, and current flags: graphics/ccc.gif 0xc0) to raw device   0x1f052000   (with priority: 1, and current flags: 0x0). Oct 30 17:29:02 hpeos003 vmunix: LVM: Recovered Path (device   0x1f051000   ) to PV 0 in VG 1. Oct 30 17:29:02 hpeos003 vmunix: LVM: Recovered Path (device   0x1f052000   ) to PV 1 in VG 1. Oct 30 17:29:02 hpeos003 vmunix: LVM: Recovered Path (device   0x1f053000   ) to PV 2 in VG 1. Oct 30 17:29:02 hpeos003 vmunix: LVM: Restored PV 0 to VG 1. Oct 30 17:29:02 hpeos003 vmunix: LVM: Restored PV 1 to VG 1. Oct 30 17:29:02 hpeos003 vmunix: LVM: Restored PV 2 to VG 1. ... root@hpeos003[] 

If the fault is intermittent, you may see several messages in syslog as LVM bounces between Primary and Alternate Links . In such a situation, I would place a Hardware Call with my local Response Center for an engineer to investigate any potential hardware problems with my interface cards/cables/terminators.

I want you to remember one important thing regarding Alternate PV Links: they are there to provide redundancy against the failure of an interface. Some people would like the Alternate PV Link to be used to improve overall IO performance. (It's another path to our data, so we could spread the IO burden over all interfaces.) LVM does not support this configuration. Other software would have to be used to provide such a solution, e.g., Veritas Volume Manager with Dynamic Multi Pathing, HP AutoPath/VA, or AutoPath/XP, but these require the use of a VA or XP disk array. Alternate PV Links are used only in the event of an interface failure. Period.



HP-UX CSE(c) Official Study Guide and Desk Reference
HP-UX CSE(c) Official Study Guide and Desk Reference
ISBN: N/A
EAN: N/A
Year: 2006
Pages: 434

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net