Reasoning about ASM PST

Following this article I did a few tests. I’ve just created a new diskgroup with normal redundancy. Bellow you can see that partners of disks is ok.

SQL> select  disk,count(NUMBER_KFDPARTNER), count(ACTIVE_KFDPARTNER) from x$kfdpartner where grp=1 group by disk;

      DISK COUNT(NUMBER_KFDPARTNER) COUNT(ACTIVE_KFDPARTNER)
---------- ------------------------ ------------------------
         0                        7                        7
         1                        7                        7
         2                        7                        7
         3                        7                        7
         4                        7                        7
         5                        7                        7
         6                        7                        7
         7                        7                        7
         8                        7                        7
         9                        7                        7
        10                        7                        7

      DISK COUNT(NUMBER_KFDPARTNER) COUNT(ACTIVE_KFDPARTNER)
---------- ------------------------ ------------------------
        11                        7                        7
        12                        7                        7
        13                        7                        7

Also I’ve checked PST_DATA in header of disks. For example in my disk /dev/oracleasm/disk/DISK1 by aun=1 and blkn=3 has this information.

kfdpDtaEv0[0].status:         117440512 ; 0x000: V=1 R=1 W=1
kfdpDtaEv0[0].index:          117900549 ; 0x004: CURR=0x5 CURR=0x5 FORM=0x7 FORM=0x7
kfdpDtaEv0[0].partner[0]:            12 ; 0x008: 0x000c
kfdpDtaEv0[0].partner[1]:            10 ; 0x00a: 0x000a
kfdpDtaEv0[0].partner[2]:             9 ; 0x00c: 0x0009
kfdpDtaEv0[0].partner[3]:             8 ; 0x00e: 0x0008
kfdpDtaEv0[0].partner[4]:             7 ; 0x010: 0x0007
kfdpDtaEv0[0].partner[5]:            13 ; 0x012: 0x000d
kfdpDtaEv0[0].partner[6]:            11 ; 0x014: 0x000b
kfdpDtaEv0[0].partner[7]:             0 ; 0x016: 0x0000
kfdpDtaEv0[0].partner[8]:             0 ; 0x018: 0x0000
kfdpDtaEv0[0].partner[9]:             0 ; 0x01a: 0x0000
kfdpDtaEv0[0].partner[10]:            0 ; 0x01c: 0x0000
kfdpDtaEv0[0].partner[11]:            0 ; 0x01e: 0x0000
kfdpDtaEv0[0].partner[12]:            0 ; 0x020: 0x0000
kfdpDtaEv0[0].partner[13]:            0 ; 0x022: 0x0000
kfdpDtaEv0[0].partner[14]:            0 ; 0x024: 0x0000
kfdpDtaEv0[0].partner[15]:            0 ; 0x026: 0x0000
kfdpDtaEv0[0].partner[16]:            0 ; 0x028: 0x0000
kfdpDtaEv0[0].partner[17]:            0 ; 0x02a: 0x0000
kfdpDtaEv0[0].partner[18]:            0 ; 0x02c: 0x0000
kfdpDtaEv0[0].partner[19]:            0 ; 0x02e: 0x0000

As you can see PST table has limit by 20 partners disks. After deleted and added some disks without rebalancing we’ve got more than 7 partners for disks.

kfdpDtaEv0[0].status:         117440512 ; 0x000: V=1 R=1 W=1
kfdpDtaEv0[0].index:          151586567 ; 0x004: CURR=0x7 CURR=0x7 FORM=0x9 FORM=0x9
kfdpDtaEv0[0].partner[0]:            12 ; 0x008: 0x000c
kfdpDtaEv0[0].partner[1]:            10 ; 0x00a: 0x000a
kfdpDtaEv0[0].partner[2]:             9 ; 0x00c: 0x0009
kfdpDtaEv0[0].partner[3]:             8 ; 0x00e: 0x0008
kfdpDtaEv0[0].partner[4]:             7 ; 0x010: 0x0007
kfdpDtaEv0[0].partner[5]:            14 ; 0x012: 0x000e
kfdpDtaEv0[0].partner[6]:            15 ; 0x014: 0x000f
kfdpDtaEv0[0].partner[7]:            13 ; 0x016: 0x000d
kfdpDtaEv0[0].partner[8]:            11 ; 0x018: 0x000b
kfdpDtaEv0[0].partner[9]:             0 ; 0x01a: 0x0000
kfdpDtaEv0[0].partner[10]:            0 ; 0x01c: 0x0000
kfdpDtaEv0[0].partner[11]:            0 ; 0x01e: 0x0000
kfdpDtaEv0[0].partner[12]:            0 ; 0x020: 0x0000
kfdpDtaEv0[0].partner[13]:            0 ; 0x022: 0x0000
kfdpDtaEv0[0].partner[14]:            0 ; 0x024: 0x0000
kfdpDtaEv0[0].partner[15]:            0 ; 0x026: 0x0000
kfdpDtaEv0[0].partner[16]:            0 ; 0x028: 0x0000
kfdpDtaEv0[0].partner[17]:            0 ; 0x02a: 0x0000
kfdpDtaEv0[0].partner[18]:            0 ; 0x02c: 0x0000
kfdpDtaEv0[0].partner[19]:            0 ; 0x02e: 0x0000

And different disks has different states.

SQL> select  disk,count(NUMBER_KFDPARTNER), count(ACTIVE_KFDPARTNER) from x$kfdpartner where grp=1 group by disk;

      DISK COUNT(NUMBER_KFDPARTNER) COUNT(ACTIVE_KFDPARTNER)
---------- ------------------------ ------------------------
         0                       10                       10
         1                       16                       16
         2                        7                        7
         3                        7                        7
         4                        9                        9
         5                        9                        9
         6                       10                       10
         7                       13                       13
         8                       16                       16
         9                       16                       16
        10                       12                       12

      DISK COUNT(NUMBER_KFDPARTNER) COUNT(ACTIVE_KFDPARTNER)
---------- ------------------------ ------------------------
        11                       17                       17
        12                       14                       14
        13                       19                       19

Eventually we’ve got ORA-15074. And only one way to resolve this, it’s execute rebalance.

SQL> alter diskgroup data drop disk DISK8 force rebalance power 0;
alter diskgroup data drop disk DISK8 force rebalance power 0
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15074: diskgroup DATA requires rebalance completion
SQL> alter diskgroup data rebalance power 10;

SQL> alter diskgroup data drop disk DISK8 force rebalance power 0;

Diskgroup altered.

SQL>
SQL> alter diskgroup data add disk 'ORCL:DISK8' force rebalance power 0;

Diskgroup altered.

Even though we’ve deleted and added the same disk we’ve got unbalanced state for PST table, and we need to execute rebalance ASAP.

SQL> select  disk,count(NUMBER_KFDPARTNER), count(ACTIVE_KFDPARTNER) from x$kfdpartner where grp=1 group by disk;

      DISK COUNT(NUMBER_KFDPARTNER) COUNT(ACTIVE_KFDPARTNER)
---------- ------------------------ ------------------------
         0                        8                        8
         1                       10                       10
         7                        7                        7
         8                        7                        7
         9                        8                        8
        10                        8                        8
        11                       10                       10
        12                        8                        8
        13                       11                       11
        14                        8                        8
        15                        8                        8

      DISK COUNT(NUMBER_KFDPARTNER) COUNT(ACTIVE_KFDPARTNER)
---------- ------------------------ ------------------------
        16                       11                       11
        17                       10                       10
        19                       11                       11
        23                       11                       11

15 rows selected.
SQL> alter diskgroup data rebalance power 10;

Diskgroup altered.

SQL> select  disk,count(NUMBER_KFDPARTNER), count(ACTIVE_KFDPARTNER) from x$kfdpartner where grp=1 group by disk;

      DISK COUNT(NUMBER_KFDPARTNER) COUNT(ACTIVE_KFDPARTNER)
---------- ------------------------ ------------------------
         0                        8                        8
         1                        8                        8
         8                        8                        8
         9                        7                        7
        10                        8                        8
        11                        8                        8
        12                        8                        8
        13                        8                        8
        14                        8                        8
        15                        7                        7
        16                        8                        8

      DISK COUNT(NUMBER_KFDPARTNER) COUNT(ACTIVE_KFDPARTNER)
---------- ------------------------ ------------------------
        17                        8                        8
        19                        8                        8
        23                        8                        8

14 rows selected.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s