How to escape ORA-15033 during re-add disks

Earlier I stumbled upon situation when my ISCSI target has went offline. And I didn’t have time to repair ISCSI target until ASM repair timer was run out. Obviously ASM has offline disks and lately had dropped disks. After that when I could repair ISCSI target and all disks be able visible on database side I could re-add they to ASM disk group. But it can’t be easily do.

SQL> alter diskgroup FASTVOLUME add failgroup FG2 disk 'ORCL:DISK2E204',...... rebalance power 9;
alter diskgroup FASTVOLUME add failgroup FG2 disk 'ORCL:DISK2E204',...... rebalance power 9
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15033: disk 'ORCL:DISK2E204' belongs to diskgroup "GROUP"
...
...

 

This errors has happened when ASM can’t write correct information to disks headers since ASM had dropped the disks and write information to PST table. Obviously you have one way out. You should erase disk header through “dd” and re-add it again. If you use asmlib you should care about asmlib labels and before erase disk headers, delete disks from asmlib. I think this is not comfortable situation, that’s why I figure out how we can re-add disks without erasing disks headers.

All we need, read information by AU=0 and block=0 from dropped disk and replace some flags.

 

/opt/oracle/product/grid/11.2.0.4/bin/kfed read /dev/oracleasm/disks/DISK2E204 aun=0 blkn=0 > /tmp/disk2e204

 
Replace number in “hdrsts” flag, from 3 to 2. This replaced “MEMBER” status to “CANDIDATE”.

 

...
...
kfdhdb.grptyp:                        2 ; 0x026: KFDGTP_NORMAL
kfdhdb.hdrsts:                        3 ; 0x027: KFDHDR_MEMBER -- this should be 2 instead of 3
...
...

 

Then we should write replaced information back to disk.

 

/opt/oracle/product/grid/11.2.0.4/bin/kfed write /dev/oracleasm/disks/DISK2E204 aun=0 blkn=0 text=/tmp/disk2e204

 

Repeat this steps for all of yours dropped disks. After that you can re-add disks back to disk group without “force” options.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s