Earlier I stumbled upon situation when my ISCSI target has went offline. And I didn’t have time to repair ISCSI target until ASM repair timer was run out. Obviously ASM has offline disks and lately had dropped disks. After that when I could repair ISCSI target and all disks be able visible on database side I could re-add they to ASM disk group. But it can’t be easily do.
SQL> alter diskgroup FASTVOLUME add failgroup FG2 disk 'ORCL:DISK2E204',...... rebalance power 9; alter diskgroup FASTVOLUME add failgroup FG2 disk 'ORCL:DISK2E204',...... rebalance power 9 * ERROR at line 1: ORA-15032: not all alterations performed ORA-15033: disk 'ORCL:DISK2E204' belongs to diskgroup "GROUP" ... ...
This errors has happened when ASM can’t write correct information to disks headers since ASM had dropped the disks and write information to PST table. Obviously you have one way out. You should erase disk header through “dd” and re-add it again. If you use asmlib you should care about asmlib labels and before erase disk headers, delete disks from asmlib. I think this is not comfortable situation, that’s why I figure out how we can re-add disks without erasing disks headers.
All we need, read information by AU=0 and block=0 from dropped disk and replace some flags.
/opt/oracle/product/grid/184.108.40.206/bin/kfed read /dev/oracleasm/disks/DISK2E204 aun=0 blkn=0 > /tmp/disk2e204
Replace number in “hdrsts” flag, from 3 to 2. This replaced “MEMBER” status to “CANDIDATE”.
... ... kfdhdb.grptyp: 2 ; 0x026: KFDGTP_NORMAL kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER -- this should be 2 instead of 3 ... ...
Then we should write replaced information back to disk.
/opt/oracle/product/grid/220.127.116.11/bin/kfed write /dev/oracleasm/disks/DISK2E204 aun=0 blkn=0 text=/tmp/disk2e204
Repeat this steps for all of yours dropped disks. After that you can re-add disks back to disk group without “force” options.