Applies to:

Symptoms

Database which stores its data in ASM will not mount and throw the following errors:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
Thu Apr 19 13:59:31 2012
Errors in file /eas/u01/app/oracle/product/diag/rdbms/easp1/easp1/trace/easp1_asmb_24508.trc:
ORA-15064: communication failure with ASM instance
ORA-00600: internal error code, arguments: [ORA_NPI_ERROR], [600], [ORA-00600: internal error code, arguments: [kfnsInstanceReg00],
[easp1:easp1], [30000], [], [], [], [], [], [], [], [], []], [], [], [], [], [], [], [], [], []
...
Errors in file /eas/u01/app/oracle/product/diag/rdbms/easp1/easp1/trace/easp1_asmb_24617.trc (incident=254557):
ORA-00600: internal error code, arguments: [OSDEP_INTERNAL], [], [], [], [], [], [], [], [], [], [], []
ORA-27302: failure occurred at: skfnbeqOpenI
Incident details in: /eas/u01/app/oracle/product/diag/rdbms/easp1/easp1/incident/incdir_254557/easp1_asmb_24617_i254557.trc
...
ORA-15083: failed to communicate with ASMB background process
ORA-00210: cannot open the specified control file
ORA-00202: control file: '+DATA/easp1/control01.ctl'
ORA-17503: ksfdopn:2 Failed to open file +DATA/easp1/control01.ctl
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-15055: unable to connect to ASM instance

---------------------------------------------------------------------------------------------------------------------------------------------------------------------

These errors occur at the time the database attempts to mount, thus access the ASM instance for the database files.

Seems the database processes cannot connect to the started ASM instance (which has all required ASM disk groups mounted)

Cause

For this particular environment, the Job Role Separation (JRS) configuration was improper. There were no separate users for the gi_home and the db_home nor the 'asmadmin' OS group was created or assigned to the Grid Infrastructure (GI) software owner (asm_home)

The problem was that the database home's 'oracle' executable was assigned the 'asmadmin' group, when it should have been assigned the 'dba' OS group:

-------------------------------------------------------------------------------------------------------------------------------------------------------

Oracle software owner (for gi_home and db_home) and OS groups assigned to it:
$id
uid=1005(oraeas) gid=520(dba)

Therefore, the gi_home's oracle executable owner (oraeas for this example), has only 'dba' as the OS group:
$ls -ltra /grid/u01/app/oracle/product/11.2.0/grid2/bin/oracle
-rwsr-s--x 1 oraeas dba 217813688 Mar 28 09:11 /grid/u01/app/oracle/product/11.2.0/grid2/bin/oracle

However for the db_home's oracle executable owner (oraeas, also for this example), 'asmadmin' was assigned as the OS group

$ls -ltra /eas/u01/app/oracle/product/11.2.0.3/bin/oracle
-rwsr-s--x 1 oraeas asmadmin 246074792 Apr 9 12:06 /eas/u01/app/oracle/product/11.2.0.3/bin/oracle

-------------------------------------------------------------------------------------------------------------------------------------------------------

Solution

Therefore do:

$ su -
$ cd /bin
$ ./setasmgidwrap o=/bin/oracle

ChecK to see that 'dba' was assigned to db_home's oracle executable:
$ ls -ltra /eas/u01/app/oracle/product/11.2.0.3/bin/oracle

Then, start up the database.