首页 » Cloud, ORACLE 9i-23ai » Instance Startup Fails With ORA-00205, ORA-15040 After Applying Patch (安装补丁后实例启动失败)
Instance Startup Fails With ORA-00205, ORA-15040 After Applying Patch (安装补丁后实例启动失败)
最近安装一套12.2.0.1.190417 的数据库安装一个补丁时，安装后数据库无法启动，之前安装过相同的补丁在同平台，发现有时sqlplus/srvctl start instance不同的启动方法结果也是不一样的， 这里不得不吐槽ORACLE 补丁的readme中太不严禁, 而且AIX和suse同样的命令结果也是不一样的。
oracle@anbob1:/home/oracle/opatch/28423598> ps -ef|grep ora_ oracle 24687 19655 0 19:23 pts/7 00:00:00 grep --color=auto ora_ oracle@anbob1:/home/oracle/opatch/28423598> opatch apply Oracle Interim Patch Installer version 12.2.0.1.14 Copyright (c) 2019, Oracle Corporation. All rights reserved. Oracle Home : /oracle/app/oracle/product/12.2.0/db_1 Central Inventory : /oracle/app/oraInventory from : /oracle/app/oracle/product/12.2.0/db_1/oraInst.loc OPatch version : 12.2.0.1.14 OUI version : 12.2.0.1.4 Log file location : /oracle/app/oracle/product/12.2.0/db_1/cfgtoollogs/opatch/opatch2019-06-18_19-23-21PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 28423598 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/oracle/app/oracle/product/12.2.0/db_1') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '28423598' to OH '/oracle/app/oracle/product/12.2.0/db_1' Patching component oracle.rdbms, 12.2.0.1.0... Patch 28423598 successfully applied. Log file location: /oracle/app/oracle/product/12.2.0/db_1/cfgtoollogs/opatch/opatch2019-06-18_19-23-21PM_1.log OPatch succeeded. oracle@anbob1:/home/oracle/opatch/28423598> opatch lsinv|grep 28423598 Patch 28423598 : applied on Tue Jun 18 19:23:43 CST 2019
Note: 从上面看出28423598 patch已经成功安装，接下来使用sqlplus启动数据库。
$ sqlplus / as sysdba
SQL> startup nomount;
SQL> alter database mount;
alter database mount
*
ERROR at line 1:
ORA-00205: error in identifying control file, check alert log for more info
启动失败，检查数据库alert log
NOTE: ASMB0 (9831090) connected to ASM instance +ASM1, osid: 3015490 (Flex mode; client id 0x10001) NOTE: initiating MARK startup TMON started with pid=69, OS id=3539800 Starting background process MARK MARK started with pid=70, OS id=4391266 NOTE: MARK has subscribed ORACLE_BASE from environment = /oracle/app/oracle 2019-06-18 18:20:36.013000 +08:00 ALTER DATABASE MOUNT This instance was first to mount 2019-06-18 18:20:37.031000 +08:00 NOTE: ASMB mounting group 2 (DATADG) NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) WARNING: group 2 (DATADG) has missing disks ORA-15040: diskgroup is incomplete WARNING: group 2 is being dismounted. WARNING: ASMB force dismounting group 2 (DATADG) due to missing disks SUCCESS: diskgroup DATADG was dismounted NOTE: ASMB mounting group 2 (DATADG) NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) WARNING: group 2 (DATADG) has missing disks ORA-15040: diskgroup is incomplete WARNING: group 2 is being dismounted. WARNING: ASMB force dismounting group 2 (DATADG) due to missing disks SUCCESS: diskgroup DATADG was dismounted NOTE: ASMB mounting group 2 (DATADG) NOTE: ASM background process initiating disk discovery for grp 2 (reqid:0) WARNING: group 2 (DATADG) has missing disks ORA-15040: diskgroup is incomplete WARNING: group 2 is being dismounted. WARNING: ASMB force dismounting group 2 (DATADG) due to missing disks SUCCESS: diskgroup DATADG was dismounted ORA-00210: cannot open the specified control file ORA-00202: control file: '+DATADG/STDSVP/CONTROLFILE/current.268.998251971' ORA-17503: ksfdopn:2 Failed to open file +DATADG/STDSVP/CONTROLFILE/current.268.998251971 ORA-15001: diskgroup "DATADG" does not exist or is not mounted ORA-15040: diskgroup is incomplete ORA-00210: cannot open the specified control file ORA-00202: control file: '+DATADG/STDSVP/CONTROLFILE/current.269.998251971' ORA-17503: ksfdopn:2 Failed to open file +DATADG/STDSVP/CONTROLFILE/current.269.998251971 ORA-15001: diskgroup "DATADG" does not exist or is not mounted ORA-15040: diskgroup is incomplete ORA-00210: cannot open the specified control file ORA-00202: control file: '+DATADG/STDSVP/CONTROLFILE/current.270.998251971' ORA-17503: ksfdopn:2 Failed to open file +DATADG/STDSVP/CONTROLFILE/current.270.998251971 ORA-15001: diskgroup "DATADG" does not exist or is not mounted ORA-15040: diskgroup is incomplete
Note: 从db alert log检查是+DATADG ASM diskgroup未MOUNT,因为丢失磁盘。 是真的吗？
grid@anbob1:/home/grid> asmcmd ASMCMD> lsdg State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED EXTERN N 512 512 4096 4194304 2097152 2030188 0 2030188 0 N ARCHDG/ MOUNTED EXTERN N 512 512 4096 4194304 12582912 1265528 0 1265528 0 N DATADG/ MOUNTED EXTERN N 512 512 4096 4194304 102400 66164 0 66164 0 N MGMT/ MOUNTED NORMAL N 512 512 4096 4194304 10240 9348 2048 3650 0 Y OCRDG/ grid@anbob1:/home/grid> kfod disks=all -------------------------------------------------------------------------------- Disk Size Path User Group ================================================================================ 1: 524288 MB /dev/rhdisk10 grid asmadmin 2: 524288 MB /dev/rhdisk11 grid asmadmin 3: 524288 MB /dev/rhdisk12 grid asmadmin 4: 524288 MB /dev/rhdisk13 grid asmadmin 5: 524288 MB /dev/rhdisk14 grid asmadmin 6: 524288 MB /dev/rhdisk15 grid asmadmin 7: 524288 MB /dev/rhdisk16 grid asmadmin 8: 524288 MB /dev/rhdisk17 grid asmadmin 9: 524288 MB /dev/rhdisk18 grid asmadmin 10: 524288 MB /dev/rhdisk19 grid asmadmin 11: 524288 MB /dev/rhdisk20 grid asmadmin 12: 524288 MB /dev/rhdisk21 grid asmadmin 13: 524288 MB /dev/rhdisk22 grid asmadmin 14: 524288 MB /dev/rhdisk23 grid asmadmin 15: 524288 MB /dev/rhdisk24 grid asmadmin 16: 524288 MB /dev/rhdisk25 grid asmadmin 17: 524288 MB /dev/rhdisk26 grid asmadmin 18: 524288 MB /dev/rhdisk27 grid asmadmin 19: 524288 MB /dev/rhdisk28 grid asmadmin 20: 524288 MB /dev/rhdisk29 grid asmadmin 21: 524288 MB /dev/rhdisk30 grid asmadmin 22: 524288 MB /dev/rhdisk31 grid asmadmin 23: 102400 MB /dev/rhdisk32 grid asmadmin 24: 2048 MB /dev/rhdisk33 grid asmadmin 25: 2048 MB /dev/rhdisk34 grid asmadmin 26: 2048 MB /dev/rhdisk35 grid asmadmin 27: 2048 MB /dev/rhdisk36 grid asmadmin 28: 2048 MB /dev/rhdisk37 grid asmadmin 29: 524288 MB /dev/rhdisk4 grid asmadmin 30: 524288 MB /dev/rhdisk5 grid asmadmin 31: 524288 MB /dev/rhdisk6 grid asmadmin 32: 524288 MB /dev/rhdisk7 grid asmadmin 33: 524288 MB /dev/rhdisk8 grid asmadmin 34: 524288 MB /dev/rhdisk9 grid asmadmin -------------------------------------------------------------------------------- ORACLE_SID ORACLE_HOME ================================================================================ oracle@anbob1:/home/oracle> which oracle /oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/oracle oracle@anbob1:/home/oracle> ls -l /oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/oracle -rwsr-s--x 1 oracle oinstall 639389170 Jun 18 18:07 /oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/oracle oracle@anbob1:/home/oracle> exit root@anbob1:/> cd /oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/ root@anbob1:/oracle/app/oracle/product/12.2.0.1/dbhome_1/bin> chgrp asmadmin oracle root@anbob1:/oracle/app/oracle/product/12.2.0.1/dbhome_1/bin> ls -l oracle -rwsr-s--x 1 oracle asmadmin 639389170 Jun 18 18:07 oracle root@anbob1:/oracle/app/oracle/product/12.2.0.1/dbhome_1/bin> su - oracle oracle@anbob1:/home/oracle> sqlplus / as sysdba SQL*Plus: Release 12.2.0.1.0 Production on Tue Jun 18 18:47:36 2019 Copyright (c) 1982, 2017, Oracle. All rights reserved. Connected. SQL> shutdown abort ORACLE instance shut down. SQL> startup ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance ORACLE instance started. Total System Global Area 6.4425E+10 bytes Fixed Size 29967376 bytes Variable Size 3.2078E+10 bytes Database Buffers 3.2212E+10 bytes Redo Buffers 104247296 bytes Database mounted. Database opened.
Note:
安装补丁后oracle binary 执行文件的属组从asmadmin变成oinstall了，手动修改后恢复正常， 那为什么之前安装没有问题呢？
上面是在AIX上，这次换个SUSE平台试试
root@anbob1:/root> cd /oracle/app/oracle/product/12.2.0/db_1/bin/ root@anbob1:/root> ls -l /oracle/app/oracle/product/12.2.0/db_1/bin/oracle -rwsr-s--x 1 oracle oinstall 408547136 Jun 19 16:02 /oracle/app/oracle/product/12.2.0/db_1/bin/oracle root@anbob1:/oracle/app/oracle/product/12.2.0/db_1/bin> chgrp asmadmin oracle root@anbob1:/oracle/app/oracle/product/12.2.0/db_1/bin> ls -l oracle -rwsr-s--x 1 oracle asmadmin 408547136 Jun 18 19:23 oracle -- 同样要检查S粘滞位, 有些OS执行chgrp后可能会丢失S粘滞位 root@anbob1:/oracle/app/oracle/product/12.2.0/db_1/bin> chmod 6751 oracle root@anbob1:/oracle/app/oracle/product/12.2.0/db_1/bin> ls -l oracle -rwsr-s--x 1 oracle asmadmin 408547136 Jun 18 19:23 oracle
SUSE平台发现一样的问题。基于对于手动修改，oracle还提供了另一个工具setasmgidwrap 在$GRID_HOME/bin目录下，它可以自动判断OS ASM组，并修改oracle执行文件的属组.
oracle@anbob1:/home/oracle> ls -l /oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/oracle -rwsr-s--x 1 oracle oinstall 639380520 Jun 20 14:32 /oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/oracle oracle@anbob1:/home/oracle> env|grep HOME GI_HOME=/oracle/app/12.2.0.1/grid HOME=/home/oracle ORACLE_HOME=/oracle/app/oracle/product/12.2.0.1/dbhome_1 oracle@anbob1:/home/oracle> $GI_HOME/bin/setasmgidwrap o=/oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/oracle oracle@anbob1:/home/oracle> ls -l /oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/oracle -rwsr-s--x 1 oracle asmadmin 639380520 Jun 20 14:32 /oracle/app/oracle/product/12.2.0.1/dbhome_1/bin/oracle
上次为什么安装相同补丁后，没有手动oracle执行修改权限同样也可以启动数据库呢？区别就在启动方式
–SUSE —
oracle@anbob1:/home/oracle/28423598> ls -l /oracle/app/oracle/product/12.2.0/db_1/bin/oracle -rwsr-s--x 1 oracle oinstall 408547136 Jun 19 16:42 /oracle/app/oracle/product/12.2.0/db_1/bin/oracle oracle@anbob1:/home/oracle/28423598> srvctl start instance -d stdorder -i order2 -o mount oracle@anbob1:/home/oracle/28423598> ls -l /oracle/app/oracle/product/12.2.0/db_1/bin/oracle -rwsr-s--x 1 oracle asmadmin 408547136 Jun 19 16:42 /oracle/app/oracle/product/12.2.0/db_1/bin/oracle racle@anbob1:/home/oracle/28423598> ss SQL*Plus: Release 12.2.0.1.0 Production on Wed Jun 19 16:43:40 2019 Copyright (c) 1982, 2016, Oracle. All rights reserved. Connected to: Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production SQL> shutdown immediate; ORA-01109: database not open Database dismounted. ORACLE instance shut down. SQL> exit Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production oracle@anbob1:/home/oracle/28423598> ss SQL*Plus: Release 12.2.0.1.0 Production on Wed Jun 19 16:44:30 2019 Copyright (c) 1982, 2016, Oracle. All rights reserved. Connected to an idle instance. SQL> startup mount ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance ORACLE instance started. Total System Global Area 1.9327E+11 bytes Fixed Size 29867600 bytes Variable Size 1.6052E+11 bytes Database Buffers 3.2212E+10 bytes Redo Buffers 507002880 bytes Database mounted.
Note:
用SRVCTL 启动实例，同样有可能会修改oracle文件的属组， 这里我用srvctl 启动一次实例后oracle执行文件从oinstall变成了asmadmin。 再次用sqlplus 启动就不再有问题了。
Conclusion:
安装PSU或安装one-off patch补丁时，在relink编译oracle执行文件后文件的group ownership属组可能会发生改变，对比可以和oracle 执行文件相同路径下的oracle0文件(上一版oracle的mv)查看。 当oracle用户和oracle执行文件都没有OS ASM属组如asmadmin时，会无法读取ASM disk.
解决方法，手动修改或使用setasmgidwrap 修改；使用srvctl 启动实例，让oracle自动修改，但如果srvctl未修改，同样需要手动修改。
注意：
使用srvctl start instance 时如果没有修改为正确的权限，很可能是srvctl add database有不正确的地方，我这有一套库发现没有srvctl add database 时没有指定disk group ，所以无法根据ASM DISKGROUP 权限修改oracle group ownership.
微信扫一扫，打赏作者吧～
对不起，这篇文章暂时关闭评论。