after OS reboot ,11g RAC asm not automatic startup ORA-27102
昨安装成功了11G 203 RAC的GIRD,当时查看正常,今天重启了一下操作系统linux,发现grid没启动,crs_stst -t 查看集群件资源无法查看,crsctl check crs 检查Cluster Ready Services没启,
查看日志
cd ORACLE_HOME/log/[grid@znode1 znode1]$ vi + alertznode1.log 2012-07-05 09:21:10.275 [gpnpd(4031)]CRS-2329:GPNPD on node znode1 shutdown. 2012-07-05 09:24:19.367 [ohasd(4507)]CRS-2112:The OLR service started on node znode1. 2012-07-05 09:24:19.519 [ohasd(4507)]CRS-1301:Oracle High Availability Service started on node znode1. 2012-07-05 09:24:19.549 [ohasd(4507)]CRS-8017:location: /etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors occurred 2012-07-05 09:24:37.184 [gpnpd(5167)]CRS-2328:GPNPD started on node znode1. 2012-07-05 09:24:44.567 [ohasd(4507)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE 2012-07-05 09:24:44.920 [cssd(5397)]CRS-1713:CSSD daemon is started in clustered mode 2012-07-05 09:25:01.577 [cssd(5397)]CRS-1707:Lease acquisition for node znode1 number 1 completed 2012-07-05 09:25:02.919 [cssd(5397)]CRS-1605:CSSD voting file is online: /dev/raw/raw2; details in /u01/app/11.2.0/grid/log/znode1/cssd/ocssd.log. 2012-07-05 09:25:02.947 [cssd(5397)]CRS-1605:CSSD voting file is online: /dev/raw/raw3; details in /u01/app/11.2.0/grid/log/znode1/cssd/ocssd.log. 2012-07-05 09:25:02.984 [cssd(5397)]CRS-1605:CSSD voting file is online: /dev/raw/raw1; details in /u01/app/11.2.0/grid/log/znode1/cssd/ocssd.log. 2012-07-05 09:25:07.733 [cssd(5397)]CRS-1601:CSSD Reconfiguration complete. Active nodes are znode1 znode2 . 2012-07-05 09:25:10.197 [ctssd(5534)]CRS-2401:The Cluster Time Synchronization Service started on host znode1. 2012-07-05 09:25:10.197 [ctssd(5534)]CRS-2407:The new Cluster Time Synchronization Service reference node is host znode2. 2012-07-05 09:25:25.352 [ctssd(5534)]CRS-2408:The clock on host znode1 has been updated by the Cluster Time Synchronization Service to be synchronous with the mean cluster time. 2012-07-05 09:25:36.701 [ohasd(4507)]CRS-2807:Resource 'ora.asm' failed to start automatically. 2012-07-05 09:25:36.701 [ohasd(4507)]CRS-2807:Resource 'ora.crsd' failed to start automatically. 2012-07-05 10:00:15.382 [ctssd(5534)]CRS-2408:The clock on host znode1 has been updated by the Cluster Time Synchronization Service to be synchronous with the mean cluster time. 2012-07-05 10:21:51.427 [client(6963)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/znode1/client/crsctl_oracle.log. 2012-07-05 10:22:54.060 [client(7091)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/znode1/client/crsctl_oracle.log.
手动启asm实例
[grid@znode1 ohasd]$ sqlplus / as sysasmSQL*Plus: Release 11.2.0.3.0 Production on Thu Jul 5 10:40:01 2012
Copyright (c) 1982, 2011, Oracle. All rights reserved.Connected to an idle instance.
SQL> startup nomount
ORA-27102: out of memory
Linux Error: 22: Invalid argument[grid@znode1 ohasd]$ free
total used free shared buffers cached
Mem: 2061864 1560480 501384 0 39344 981324
-/+ buffers/cache: 539812 1522052
Swap: 4192956 0 4192956就一ASM不应该os内存不足呀
怀疑是kernel 参数问题检查
/etc/sysctl.conf发现一可疑处
# Oracle-Validated setting for kernel.shmmax is 4398046511104 on x86_64 and 4294967295 on i386 architecture. Refer Note id 567506.1
kernel.shmmax = 4398046511104因为我是没装validated,从其实机器copy的配置文件,并且我的本机是oel5.8 32bit,所以这个参数太大。
修改为kernel.shmmax=4294967295os reboot。
[root@znode1 ~]# ps -ef|grep grid
root 4510 1 3 11:13 ? 00:00:02 /u01/app/11.2.0/grid/bin/ohasd.bin reboot
grid 5058 1 0 11:13 ? 00:00:00 /u01/app/11.2.0/grid/bin/oraagent.bin
grid 5072 1 0 11:13 ? 00:00:00 /u01/app/11.2.0/grid/bin/mdnsd.bin
grid 5084 1 0 11:13 ? 00:00:00 /u01/app/11.2.0/grid/bin/gpnpd.bin
grid 5098 1 1 11:13 ? 00:00:00 /u01/app/11.2.0/grid/bin/gipcd.bin
root 5099 1 0 11:13 ? 00:00:00 /u01/app/11.2.0/grid/bin/orarootagent.bin
root 5118 1 2 11:13 ? 00:00:01 /u01/app/11.2.0/grid/bin/osysmond.bin
root 5133 1 0 11:13 ? 00:00:00 /u01/app/11.2.0/grid/bin/cssdmonitor
root 5154 1 0 11:13 ? 00:00:00 /u01/app/11.2.0/grid/bin/cssdagent
grid 5168 1 2 11:13 ? 00:00:01 /u01/app/11.2.0/grid/bin/ocssd.bin
root 5263 1 0 11:13 ? 00:00:00 /u01/app/11.2.0/grid/bin/octssd.bin reboot
grid 5290 1 0 11:14 ? 00:00:00 /u01/app/11.2.0/grid/bin/evmd.bin
grid 5303 1 26 11:14 ? 00:00:02 oracle+ASM1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
root 5306 1 10 11:14 ? 00:00:01 /u01/app/11.2.0/grid/bin/ologgerd -M -d /u01/app/11.2.0/grid/crf/db/znode1
grid 5400 1 1 11:14 ? 00:00:00 asm_pmon_+ASM1
grid 5404 1 1 11:14 ? 00:00:00 asm_psp0_+ASM1
grid 5408 1 3 11:14 ? 00:00:00 asm_vktm_+ASM1
grid 5414 1 1 11:14 ? 00:00:00 asm_gen0_+ASM1
grid 5418 1 3 11:14 ? 00:00:00 asm_diag_+ASM1
grid 5422 1 2 11:14 ? 00:00:00 asm_ping_+ASM1
grid 5426 1 5 11:14 ? 00:00:00 asm_dia0_+ASM1
grid 5430 1 6 11:14 ? 00:00:00 asm_lmon_+ASM1
grid 5434 1 5 11:14 ? 00:00:00 asm_lmd0_+ASM1
grid 5438 1 5 11:14 ? 00:00:00 asm_lms0_+ASM1
grid 5444 1 2 11:14 ? 00:00:00 asm_lmhb_+ASM1
grid 5448 1 2 11:14 ? 00:00:00 asm_mman_+ASM1
grid 5452 1 3 11:14 ? 00:00:00 asm_dbw0_+ASM1
grid 5456 1 2 11:14 ? 00:00:00 asm_lgwr_+ASM1
grid 5460 1 3 11:14 ? 00:00:00 asm_ckpt_+ASM1
grid 5464 1 2 11:14 ? 00:00:00 asm_smon_+ASM1
grid 5468 1 5 11:14 ? 00:00:00 asm_rbal_+ASM1
grid 5472 1 5 11:14 ? 00:00:00 asm_gmon_+ASM1
grid 5476 1 5 11:14 ? 00:00:00 asm_mmon_+ASM1
grid 5480 1 5 11:14 ? 00:00:00 asm_mmnl_+ASM1
grid 5484 1 0 11:14 ? 00:00:00 asm_lck0_+ASM1
root 5488 5327 0 11:14 pts/1 00:00:00 grep grid[grid@znode1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online一切正常!
对不起,这篇文章暂时关闭评论。