今天處理一個oracle優化管理工具oms啓動異常的問題。oms問題處理之前，oms所在服務器上的一套asm和db所用
的審計目錄文件系統inode耗盡導致asm和db異常宕機，經清理審計目錄的日誌後asm和db正常啓動，但是oms啓動異常。
一、OMS啓動異常分析
1、OMS啓動異常信息
[oracle@orcl bin]$ ./emctl start oms
Oracle Enterprise Manager Cloud Control 12c Release 4
Copyright (c) 1996, 2014 Oracle Corporation. All rights reserved.
Starting Oracle Management Server...
Starting WebTier...
WebTier Successfully Started
Oracle Management Server Could Not Be Started
Check EM Server log file for details: /u01/app/oracle/oms/domains/GCDomain/servers/ORCL_OMS1/logs/ORCL_OMS1.out
Oracle Management Server is Down
[oracle@orcl bin]$
2、查看ORCL_OMS1.out日誌
Mar 19, 2020 5:55:44 PM oracle.security.jps.internal.audit.AuditServiceImpl registerInternal
WARNING: Cannot register to audit service for component "JPS".
Mar 19, 2020 5:55:44 PM oracle.security.jps.internal.policystore.ldap.LdapPolicyStore initial
INFO: Container is not ready at the point null
<Mar 19, 2020 5:55:53 PM CST> <Error> <NodeManager> <BEA-300040> <I/O error while writing node manager status change message.
java.io.IOException: No space left on device
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.checkAndCreate(File.java:1705)
at java.io.File.createTempFile0(File.java:1726)
at java.io.File.createTempFile(File.java:1803)
at weblogic.nodemanager.util.ConcurrentUnixFile.write(ConcurrentUnixFile.java:38)
Truncated. see log file for complete stacktrace
>
<Mar 19, 2020 5:55:54 PM> <FINEST> <NodeManager> <Waiting for the process to die: 6906>
<Mar 19, 2020 5:55:54 PM> <INFO> <NodeManager> <Server failed during startup so will not be restarted>
<Mar 19, 2020 5:55:54 PM> <FINEST> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>
3、根據.out日誌提示，跟之前asm和db啓動異常報錯一致，也是之前oracle數據庫審計日誌過多導致文件系統inode耗盡
其他進程無法寫入導致。
二、問題處理
1、對於oms來說，單純的重啓oms無法解決問題，因爲nodemanager進程未正常停止
2、使用ps -ef|grep java，把oms相關的java進程殺掉kill -9 java_process_number
3、再次嘗試啓動oms進程，成功啓動
