今天，偶然發現一數據庫某個oracle進程佔用大量的CPU，非大數據量，僅64條記錄，沒有觸發器，邏輯讀寫很低，查詢等待，發現
SQL> select s.sid,sw.event,sw.wait_time,sw.state
2 from v$session s,v$process p,v$session_wait sw
3 where s.PADDR = p.ADDR
4 and sw.SID = s.sid
5 and p.SPID=237780;
SID EVENT
---------- ----------------------------------------------------------------
WAIT_TIME STATE
---------- -------------------
503 kksfbc child completion
5 WAITED KNOWN TIME
有一種說法是bug，1、Bug 6795880，相關的doc id爲：6795880.8
<Bug 6795880 – Session spins / OERI after ‘kksfbc child completion’ wait – superceded [ID 6795880.8]>
我metalink到期了。沒法查了。
Bug的描述是在發生’kksfbc child completion’等待事件後會話陷入無休止的自旋(spins)中，這種自旋(spins)發生在由堆棧調用(stack call)kksSearchChildList->kkshgnc陷入對kksSearchChildList函數的無限循環中。
直接影響版本11.1.0.6、10.2.0.3和10.2.0.4。
| A session may go into an infinite spin just after a wait for 'kksfbc child completion'. The spin occurs with a stack including kksSearchChildList -> kkshgnc where kksSearchChildList loops forever. This problem can also lead to internal error such as any of ORA-600 [kksSearchChildList1], ORA-600 [kksSearchChildList2] ORA-600 [kksSearchChildList3], ORA-600 [kkshgnc-nextchild] Note: Fixes for this bug in 10g and 11gR1 are disabled by default. To enable this fix you must explicitly set the following parameter for instance startup: "_cursor_features_enabled" = 10 |
過了一會，又變成另外一種等待
SQL> select event,wait_time ,state from v$session_wait where sid=503;
EVENT WAIT_TIME
---------------------------------------------------------------- ----------
STATE
-------------------
ksdxexeotherwait -1
WAITED SHORT TIME
有人說，這也是bug，由new child listener process forked 引起
- OR -
Apply Patch 4518443 for the problem (if a patch is available)
- OR -
As a workaround, the following parameter can be added to listener.ora
SUBSCRIBE_FOR_NODE_DOWN_EVENT_<listener_name>=OFF
Where <listener_name> should be replaced with the actual listener name configured in the LISTENER.ORA file.
For example, if the listener name is LISTENER (default), the parameter would be:
SUBSCRIBE_FOR_NODE_DOWN_EVENT_LISTENER=OFF
This will prevent the listener from registering against ONS (Oracle Notification Services), which is the area affected by bug:4518443. For more information on ONS, please refer to eg. the Oracle10g Release 2 documentation ("Oracle Clusterware and Oracle Real Application Clusters Administration and Deployment Guide").
Please note, adding SUBSCRIBE_FOR_NODE_DOWN_EVENT_<listener_name> to listener.ora file on RAC, will mean that FAN (fast application notification) will not be possible. See Note 220970.1 RAC: Frequently Asked Questions for further information on FAN
session不會自動釋放連接，及時在庫一層kill掉，仍然佔用CPU，只好在kill -9 237780 在操作系統層殺掉，釋放CPU。
繼續關注，如果再次或者頻繁出現，或許真是bug，需要打補丁了。
過程中總結的：
SQL> oradebug setospid 3723390
SQL> oradebug unlimit;
Statement processed.
SQL> oradebug short_stack;
SQL> oradebug dump processstate 10;
Statement processed.
SQL> oradebug dump systemstate 266;
Statement processed.
awk -f ass109.awk /oracle/admin/budget/udump/budget_ora_791256.trc
sqlplus -prelim '/ as sysdba'
補充一句，
ftp://updates.oracle.com/8575528/p8575528_10.2.0.4_AIX64-5L.zip
下載8575528的補丁，然後修改參數"_cursor_features_enabled" = 10