java GC筆記

以hbase爲例：在hbase的配置文件路徑下，設置了GC log輸出路徑/app/hbase-config/hbase-env.sh

export HBASE_OPTS="-Xmx16384m -Xms16384m -Xmn8192m -XX:PermSize=160M -XX:MaxPermSize=160M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC  -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSCompactAtFullCollection -XX:CMSFullGCsBeforeCompaction=3  -XX:+CMSParallelRemarkEnabled -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime -XX:PrintFLSStatistics=1 -Xloggc:/app/hbase/logs/hbase_gc.log $HBASE_OPTS"

可以查詢到log的相關信息

#jstat -gccause 79751 1000 1000 //獲取GC相關指標，表示每1000毫秒查詢一次79751垃圾收集狀況。
Usage: jstat -help|-options
       jstat -<option> [-t] [-h<lines>] <vmid> [<interval> [<count>]]
參數解釋：
Options — 選項，我們一般使用 -gcutil 查看gc情況
vmid    — VM的進程號，即當前運行的java進程號
interval– 間隔時間，單位爲秒或者毫秒
count   — 打印次數，如果缺省則打印無數次

    S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT    LGCC                 GCC
  0.00   8.98  11.16  69.83  12.33    156   50.788    16  280.103  330.891 unknown GCCause      No GC
  0.00   8.98  12.79  69.83  12.33    156   50.788    16  280.103  330.891 unknown GCCause      No GC
  0.00   8.98  14.94  69.83  12.33    156   50.788    16  280.103  330.891 unknown GCCause      No GC
  0.00   8.98  16.04  69.83  12.33    156   50.788    16  280.103  330.891 unknown GCCause      No GC
  0.00   8.98  17.98  69.83  12.33    156   50.788    16  280.103  330.891 unknown GCCause      No GC
  0.00   8.98  20.34  69.83  12.33    156   50.788    16  280.103  330.891 unknown GCCause      No GC
  0.00   8.98  21.83  69.83  12.33    156   50.788    16  280.103  330.891 unknown GCCause      No GC
  0.00   8.98  22.64  69.83  12.33    156   50.788    16  280.103  330.891 unknown GCCause      No GC

E列代表
O E-->O 新生代轉移到老代，Ｏ積累到80%就會觸發full gc

具體意思如下：
E（表示，Eden），代表這臺服務器的新生代Eden區使用了11.16的空間，
S0 S1 兩個Survivor區(S0,S1表示Survivor0、Survivor1),Survivor0裏面是空的，Survivor1佔了8.98%

老年代（O，表示old) 和永久帶（P，表示Permanent）分別使用了69.83% 和12.33%的空間。
程序運行以來共發生Minor GC(YGC ,表示young gc)156次，總耗時50.788秒；
發生Fulle GC （FGC，表示full GC ）16次，full GC總耗時（FGCT，表示full gc time）爲330.891秒，總的GC總耗時（GCT，表示GC Time）爲330.891秒
LGCC Cause of last Garbage Collection.
GCC Cause of current Garbage Collection.

Java 6 JDK輸出如下：
[hadoop@0321 logs]$ jmap -heap 29877

Attaching to process ID 29877, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.45-b01

using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 21474836480 (20480.0MB)
   NewSize          = 4294967296 (4096.0MB)
   MaxNewSize       = 4294967296 (4096.0MB)
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 67108864 (64.0MB)
   MaxPermSize      = 67108864 (64.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 3865509888 (3686.4375MB)
   used     = 481048576 (458.763671875MB)
   free     = 3384461312 (3227.673828125MB)
   12.444634470949257% used
Eden Space:
   capacity = 3436052480 (3276.875MB)
   used     = 481048576 (458.763671875MB)
   free     = 2955003904 (2818.111328125MB)
   14.00003576196834% used
From Space:
   capacity = 429457408 (409.5625MB)
   used     = 0 (0.0MB)
   free     = 429457408 (409.5625MB)
   0.0% used
To Space:
   capacity = 429457408 (409.5625MB)
   used     = 0 (0.0MB)
   free     = 429457408 (409.5625MB)
   0.0% used
concurrent mark-sweep generation:
   capacity = 12884901888 (12288.0MB)
   used     = 0 (0.0MB)
   free     = 12884901888 (12288.0MB)
   0.0% used
Perm Generation:
   capacity = 67108864 (64.0MB)
   used     = 21153816 (20.173851013183594MB)
   free     = 45955048 (43.826148986816406MB)
   31.521642208099365% used

Eden from to 3個加起來爲Young 區

[hbase@0321 ~]$ jps
10424 Jps
8229 HRegionServer

[hbase@0321 ~]$ jmap -heap 8229
Attaching to process ID 8229, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 24.71-b01

using thread-local object allocation.
Garbage-First (G1) GC with 6 thread(s)

Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 25769803776 (24576.0MB)
NewSize = 1363144 (1.2999954223632812MB)
MaxNewSize = 17592186044415 MB
OldSize = 5452592 (5.1999969482421875MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 16777216 (16.0MB)
MaxPermSize = 83886080 (80.0MB)
G1HeapRegionSize = 8388608 (8.0MB)

Heap Usage:
G1 Heap:
regions = 3072
capacity = 25769803776 (24576.0MB)
used = 20409483264 (19464.0MB)
free = 5360320512 (5112.0MB)
79.19921875% used
G1 Young Generation:
Eden Space:
regions = 89
capacity = 1182793728 (1128.0MB)
used = 746586112 (712.0MB)
free = 436207616 (416.0MB)
63.12056737588652% used
Survivor Space:
regions = 20
capacity = 167772160 (160.0MB)
used = 167772160 (160.0MB)
free = 0 (0.0MB)
100.0% used
G1 Old Generation:
regions = 2324
capacity = 24419237888 (23288.0MB)
used = 19495124992 (18592.0MB)
free = 4924112896 (4696.0MB)
79.83510821023704% used
Perm Generation:
capacity = 50331648 (48.0MB)
used = 42303832 (40.344078063964844MB)
free = 8027816 (7.655921936035156MB)
84.05016263326009% used

G1 GC筆記：

關於最大gc停頓時間
在做minor gc和mixed gc時，收集器會在內存中維繫一個remembered set，這個set包含了heap中所有對象的引用，用以確定哪些可以被回收。
每次gc時，g1會通過一個預測模型來計算每個region進行回收的時間，從而從中選出停頓時間在MaxGCPauseMillis之內的region進行垃圾回收

關於mixed gc

mixed gc是一種既回收young區，也回收old區的垃圾回收方式，它觸發的條件是-XX:InitiatingHeapOccupancyPercent，mixed gc的目的就是爲了延遲full gc的產生

同理YGC指的是回收young的垃圾回收方式

Full GC 指的是回收Old去的垃圾回收方式，因爲old區比較大，導致GC時間比較長，並且在GC期間java進程停止對外相應，並且自身也不對外響應，導致regionserver無法向zookeeper註冊心跳信息，超過zookeeper （ <name>zookeeper.session.timeout</name>） session時間，就會被zookeeper置爲dead。

Hbase G1參數調整：

待修改的參數，修改GC的參數需要重啓集羣，安排在下次重啓。

-XX:ConcGCThreads=12併發標記的執行線程數 =================== 測試數據ConcGCThreads 不能大於-XX:ParallelGCThreads，否則會報錯
-XX:InitiatingHeapOccupancyPercent=60 堆佔用了多少的時候就觸發GC，默認爲45
-XX:ParallelGCThreads=12 30

[hbase@0321 ~]$ jstat -gccapacity 61540
NGCMN NGCMX NGC S0C S1C EC OGCMN OGCMX OGC OC PGCMN PGCMX PGC PC YGC FGC
0.0 52428800.0 2490368.0 0.0 294912.0 2195456.0 0.0 52428800.0 44695552.0 44695552.0 16384.0 81920.0 49152.0 49152.0 2 0

GC 常見命令：

[hbase@ 0321 ~]$ jmap -heap 61540

Attaching to process ID 61540, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 24.71-b01

using thread-local object allocation.
Garbage-First (G1) GC with 30 thread(s)

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 53687091200 (51200.0MB)
   NewSize          = 1363144 (1.2999954223632812MB)
   MaxNewSize       = 17592186044415 MB
   OldSize          = 5452592 (5.1999969482421875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 16777216 (16.0MB)
   MaxPermSize      = 83886080 (80.0MB)
   G1HeapRegionSize = 16777216 (16.0MB)

Heap Usage:
G1 Heap:
   regions  = 2880
   capacity = 48318382080 (46080.0MB)
   used     = 2519304864 (2402.5963439941406MB)
   free     = 45799077216 (43677.40365600586MB)
   5.213967760403951% used
G1 Young Generation:
Eden Space:
   regions  = 7
   capacity = 2248146944 (2144.0MB)
   used     = 117440512 (112.0MB)
   free     = 2130706432 (2032.0MB)
   5.223880597014926% used
Survivor Space:
   regions  = 18
   capacity = 301989888 (288.0MB)
   used     = 301989888 (288.0MB)
   free     = 0 (0.0MB)
   100.0% used
G1 Old Generation:
   regions  = 126
   capacity = 45768245248 (43648.0MB)
   used     = 2099874464 (2002.5963439941406MB)
   free     = 43668370784 (41645.40365600586MB)
   4.588059805705051% used
Perm Generation:
   capacity = 50331648 (48.0MB)
   used     = 42610464 (40.636505126953125MB)
   free     = 7721184 (7.363494873046875MB)
   84.65938568115234% used

13018 interned Strings occupying 1388240 bytes.

[hbase@0321 ~]$ jstat -gccause -h 10 61540 1000 1000

  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT    LGCC                 GCC
  0.00 100.00  92.54  20.26  85.06     16    1.794     0    0.000    1.794 G1 Evacuation Pause  No GC
  0.00 100.00  92.54  20.26  85.06     16    1.794     0    0.000    1.794 G1 Evacuation Pause  No GC
  0.00 100.00  92.54  20.26  85.06     16    1.794     0    0.000    1.794 G1 Evacuation Pause  No GC
  0.00 100.00  93.28  20.26  85.06     16    1.794     0    0.000    1.794 G1 Evacuation Pause  No GC
  0.00 100.00  93.28  20.26  85.06     16    1.794     0    0.000    1.794 G1 Evacuation Pause  No GC
  0.00 100.00  93.28  20.26  85.06     16    1.794     0    0.000    1.794 G1 Evacuation Pause  No GC
  0.00 100.00   1.42  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   1.42  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   3.55  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   3.55  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT    LGCC                 GCC
  0.00 100.00   3.55  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   3.55  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   3.55  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   3.55  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   3.55  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   3.55  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT    LGCC                 GCC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC
  0.00 100.00   4.26  21.24  85.06     17    1.877     0    0.000    1.877 G1 Evacuation Pause  No GC

#jstat -gcutil 27912 1s 50

G1 GC筆記：

關於mixed gc

Hbase G1參數調整：

GC 常見命令：

hadoop的5個進程解釋

ansible部署

java GC筆記

記一次https報錯

windows python MySQL-python安裝過程

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結