Android內存優化(一)之Java層內存泄露監測工具原理（Leakcanary）

近期時間相對寬裕一些,把相關的知識點整理下放到博客~
封裝的Java層內存泄露監測工具主要基於開源的leakcanary project，下面對Leakcanary原理淺析

Leakcanary簡介

Leakcanary工具是用來檢測Java層內存泄露的工具,嚴格的說是檢測Activty的內存泄露(監測的是Activity的onDestroy方法,後面會提這一點),能幫助我們發現很多隱藏的內存問題,降低應用內存泄露及OOM的概率.

什麼是內存泄露?

內存泄漏（Memory Leak）是指程序中己動態分配的堆內存由於某種原因程序未釋放或無法釋放，造成系統內存的浪費，導致程序運行速度減慢甚至系統崩潰等嚴重後果。

在介紹原理前留幾個問題,相信後面看完淺析就會有答案~
Leakcanary能檢測什麼樣的泄露問題?
能不能發現所有的內存泄露?
怎麼集成Leakcanary?
如何自定義二次開發?
深一點的問題:一個泄露點,是不是每次泄露路徑都是一樣的?

原理介紹

初始化部分

Leakcanary的入口函數是LeakCanary.install()方法,調用時機是在Application.onCreate()方法中,集成方式有多種
1:可以把LeakCanary.install()方法直接放在Application的onCreate()方法中,然後重新打包生成apk後開始測試,這種方式每個App都需要重新打包後測試,當App較多且升級較爲頻繁時時比較繁瑣;
2:把LeakCanary.install()方法放到系統中,通過系統屬性來控制,默認不開啓,這種方式需要改動系統源碼造成系統較爲冗餘,而且應用每次創建時都會判斷屬性,但相對於方案1,其實比較省力了;
3:採用hook方式,hook Application.onCreate方法,測試時自動帶着LeakCanary.install(),這種方案相對來講對於系統的方式比較友好,測試的代碼跟系統運行是分離的,也比較頭疼,因爲每個版本都需要處理hook的問題~
本人選的是方式3,集成方案因人而異,根據不同需求選擇不同的集成方式~
看一下LeakCanary.install()後的主要邏輯:

  public static RefWatcher install(Application application) {
    return refWatcher(application).listenerServiceClass(DisplayLeakService.class)
        .excludedRefs(AndroidExcludedRefs.createAppDefaults().build())
        .buildAndInstall();
  }

install方法中包含了3個方法,主要通過AndroidRefWatcherBuilder這個輔助類來實現初始化功能

  /** Builder to create a customized {@link RefWatcher} with appropriate Android defaults. */
  public static AndroidRefWatcherBuilder refWatcher(Context context) {
    return new AndroidRefWatcherBuilder(context);
  }

listenerServiceClass方法主要完成結果分析的服務綁定功能,excludedRefs方法主要完成排除可以忽略的泄露路徑(有些泄露點跟系統實現相關,需要排除掉),內容在AndroidExcludedRefs中有枚舉,buildAndInstall完成主要的初始化工作~

  public RefWatcher buildAndInstall() {
    RefWatcher refWatcher = build();
    if (refWatcher != DISABLED) {
      LeakCanary.enableDisplayLeakActivity(context);
      ActivityRefWatcher.install((Application) context, refWatcher);
    }
    return refWatcher;
  }

buildAndInstall方法中實例化RefWatcher,並完成ActivityRefWatcher的相應初始化,主角慢慢登場了~

  public static void install(Application application, RefWatcher refWatcher) {
    new ActivityRefWatcher(application, refWatcher).watchActivities();
  }
  public void watchActivities() {
    // Make sure you don't get installed twice.
    stopWatchingActivities();
    application.registerActivityLifecycleCallbacks(lifecycleCallbacks);
  }

在install方法中,會去watchActivities,註冊Activity生命週期回調~

  private final Application.ActivityLifecycleCallbacks lifecycleCallbacks =
      new Application.ActivityLifecycleCallbacks() {
        @Override public void onActivityCreated(Activity activity, Bundle savedInstanceState) {
        }

        @Override public void onActivityStarted(Activity activity) {
        }

        @Override public void onActivityResumed(Activity activity) {
        }

        @Override public void onActivityPaused(Activity activity) {
        }

        @Override public void onActivityStopped(Activity activity) {
        }

        @Override public void onActivitySaveInstanceState(Activity activity, Bundle outState) {
        }

        @Override public void onActivityDestroyed(Activity activity) {
          ActivityRefWatcher.this.onActivityDestroyed(activity);
        }
      };

  void onActivityDestroyed(Activity activity) {
    refWatcher.watch(activity);
  }

這個refWatcher是在RefWatcher refWatcher = build()時創建的,到這初始化基本完成,運行場景差不多是當某個Activity執行onDestroy方法時,會回調refWatcher.watch方法,在此方法中判斷是否有內存泄漏
在這裏補充一句:
onActivityDestroyed方法是在Application.DispatchActivityDestroyed方法中回調的,DispatchActivityDestroyed又是在Activity.onDestroy方法中回調的,也就是說當某個Activity執行super.onDestroy時,就會執行註冊的生命週期回調onActivityDestroyed,開始watch

    protected void onDestroy() {
            .......
            getApplication().dispatchActivityDestroyed(this);
            .......
    }

怎麼確定是泄露的?

看下RefWatcher的watch方法:

  public void watch(Object watchedReference) {
    watch(watchedReference, "");
  }
  public void watch(Object watchedReference, String referenceName) {
    if (this == DISABLED) {
      return;
    }
    checkNotNull(watchedReference, "watchedReference");
    checkNotNull(referenceName, "referenceName");
    final long watchStartNanoTime = System.nanoTime();
    String key = UUID.randomUUID().toString();
    retainedKeys.add(key);
    final KeyedWeakReference reference =
        new KeyedWeakReference(watchedReference, key, referenceName, queue);

    ensureGoneAsync(watchStartNanoTime, reference);
  }

watchedReference是傳下來的Activity引用,key是隨機生成的唯一字符串,referenceName是”“,queue是創建RefWatcher時初始化的ReferenceQueue
這個地方需要一點WeakReference和ReferenceQueue的基礎,每次WeakReference所指向的對象被GC後，這個弱引用都會被放入這個與之相關聯的ReferenceQueue隊列中(Reference源碼),如果我們期望某個對象被回收,那麼在預期時間後依然沒出現在ReferenceQueue中,此時就可以判斷是有泄露,後面的判斷與分析也是基於此邏輯,我感覺,這應該Leakcanary的核心原理~

  RefWatcher(WatchExecutor watchExecutor, DebuggerControl debuggerControl, GcTrigger gcTrigger,
      HeapDumper heapDumper, HeapDump.Listener heapdumpListener, ExcludedRefs excludedRefs) {
    this.watchExecutor = checkNotNull(watchExecutor, "watchExecutor");
    this.debuggerControl = checkNotNull(debuggerControl, "debuggerControl");
    this.gcTrigger = checkNotNull(gcTrigger, "gcTrigger");
    this.heapDumper = checkNotNull(heapDumper, "heapDumper");
    this.heapdumpListener = checkNotNull(heapdumpListener, "heapdumpListener");
    this.excludedRefs = checkNotNull(excludedRefs, "excludedRefs");
    retainedKeys = new CopyOnWriteArraySet<>();
    queue = new ReferenceQueue<>();
  }

解釋下這幾個成員變量:
watchExecutor: 執行內存泄露檢測的異步線程executor,是AndroidWatchExecutor的實例
debuggerControl ：用於查詢是否正在調試中，調試中不會執行內存泄露檢測
queue ： 用於判斷弱引用所持有的對象是否已被GC。
gcTrigger： 用於在判斷內存泄露之前，再給一次GC的機會
headDumper: 用於在產生內存泄露室執行dump 內存heap
heapdumpListener: 用於分析前面產生的dump文件，找到內存泄露的原因
excludedRefs: 用於排除某些系統bug導致的內存泄露
retainedKeys： 持有那些待檢測以及產生內存泄露的引用的key。

然後將其封裝至KeyedWeakReference,核心方法是ensureGoneAsync,異步判斷過程,看下邏輯:

  private void ensureGoneAsync(final long watchStartNanoTime, final KeyedWeakReference reference) {
    watchExecutor.execute(new Retryable() {
      @Override public Retryable.Result run() {
        return ensureGone(reference, watchStartNanoTime);
      }
    });
  }

調用AndroidWatcherExecutor的execute方法,看下具體邏輯:

  @Override public void execute(Retryable retryable) {
    if (Looper.getMainLooper().getThread() == Thread.currentThread()) {
      waitForIdle(retryable, 0);
    } else {
      postWaitForIdle(retryable, 0);
    }
  }
  void postWaitForIdle(final Retryable retryable, final int failedAttempts) {
    mainHandler.post(new Runnable() {
      @Override public void run() {
        waitForIdle(retryable, failedAttempts);
      }
    });
  }

  void waitForIdle(final Retryable retryable, final int failedAttempts) {
    // This needs to be called from the main thread.
    Looper.myQueue().addIdleHandler(new MessageQueue.IdleHandler() {
      @Override public boolean queueIdle() {
        postToBackgroundWithDelay(retryable, failedAttempts);
        return false;
      }
    });
  }

  void postToBackgroundWithDelay(final Retryable retryable, final int failedAttempts) {
    long exponentialBackoffFactor = (long) Math.min(Math.pow(2, failedAttempts), maxBackoffFactor);
    long delayMillis = initialDelayMillis * exponentialBackoffFactor;
    backgroundHandler.postDelayed(new Runnable() {
      @Override public void run() {
        Retryable.Result result = retryable.run();
        if (result == RETRY) {
          postWaitForIdle(retryable, failedAttempts + 1);
        }
      }
    }, delayMillis);
  }

在AndroidRefWatcherBuilder中定義了默認時間,5s
private static final long DEFAULT_WATCH_DELAY_MILLIS = SECONDS.toMillis(5);
從代碼邏輯中可知,如果是主線程,則在主線程空閒的時候延時5s處理泄露問題,如果不是主線程,則向主線程post消息,讓主線程去處理,出現成空心啊的時候延時5s處理泄露問題,看下主線程需要處理些啥?

//核心方法  
  Retryable.Result ensureGone(final KeyedWeakReference reference, final long watchStartNanoTime) {
    long gcStartNanoTime = System.nanoTime();
    long watchDurationMs = NANOSECONDS.toMillis(gcStartNanoTime - watchStartNanoTime);//

    removeWeaklyReachableReferences(); //檢查一次弱引用是否已回收

    if (debuggerControl.isDebuggerAttached()) {
      // The debugger can create false leaks.
      return RETRY;
    }
    if (gone(reference)) { //如果回收了,activity沒有泄露
      return DONE;
    }
    gcTrigger.runGc();   //還是沒有回收,觸發GC一次
    removeWeaklyReachableReferences(); //再檢查一次弱引用是否已回收
    if (!gone(reference)) { //還沒回收,懷疑是內存泄露,dump內存快照hprof,再做分析
      long startDumpHeap = System.nanoTime();
      long gcDurationMs = NANOSECONDS.toMillis(startDumpHeap - gcStartNanoTime);

      File heapDumpFile = heapDumper.dumpHeap();
      if (heapDumpFile == RETRY_LATER) {
        // Could not dump the heap.
        return RETRY;
      }
      long heapDumpDurationMs = NANOSECONDS.toMillis(System.nanoTime() - startDumpHeap);
      heapdumpListener.analyze(
          new HeapDump(heapDumpFile, reference.key, reference.name, excludedRefs, watchDurationMs,
              gcDurationMs, heapDumpDurationMs));
    }
    return DONE;
  }

看下removeWeaklyReachableReferences具體實現:

  private void removeWeaklyReachableReferences() {
    // WeakReferences are enqueued as soon as the object to which they point to becomes weakly
    // reachable. This is before finalization or garbage collection has actually happened.
    KeyedWeakReference ref;
    while ((ref = (KeyedWeakReference) queue.poll()) != null) {
      retainedKeys.remove(ref.key);
    }
  }

上面提到過,如果一個WeakReference所指向的對象被GC了,那麼WeakReference會出現在WeakReference對應的ReferenceQueue中,而這個地方就是把已經GC對象的WeakReference所對應的key清除掉,那麼剩下的key所對應的弱引用所指向的對象就是發生泄露的對象~

  private boolean gone(KeyedWeakReference reference) {
    return !retainedKeys.contains(reference.key);
  }

所以gone方法比較好理解,只要看看KeyedWeakReference對應的key是否還在retainedKeys Set集合裏,如果依然在,說明可能有泄露
ensureGone方法中做了一層雙保險,如果發現沒被回收時,會主動觸發一次GC,再去看是否被回收了,如果還沒被回收,則去dump內存快照然後分析~

如何從內存快照中提取內存泄露信息的?

從ensureGone的最後可知,會執行 heapdumpListener.analyze方法,heapdumpListener是ServiceHeapDumpListener類型

  public AndroidRefWatcherBuilder listenerServiceClass(
      Class<? extends AbstractAnalysisResultService> listenerServiceClass) {
    return heapDumpListener(new ServiceHeapDumpListener(context, listenerServiceClass));
  }

看一下ServiceHeapDumpListener的analyze方法:

  @Override public void analyze(HeapDump heapDump) {
    checkNotNull(heapDump, "heapDump");
    HeapAnalyzerService.runAnalysis(context, heapDump, listenerServiceClass);
  }

主要執行了HeapAnalyzerService.runAnalysis方法,看下主要邏輯:

  public static void runAnalysis(Context context, HeapDump heapDump,
      Class<? extends AbstractAnalysisResultService> listenerServiceClass) {
    Intent intent = new Intent(context, HeapAnalyzerService.class);
    intent.putExtra(LISTENER_CLASS_EXTRA, listenerServiceClass.getName());
    intent.putExtra(HEAPDUMP_EXTRA, heapDump);
    context.startService(intent);
  }
  @Override protected void onHandleIntent(Intent intent) {
    if (intent == null) {
      CanaryLog.d("HeapAnalyzerService received a null intent, ignoring.");
      return;
    }
    String listenerClassName = intent.getStringExtra(LISTENER_CLASS_EXTRA);
    HeapDump heapDump = (HeapDump) intent.getSerializableExtra(HEAPDUMP_EXTRA);

    HeapAnalyzer heapAnalyzer = new HeapAnalyzer(heapDump.excludedRefs);

    AnalysisResult result = heapAnalyzer.checkForLeak(heapDump.heapDumpFile, heapDump.referenceKey);
    AbstractAnalysisResultService.sendResultToListener(this, listenerClassName, heapDump, result);
  }
}

創建了HeapAnalyzerService來處理heapDump,HeapAnalyzerService是個IntentService,創建後會回調onHandleIntent,從而找出最短路徑並展示,主要看下checkForLeak方法:

  public AnalysisResult checkForLeak(File heapDumpFile, String referenceKey) {
    long analysisStartNanoTime = System.nanoTime();

    if (!heapDumpFile.exists()) {
      Exception exception = new IllegalArgumentException("File does not exist: " + heapDumpFile);
      return failure(exception, since(analysisStartNanoTime));
    }

    try {
      HprofBuffer buffer = new MemoryMappedFileBuffer(heapDumpFile);
      HprofParser parser = new HprofParser(buffer);
      Snapshot snapshot = parser.parse();
      deduplicateGcRoots(snapshot);

      Instance leakingRef = findLeakingReference(referenceKey, snapshot);

      // False alarm, weak reference was cleared in between key check and heap dump.
      if (leakingRef == null) {
        return noLeak(since(analysisStartNanoTime));
      }

    } catch (Throwable e) {
      return failure(e, since(analysisStartNanoTime));
    }
  }

checkForLeak有幾個方法很關鍵,也是四個步驟:
1:HprofParser的parse方法,把hprof轉爲SnapShot對象,MAT工具解析hporof文件時也是用此方式,裏面應該主要是關係樹結構,各個引用鏈,我們可以隨意查看
2:HeapAnalyzer的deduplicateGcRoots方法,去除重複性的gc root,減小內存開銷,這個裏面邏輯不復雜~

  /**
   * Pruning duplicates reduces memory pressure from hprof bloat added in Marshmallow.
   */
  void deduplicateGcRoots(Snapshot snapshot) {
    // THashMap has a smaller memory footprint than HashMap.
    final THashMap<String, RootObj> uniqueRootMap = new THashMap<>();

    final Collection<RootObj> gcRoots = snapshot.getGCRoots();
    for (RootObj root : gcRoots) {
      String key = generateRootKey(root);
      if (!uniqueRootMap.containsKey(key)) {
        uniqueRootMap.put(key, root);
      }
    }

    // Repopulate snapshot with unique GC roots.
    gcRoots.clear();
    uniqueRootMap.forEach(new TObjectProcedure<String>() {
      @Override public boolean execute(String key) {
        return gcRoots.add(uniqueRootMap.get(key));
      }
    });
  }

3:HeapAnalyzer的findLeakingReference方法,主要是根據泄露的Key信息,從snapshot中查找到泄露的實例,方法不復雜
有人會有疑問,如果內存中有多個實例呢?或者有的實例是泄露,有的實例不是泄露的呢?請注意,key生成是與對象一一對應的,也就是通過key,就可以找到真正泄露的對象,其他的在內存中存在的實例不會處理

  private Instance findLeakingReference(String key, Snapshot snapshot) {
    ClassObj refClass = snapshot.findClass(KeyedWeakReference.class.getName());
    List<String> keysFound = new ArrayList<>();
    for (Instance instance : refClass.getInstancesList()) {
      List<ClassInstance.FieldValue> values = classInstanceValues(instance);
      String keyCandidate = asString(fieldValue(values, "key"));
      if (keyCandidate.equals(key)) {
        return fieldValue(values, "referent");
      }
      keysFound.add(keyCandidate);
    }
    throw new IllegalStateException(
        "Could not find weak reference with key " + key + " in " + keysFound);
  }

4:HeapAnalyzer的findLeakTrace方法,關鍵方法,主要工作是計算到GC ROOT的最短路徑,並確認是否是泄露,如果確定是泄露,生成泄露的引用鏈.邏輯相對複雜一些,在代碼里加些註釋

  private AnalysisResult findLeakTrace(long analysisStartNanoTime, Snapshot snapshot,
      Instance leakingRef) {

    ShortestPathFinder pathFinder = new ShortestPathFinder(excludedRefs); //從字面意思也能理解,主要負責生成泄漏點到GC ROOT的最短路徑,排除掉系統性問題,這些不會出現在最短路徑上
    ShortestPathFinder.Result result = pathFinder.findPath(snapshot, leakingRef); //開始查找最短路徑,這個方法很複雜,基本可總結爲採用廣度優先算法,看是否可達,我在附件補充下這個方法吧,感興趣的可以看下~

    // False alarm, no strong reference path to GC Roots.
    if (result.leakingNode == null) {
      return noLeak(since(analysisStartNanoTime));
    }

    LeakTrace leakTrace = buildLeakTrace(result.leakingNode); //將最短路徑轉換爲需要顯示的LeakTrace對象,這個對象中包括了一個由路徑上各個節點LeakTraceElement組成的鏈表，代表了檢查到的最短泄漏路徑

    String className = leakingRef.getClassObj().getClassName();

    // Side effect: computes retained size.
    snapshot.computeDominators();

    Instance leakingInstance = result.leakingNode.instance;

    long retainedSize = leakingInstance.getTotalRetainedSize(); //此次泄露的總大小

    // TODO: check O sources and see what happened to android.graphics.Bitmap.mBuffer
    if (SDK_INT <= N_MR1) {
      retainedSize += computeIgnoredBitmapRetainedSize(snapshot, leakingInstance);
    }

    return leakDetected(result.excludingKnownLeaks, className, leakTrace, retainedSize,
        since(analysisStartNanoTime));
  }

最後一步就是將AnalysisResult對象交給DisplayLeakService完成保存與展示的工作(Notification通知用戶),這個地方不作爲核心原理,一筆帶過~
這個地方我個人感覺視情況而定,對於開發來說,泄露點的發現纔是重點,這個地方只是泄露的顯示部分~如果不想顯示,這個模塊可以去除,或者在顯示的同時,還有其他操作,比如將結果上傳至服務器,也是可以做~

  public static void sendResultToListener(Context context, String listenerServiceClassName,
      HeapDump heapDump, AnalysisResult result) {
    Class<?> listenerServiceClass;
    try {
      listenerServiceClass = Class.forName(listenerServiceClassName);
    } catch (ClassNotFoundException e) {
      throw new RuntimeException(e);
    }
    Intent intent = new Intent(context, listenerServiceClass);
    intent.putExtra(HEAP_DUMP_EXTRA, heapDump);
    intent.putExtra(RESULT_EXTRA, result);
    context.startService(intent);
  }

問題

Leakcanary能檢測什麼樣的泄露問題?
一句話就是:Activity生命週期相關的泄露問題

能不能發現所有的內存泄露?
我的理解是不見得,Leakcanary可以檢測組件Activity的泄露問題,Activity跟界面息息相關,而且也是四大組件中最爲重要的組件~但依我的理解,如果自定義Leakcanary,完全可以實現四大組件的內存泄露檢測(原生的Leakcanary監測的Activity的onDestroy生命週期函數,同樣的道理也可以監測Service,Provider的生命週期函數,這個地方我倒是沒做嘗試)~

怎麼集成Leakcanary?
1:可以把LeakCanary.install()方法直接放在Application的onCreate()方法中,然後重新打包生成apk後開始測試,這種方式每個App都需要重新打包後測試,當App較多且升級較爲頻繁時時比較繁瑣;
2:把LeakCanary.install()方法放到系統中,通過系統屬性來控制,默認不開啓,這種方式需要改動系統源碼造成系統較爲冗餘,而且應用每次創建時都會判斷屬性,但相對於方案1,其實比較省力了;
3:採用hook方式,hook Application.onCreate方法,測試時自動帶着LeakCanary.install(),這種方案相對來講對於系統的方式比較友好,測試的代碼跟系統運行是分離的,也比較頭疼,因爲每個版本都需要處理hook的問題~

如何自定義二次開發?
Leakcanary的核心部分是泄漏點的確定以及泄露路徑的生成,這部分基本不需要動,,二次開發需結合自身需求,集成方式有自定義開發空間,對於結果如何處理有自定義開發空間,對於ExcludedRefs集合同樣也有自定義空間

深一點的問題:一個泄露點,是不是每次泄露路徑都是一樣的?
不一定,只能講絕大多數情況是一樣的泄露路徑,但Leakcanary在尋找最短路徑時,這個泄露點到GC ROOT有多條最短路徑,這個時候輸出的泄露路徑不一定是一樣的~但換句話來講,如果有多個GC ROOT都對Activity可達,說明此Activity泄露的機率非常之高~更應該引起重視