java線程池ThreadPoolExecutor類核心方法理解

java線程池ThreadPoolExecutor類核心方法理解

李老爺子的註釋其實已經非常詳細了,這裏主要是將逐句意義和我自己的理解貼出來.

本文不涉及ctl狀態的二進制算法的理解(在此無意義).

完整構造線程池

/**
 * Creates a new {@code ThreadPoolExecutor} with the given initial
 * parameters.
 *
 * @param corePoolSize the number of threads to keep in the pool, even
 *        if they are idle, unless {@code allowCoreThreadTimeOut} is set
 * @param maximumPoolSize the maximum number of threads to allow in the
 *        pool
 * @param keepAliveTime when the number of threads is greater than
 *        the core, this is the maximum time that excess idle threads
 *        will wait for new tasks before terminating.
 * @param unit the time unit for the {@code keepAliveTime} argument
 * @param workQueue the queue to use for holding tasks before they are
 *        executed.  This queue will hold only the {@code Runnable}
 *        tasks submitted by the {@code execute} method.
 * @param threadFactory the factory to use when the executor
 *        creates a new thread
 * @param handler the handler to use when execution is blocked
 *        because the thread bounds and queue capacities are reached
 * @throws IllegalArgumentException if one of the following holds:<br>
 *         {@code corePoolSize < 0}<br>
 *         {@code keepAliveTime < 0}<br>
 *         {@code maximumPoolSize <= 0}<br>
 *         {@code maximumPoolSize < corePoolSize}
 * @throws NullPointerException if {@code workQueue}
 *         or {@code threadFactory} or {@code handler} is null
 */
public ThreadPoolExecutor(int corePoolSize,//核心線程數(核心線程不會被銷燬)
                          int maximumPoolSize,//最大線程數
                          long keepAliveTime,//超過核心線程數的線程的最大空閒生存時間,其後將可能被銷燬
                          TimeUnit unit,//keepAliveTime的單位
                          BlockingQueue<Runnable> workQueue,//線程隊列,當線程數超過核心線程數時入隊
                          ThreadFactory threadFactory,//線程工廠
                          RejectedExecutionHandler handler//當線程數滿,隊列滿時的拒絕策略
                         ) {
    if (corePoolSize < 0 ||
            maximumPoolSize <= 0 ||
            maximumPoolSize < corePoolSize ||
            keepAliveTime < 0)
        throw new IllegalArgumentException();
    if (workQueue == null || threadFactory == null || handler == null)
        throw new NullPointerException();
    this.corePoolSize = corePoolSize;
    this.maximumPoolSize = maximumPoolSize;
    this.workQueue = workQueue;
    this.keepAliveTime = unit.toNanos(keepAliveTime);
    this.threadFactory = threadFactory;
    this.handler = handler;
}

ThreadPoolExecutor::submit

//提交一個允許有返回值的任務,Future::get獲取返回值.
public Future<?> submit(Runnable task) {
    if (task == null) throw new NullPointerException();
    //RunnableFuture自己就是一個Runnable且同時是一個Future可以用來接收返回值
    RunnableFuture<Void> ftask = newTaskFor(task, null);
    //執行execute,添加woker運行指定task
    execute(ftask);
    return ftask;
}

ThreadPoolExecutor::execute

/**
* Executes the given task sometime in the future.  The task
* may execute in a new thread or in an existing pooled thread.
*
* If the task cannot be submitted for execution, either because this
* executor has been shutdown or because its capacity has been reached,
* the task is handled by the current {@link RejectedExecutionHandler}.
*
* @param command the task to execute
* @throws RejectedExecutionException at discretion of
*         {@code RejectedExecutionHandler}, if the task
*         cannot be accepted for execution
* @throws NullPointerException if {@code command} is null
*/
public void execute(Runnable command) {
	if (command == null)
		throw new NullPointerException();
    /*
     * Proceed in 3 steps:
     *
     * 1. If fewer than corePoolSize threads are running, try to
     * start a new thread with the given command as its first
     * task.  The call to addWorker atomically checks runState and
     * workerCount, and so prevents false alarms that would add
     * threads when it shouldn't, by returning false.
     *
     * 2. If a task can be successfully queued, then we still need
     * to double-check whether we should have added a thread
     * (because existing ones died since last checking) or that
     * the pool shut down since entry into this method. So we
     * recheck state and if necessary roll back the enqueuing if
     * stopped, or start a new thread if there are none.
     *
     * 3. If we cannot queue task, then we try to add a new
     * thread.  If it fails, we know we are shut down or saturated
     * and so reject the task.
     */
	/*
    * 分三步進行:
    * 1. 如果運行的線程少於corePoolSize,
    * 嘗試以command作爲第一個task開啓一個一個新核心線程.
    * 2. 如果成功將command入隊workQueue,
    * 雙重檢測確保線程池正RUNNING,
    * (可能有其他線程執行了shutdown).
    * 如果線程池已經shutdown,則回滾入隊操作,
    * 並執行拒絕策略
    * 3. 如果無法入隊,直接添加新的工作線程並執行command,
    * 如果操作失敗了,則說明線程池可能已經shutdown或飽和了,
    * 則執行拒絕策略
    */
    // 也就是說:1.如果核心線程沒滿開核心線程,否則將任務加入任務隊列,
    // 2.如果此時線程池關了,出隊任務並執行拒絕策略
    // 3.如果核心線程設定爲且工作線程爲0,則開非核心線程並執行隊列中的任務,
    // 4.如果隊列滿了開非核心線程,如果失敗了執行拒絕策略
    // 即:1.如果核心線程數設定大於0,只要任務隊列沒滿就最多隻會有核心線程.
    // 2.如果核心線程數設定等於0,只要任務隊列沒滿就最多隻有一個非核心線程.
	//獲取ctl快照
	int c = ctl.get();
	//第一步
	//判斷工作線程數是否少於設定的核心線程數值
	if (workerCountOf(c) < corePoolSize) {
		//添加核心工作線程
		if (addWorker(command, true))
			return;
		//重新獲取ctl快照(ctl可能已被其他線程修改)
		c = ctl.get();
	}
	//第二部
	//如果線程池正RUNNING,將command加入workQueue
	if (isRunning(c) && workQueue.offer(command)) {
		//重新獲取ctl快照
		int recheck = ctl.get();
		//雙重檢測,確保線程池沒有shutdown,如果shutdown了則將command出隊workQueue
		if (! isRunning(recheck) && remove(command))
			//執行拒絕策略
			reject(command);
		//判斷此時線程池正RUNNING,且工作線程爲0(corePoolSize可被設定爲0)
		else if (workerCountOf(recheck) == 0)
			//添加非核心線程,並從workQueue中取出首個command運行
			addWorker(null, false);
	}
	//隊列可能已滿從而失敗的情況下,直接添加非核心工作線程,並將command作爲task運行
	else if (!addWorker(command, false))
		//執行addWorker失敗(線程池關閉或飽和)則執行拒絕策略
		reject(command);
}

ThreadPoolExecutor::addWorker

/*
* Methods for creating, running and cleaning up after workers
*/

/**
* Checks if a new worker can be added with respect to current
* pool state and the given bound (either core or maximum). If so,
* the worker count is adjusted accordingly, and, if possible, a
* new worker is created and started, running firstTask as its
* first task. This method returns false if the pool is stopped or
* eligible to shut down. It also returns false if the thread
* factory fails to create a thread when asked.  If the thread
* creation fails, either due to the thread factory returning
* null, or due to an exception (typically OutOfMemoryError in
* Thread.start()), we roll back cleanly.
*
* @param firstTask the task the new thread should run first (or
* null if none). Workers are created with an initial first task
* (in method execute()) to bypass queuing when there are fewer
* than corePoolSize threads (in which case we always start one),
* or when the queue is full (in which case we must bypass queue).
* Initially idle threads are usually created via
* prestartCoreThread or to replace other dying workers.
*
* @param core if true use corePoolSize as bound, else
* maximumPoolSize. (A boolean indicator is used here rather than a
* value to ensure reads of fresh values after checking other pool
* state).
* @return true if successful
*/
private boolean addWorker(Runnable firstTask, boolean core) {
    retry:
    for (int c = ctl.get();;) {//死循環,每次循環獲取ctl最新快照
        // Check if queue empty only if necessary.
        // 必要時檢測workQueue是否爲空.(這裏利用與或行爲的短路一層一層判斷)
        // 什麼是必要條件:當且僅當線程池被SHUTDOWN的時候,且不再有新任務.
        // 即:addWorker時,如果線程池已經SHUTDOWN就不再接受新任務,但繼續消費workQueue中的任務.
        if (
            //1.檢測線程是否已經被SHUTDOWN,如果此時還是RUNNING就直接執內循環,否則如果至少是SHUTDOWN則進入下個與(進入下一個與線程池至少SHUTDOWN,甚至是STOP)
            runStateAtLeast(c, SHUTDOWN)
            && (
                //2.1.檢查線程是否已經被STOP,如果被STOP了就不再消費workQueue,返回false,如果小於STOP則進入下一個或(進入下一個或線程池必然處在SHUTDOWN)
                runStateAtLeast(c, STOP)
                //2.2.如果有指定要執行的任務,由於此時線程池已經SHUTDOWN,不接收新任務,直接返回false,如果沒給定新任務則進入下一個或
                || firstTask != null
                //2.3. 如果任務隊列爲空,此時線程池也正處在SHUTDOWN,同時也沒有新任務,則返回false,否則需要進入內循環消費workQueue剩餘任務
                || workQueue.isEmpty()
            )
        )
            //執行失敗(三種情況:1.線程池已經STOP,2.線城池是SHUTDOWN但指定了新任務,3.線城池是SHUTDOWN且workQueue爲空)
            return false;

        for (;;) {
            //當前線程數:1.如果是add核心線程,判斷是否大於等於核心線程數,否則判斷是否大於等於最大線程數
            if (workerCountOf(c)
                >= ((core ? corePoolSize : maximumPoolSize) & COUNT_MASK))
                //線程池飽和,執行失敗
                return false;
            //上面判斷都過了,說明此時可以添加任務,CAS先將線程數加一(如果後面實際添加worker執行失敗再回退),CAS執行成功則跳出外循環,執行下面的添加worker
            if (compareAndIncrementWorkerCount(c))
                break retry;
            //重新獲取ctl快照,確保獲取到的是最新的值(值傳遞)
            c = ctl.get();  // Re-read ctl
            //如果此時狀態至少是SHUTDOWN,則重新執行外循環
            if (runStateAtLeast(c, SHUTDOWN))
                continue retry;
            // else CAS failed due to workerCount change; retry inner loop
            // 否則,重新執行內循環將線程數加一
        }
    }

    boolean workerStarted = false;
    boolean workerAdded = false;
    Worker w = null;
    try {
        //新建worker,並將firstTask丟進入,以保證如果有firstTask的情況下它會最先執行
        //內部線程的run方法會runWorker方法,runWorker會循環從workQueue取任務執行
        w = new Worker(firstTask);
        //拿到worker內部新建的線程快照
        final Thread t = w.thread;
        if (t != null) {
            final ReentrantLock mainLock = this.mainLock;
            //這裏的操作需要加鎖主要是因爲workers是HashSet,線程不安全
            mainLock.lock();
            try {
                // Recheck while holding lock.
                // Back out on ThreadFactory failure or if
                // shut down before lock acquired.
                // 在獲得鎖後重新檢測,以確保線程池正處在正常運行狀態
                // 重新獲取最新快照
                int c = ctl.get();

                //如果正RUNNING,則直接添加worker到集合中
                if (isRunning(c) ||
                    //否則如果線程池是SHUTDOWN且沒有新任務的情況下才添加worker到集合中
                    (runStateLessThan(c, STOP) && firstTask == null)) {
                    //如果線程不是處與新建狀態,拋出異常(因爲後面會執行start)
                    if (t.getState() != Thread.State.NEW)
                        throw new IllegalThreadStateException();
                    //添加worker到集合中
                    workers.add(w);
                    //修改worker添加狀態
                    workerAdded = true;
                    //修改總worker數量
                    int s = workers.size();
                    if (s > largestPoolSize)
                        largestPoolSize = s;
                }
            } finally {
                //解鎖
                mainLock.unlock();
            }
            //如果已經添加了worker,說明此時worker創建成功,且內部的線程沒有開始運行,則使其運行
            if (workerAdded) {
                t.start();
                //修改worker啓動狀態
                workerStarted = true;
            }
        }
    } finally {
        //如果worker線程被啓動失敗
        if (! workerStarted)
            //回退上面的工作線程數加一操作,並將worker從集合中移除(如果worker已經被加入了集合的話),並執行tryTerminate內部的terminated鉤子
            addWorkerFailed(w);
    }
    return workerStarted;
}

ThreadPoolExecutor::runWorker

/**
 * Main worker run loop.  Repeatedly gets tasks from queue and
 * executes them, while coping with a number of issues:
 *
 * 1. We may start out with an initial task, in which case we
 * don't need to get the first one. Otherwise, as long as pool is
 * running, we get tasks from getTask. If it returns null then the
 * worker exits due to changed pool state or configuration
 * parameters.  Other exits result from exception throws in
 * external code, in which case completedAbruptly holds, which
 * usually leads processWorkerExit to replace this thread.
 *
 * 2. Before running any task, the lock is acquired to prevent
 * other pool interrupts while the task is executing, and then we
 * ensure that unless pool is stopping, this thread does not have
 * its interrupt set.
 *
 * 3. Each task run is preceded by a call to beforeExecute, which
 * might throw an exception, in which case we cause thread to die
 * (breaking loop with completedAbruptly true) without processing
 * the task.
 *
 * 4. Assuming beforeExecute completes normally, we run the task,
 * gathering any of its thrown exceptions to send to afterExecute.
 * We separately handle RuntimeException, Error (both of which the
 * specs guarantee that we trap) and arbitrary Throwables.
 * Because we cannot rethrow Throwables within Runnable.run, we
 * wrap them within Errors on the way out (to the thread's
 * UncaughtExceptionHandler).  Any thrown exception also
 * conservatively causes thread to die.
 *
 * 5. After task.run completes, we call afterExecute, which may
 * also throw an exception, which will also cause thread to
 * die. According to JLS Sec 14.20, this exception is the one that
 * will be in effect even if task.run throws.
 *
 * The net effect of the exception mechanics is that afterExecute
 * and the thread's UncaughtExceptionHandler have as accurate
 * information as we can provide about any problems encountered by
 * user code.
 *
 * @param w the worker
 */
/*
這個線程的核心是while循環不停的從隊列中拿取任務並在獨佔鎖的情況下執行(不會被中斷)
線程被終止的情況:
1. 當執行的任務拋出異常線程將終止(線程池還在運行時將有新線程替代它);
2. 非核心線程在等待任務到超時後被通過中斷終止;
3. 線程池停止時中斷自己或被中斷.
線程自己中斷的條件:線程池已經至少是STOP,且當前線程沒被中斷過
判單當前線程是否空閒:在while循環中是會上鎖的,而在getTask等待的過程中沒有,
此時其他線程通過tryLock如果獲取到了鎖則說明線程當前處於空閒狀態,
這種方式還可以保證其他線程在判斷到當前線程是空閒時,當前線程不會被getTask從阻塞中喚醒並執行task
*/
final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    //拿到指定的firstTask先執行並清空firstTask
    Runnable task = w.firstTask;
    w.firstTask = null;
    w.unlock(); // allow interrupts 允許中斷
    boolean completedAbruptly = true;
    try {
        //當沒有firstTask的時候阻塞地等待獲取task
        while (task != null || (task = getTask()) != null) {
            //拿到task了就上鎖,主要目的用來在其他線程判斷當前線程是否還在阻塞等待獲取task
            w.lock();
            // If pool is stopping, ensure thread is interrupted;
            // if not, ensure thread is not interrupted.  This
            // requires a recheck in second case to deal with
            // shutdownNow race while clearing interrupt
            // 如果線程池處在執行停止方法,確保線程已被中斷;
            // 否則,確保線程沒有被中斷.
            // 這種情況下需要在shutdownNow方法中雙重檢測以清理(處理)中斷.
            // 這裏的if需要完成的任務: 如果線程池的狀態至少是STOP狀態,則要保證當前線程已經被執行過中斷,
            // 沒有的話就立刻執行中斷,後面的task將不會執行,這種情況可能時執行了shutdownNow方法.
            if (
                (
                    //1.1.當線程池已經至少時STOP狀態  或
                    runStateAtLeast(ctl.get(), STOP) ||
                    (
                        //2.1.當前線程線程已經被中斷(如果中斷這個方法內部執行clearInterruptEvent)  且 
                        Thread.interrupted() &&
                        //2.2.線程池已經至少時STOP狀態
                        runStateAtLeast(ctl.get(), STOP)
                    )
                ) &&
                //1.2.線程未被中斷
                !wt.isInterrupted()
            )
                //中斷線程,下面的try將不再被執行
                wt.interrupt();
            try {
                //執行鉤子方法beforeExecute
                beforeExecute(wt, task);
                try {
                    task.run();
                    ////執行鉤子方法afterExecute(無異常)
                    afterExecute(task, null);
                } catch (Throwable ex) {
                    //執行鉤子方法afterExecute(異常)
                    afterExecute(task, ex);
                    //將異常拋出,這意味這個線程將消亡,如果線程池還在運行,將有新的線程替代它
                    throw ex;
                }
            } finally {
                //清除已經執行的task引用
                task = null;
                //當前線程完成任務數+1
                w.completedTasks++;
                //解鎖,則說明線程又進入空閒狀態
                w.unlock();
            }
        }
        //如果循環內拋出了異常,這裏將不會執行
        completedAbruptly = false;
    } finally {
        //執行線程退出的方法,當執行到這裏線程說明線程即將消亡(非核心線程超時或線程任務執行拋出了異常或線程池將停止)
        processWorkerExit(w, completedAbruptly);
    }
}

ThreadPoolExecutor::beforeExecute和afterExecute模板方法(鉤子方法)

//鉤子方法只在runWorker中執行
//繼承ThreadPoolExecutor以實現鉤子方法
class MyThreadPoolExecutor extends ThreadPoolExecutor {
    public MyThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
    }
    @Override
    //runWorker內部執行task.run()前執行這個鉤子方法
    protected void beforeExecute(Thread t, Runnable r) {
        super.beforeExecute(t, r);
        System.out.println("執行任務前的鉤子,已執行"+this.getTaskCount());
    }
    @Override
    //runWorker內部執行task.run()後執行這個鉤子方法,如果run拋出了異常可以在此處理
    protected void afterExecute(Runnable r, Throwable t) {
        super.afterExecute(r, t);
        System.out.println("執行任務後的鉤子,完成執行"+this.getTaskCount());
    }
}

ThreadPoolExecutor::processWorkerExit

/**
 * Performs cleanup and bookkeeping for a dying worker. Called
 * only from worker threads. Unless completedAbruptly is set,
 * assumes that workerCount has already been adjusted to account
 * for exit.  This method removes thread from worker set, and
 * possibly terminates the pool or replaces the worker if either
 * it exited due to user task exception or if fewer than
 * corePoolSize workers are running or queue is non-empty but
 * there are no workers.
 *
 * @param w the worker
 * @param completedAbruptly if the worker died due to user exception
 */
/*
線程退方法只在runWorker最後執行
主要完成一些清理工作(將當前線程從池中移除)和記錄工作(當前線程的)
如果線程池還在運行,將可能添加新線程替代這個線程(此時這個線程可能在執行中拋出了異常
*/
private void processWorkerExit(Worker w, boolean completedAbruptly) {
    //如果completedAbruptly爲true,則說明runWorker中task.run沒執行(此時這個方法是在中斷時執行)或runWorker的while循環中拋出了異常
    if (completedAbruptly) // If abrupt, then workerCount wasn't adjusted
        decrementWorkerCount();//這種情況工作線程數還沒被減一,在此減一

    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();//上鎖以線性執行,保障workers和completedTaskCount不會混亂
    try {
        completedTaskCount += w.completedTasks;//將當前線程的任務完成數加到線程池
        workers.remove(w);//把當前線程從works中移除
    } finally {
        mainLock.unlock();
    }

    //執行tryTerminate清理線程並在其內部執行terminated鉤子
    tryTerminate();

    int c = ctl.get();//獲取ctl快照
    //如線程還沒進入STOP(處在RUNNING或SHUTDOWN)
    if (runStateLessThan(c, STOP)) {
        //如果不是被突然終止(即task.run及其鉤子執行成功,沒拋出異常的情況下)
        if (!completedAbruptly) {
            //拿到最小線程數
            int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
            //如果最小線程數爲0,但此時工作隊列內還有task,將最小線程數設置爲1保證有線程執行隊列中的task
            if (min == 0 && ! workQueue.isEmpty())
                min = 1;
            //如果當前存在的工作線程數大於等於最小線程數就不用添加新線程,否則就添加一個線程執行隊列中的task
            if (workerCountOf(c) >= min)
                return; // replacement not needed
        }
        //否則(即task.run及其鉤子沒執行或拋出異常的情況下)
        addWorker(null, false);//添加一個新的工作線程替代此線程
    }
}

ThreadPoolExecutor::tryTerminate

/**
 * Transitions to TERMINATED state if either (SHUTDOWN and pool
 * and queue empty) or (STOP and pool empty).  If otherwise
 * eligible to terminate but workerCount is nonzero, interrupts an
 * idle worker to ensure that shutdown signals propagate. This
 * method must be called following any action that might make
 * termination possible -- reducing worker count or removing tasks
 * from the queue during shutdown. The method is non-private to
 * allow access from ScheduledThreadPoolExecutor.
 */
/*
這種方法被多個方法調用,主要是保證清理工作的正常進行,空閒的線程能被正常清除,
同時又至少保留一個線程以保證有線程能執行清理.
保證線程池的ctl被正確設置.
爲什麼在清理空閒線程時只清理一個?
因爲需要保證至少留有一個線程善後,一個一個的清理能保證最後一個線程一定可以善後
(如過中斷全部,由於此時沒有上鎖,其他線程也可以中斷當前線程,可能就沒線程了)
*/
final void tryTerminate() {
    //CAS死循環,不執行成功誓不罷休
    for (;;) {
        int c = ctl.get();//ctl快照
        //如果 線程正在運行  或
        if (isRunning(c) ||
                //線程已經是TIDYING或TERMINATED  或
                //(這種情況下清理工作已經做完了)
                runStateAtLeast(c, TIDYING) ||
                //線程是RUNNING或SHUTDOWN  且  工作線程不爲空
                //(這個狀態下要保證任務能得到執行,無需清理)
                (runStateLessThan(c, STOP) && ! workQueue.isEmpty()))
            return;//則退出
        //如果還有工作線程存活,則清理一個空閒線程
        if (workerCountOf(c) != 0) { // Eligible to terminate
            interruptIdleWorkers(ONLY_ONE);
            return;
        }

        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();//上鎖,保證terminated鉤子和ctl正常
        try {
            //CAS的方式將狀態設置爲TIDYING
            if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
                try {
                    terminated();//執行鉤子
                } finally {
                    //無論怎樣,此時線程池都徹底結束了,狀態設置爲TERMINATED
                    ctl.set(ctlOf(TERMINATED, 0));
                    //喚醒所有執行了awaitTermination方法並阻塞的線程,讓他們繼續執行
                    termination.signalAll();
                }
                return;//一切都結束了,線程池完成了它的使命
            }
        } finally {
            mainLock.unlock();
        }
        // else retry on failed CAS
    }
}