java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.FutureTask@3f54147b rejected from
java.util.concurrent.ThreadPoolExecutor@55407c43[Running, pool size = 10, active threads = 10, queued tasks = 0, completed tasks = 2638]
根據文檔資料得到,在默認設置下,maxQueueSize的長度爲-1,則使用了SynchronousQueue
* maxQueueSize BlockingQueue的最大隊列大小,設爲-1使用SynchronousQueue,否則使用LinkedBlockingQueue,默認值:-1
翻hystrix線程池的代碼,得到如果是-1的情況下,hystrix是不進行判斷隊列的長度的,此時,直接交由線程池去處理。
//HystrixThreadPool.java
@Override
public boolean isQueueSpaceAvailable() {
if (queueSize <= 0) {
// we don't have a queue so we won't look for space but instead
// let the thread-pool reject or not
return true;
} else {
return threadPool.getQueue().size() < properties.queueSizeRejectionThreshold().get();
}
}
通過異常,我們得知,產生的原因是因爲隊列無法加入任務被reject了,當我們設置-1的時候,此時的隊列用的實現是SynchronousQueue(0),意味着,線程池不會進行等待,如果對應的任務滿了coreSize,則直接被默認的策略reject掉。
(hystrix使用的拒絕策略是滿了直接拒絕)
//HystrixConcurrencyStrategy.java
/**
* Factory method to provide instance of {@code BlockingQueue<Runnable>} used for each {@link ThreadPoolExecutor} as constructed in {@link #getThreadPool}.
* <p>
* Note: The maxQueueSize value is provided so any type of queue can be used but typically an implementation such as {@link SynchronousQueue} without a queue (just a handoff) is preferred as
* queueing is an anti-pattern to be purposefully avoided for latency tolerance reasons.
* <p>
* <b>Default Implementation</b>
* <p>
* Implementation returns {@link SynchronousQueue} when maxQueueSize <= 0 or {@link LinkedBlockingQueue} when maxQueueSize > 0.
*
* @param maxQueueSize
* The max size of the queue requested via properties (or system default if no properties set).
* @return instance of {@code BlockingQueue<Runnable>}
*/
public BlockingQueue<Runnable> getBlockingQueue(int maxQueueSize) {
/*
* We are using SynchronousQueue if maxQueueSize <= 0 (meaning a queue is not wanted).
* <p>
* SynchronousQueue will do a handoff from calling thread to worker thread and not allow queuing which is what we want.
* <p>
* Queuing results in added latency and would only occur when the thread-pool is full at which point there are latency issues
* and rejecting is the preferred solution.
*/
if (maxQueueSize <= 0) {
return new SynchronousQueue<Runnable>();
} else {
return new LinkedBlockingQueue<Runnable>(maxQueueSize);
}
}
通過上述,解決這個異常的問題通過兩條路,一個是修改線程池的等待隊列,一個是加大工作線程數(不建議改coreSize,應該通過修改最大線程數來變更,避免有常駐線程過多)。
1、同時設置
1、同時設置隊列長度跟拒絕長度
maxQueueSize: 1000 #BlockingQueue的最大隊列數,默認值-1
queueSizeRejectionThreshold: 800 #即使maxQueueSize沒有達到,達到queueSizeRejectionThreshold該值後,請求也會被拒絕,默認值5
2、同時設置最多工作線程數以及allowMaximumSizeToDivergeFromCoreSize
maximumSize: 100
allowMaximumSizeToDivergeFromCoreSize: true