1 簡介
本篇儘量用一些簡單的數學式子和流程圖和大家一起梳理一下這些集羣容錯算法.
2 靈魂拷問
- 談談dubbo中的負載均衡算法及特點
- 最小活躍數算法中是如何統計這個活躍數的
- 簡單談談你對一致性哈希算法的認識
3 接口的繼承體系
4 RandomLoadBalance(隨機)
隨機，按權重設置隨機概率
在一個截面上碰撞的概率高，但調用量越大分佈越均勻，而且按概率使用權重後也比較均勻，有利於動態調整提供者權重。
默認策略,但是這個隨機和我們理解上的隨機還是不一樣的,因爲他還有個概念叫weight(權重),就是用來控制這個隨機的概率的,我們來看代碼實現.
package org.apache.dubbo.rpc.cluster.loadbalance;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.rpc.Invocation;
import org.apache.dubbo.rpc.Invoker;
import java.util.List;
import java.util.concurrent.ThreadLocalRandom;
/**
-
此類從多個提供者中隨機選擇一個提供者。
-
可以爲每個提供商定義權重：
-
如果權重都相同，則將使用random.nextInt（調用者數）。
-
如果權重不同，則將使用random.nextInt（w1 + w2 + … + wn）
-
請注意，如果機器的性能優於其他機器，則可以設置更大的重量。
-
如果性能不是很好，則可以設置較小的重量。
*/
public class RandomLoadBalance extends AbstractLoadBalance {public static final String NAME = “random”;
/**
- 使用隨機條件在列表之間選擇一個invoker
- @param invokers 可能的invoker列表
- @param url URL
- @param invocation Invocation
- @param <T>
- @return 被選的invoker
*/
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
// invoker的數量
int length = invokers.size();
// 每個 invoker 有相同權重
boolean sameWeight = true;
// 每個invoker的權重
int[] weights = new int[length];
// 第一個 invoker 的權重
int firstWeight = getWeight(invokers.get(0), invocation);
weights[0] = firstWeight;
// 權重之和
int totalWeight = firstWeight;
for (int i = 1; i < length; i++) {
int weight = getWeight(invokers.get(i), invocation);
// 保存以待後用
weights[i] = weight;
// Sum
totalWeight += weight;
if (sameWeight && weight != firstWeight) {
sameWeight = false;
}
}
if (totalWeight > 0 && !sameWeight) {
// 如果並非每個invoker都具有相同的權重且至少一個invoker的權重大於0，請根據totalWeight隨機選擇
int offset = ThreadLocalRandom.current().nextInt(totalWeight);
// 根據隨機值返回invoker
for (int i = 0; i < length; i++) {
offset -= weights[i];
if (offset < 0) {
return invokers.get(i);
}
}
}
// 如果所有invoker都具有相同的權重值或totalWeight = 0，則平均返回。
return invokers.get(ThreadLocalRandom.current().nextInt(length));
}
}
分析
- 流程圖
![]()
假設有四個集羣節點A,B,C,D,對應的權重分別是1,2,3,4,那麼請求到A節點的概率就爲1/(1+2+3+4) = 10%.B,C,D節點依次類推爲20%,30%,40%.
雖然這個隨機算法理解起來是比較容易的,面試一般不會問這個,但是假如我們要實現類似的功能,他這個代碼實現的思路還是很優雅的,非常具有借鑑意義
他這個實現思路從純數學角度是很好理解的,我們還是按照上面數學分析中的前提條件.我們知道總權重爲10(1+2+3+4),那麼怎麼做到按權重隨機呢?
根據10隨機出一個整數,假如爲隨機出來的是2.然後依次和權重相減,比如2(隨機數)-1(A的權重) = 1,然後1(上一步計算的結果)-2(B的權重) = -1,此時-1 < 0,那麼則調用B,其他的以此類推
5 RoundRobinLoadBalance(輪詢)
輪詢，按公約後的權重設置輪循比率
存在慢的提供者累積請求的問題，比如：第二臺機器很慢，但沒掛，當請求調到第二臺時就卡在那，久而久之，所有請求都卡在調到第二臺上
package org.apache.dubbo.rpc.cluster.loadbalance;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.rpc.Invocation;
import org.apache.dubbo.rpc.Invoker;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
/**
-
Round robin load balance.
*/
public class RoundRobinLoadBalance extends AbstractLoadBalance {
public static final String NAME = “roundrobin”;private static final int RECYCLE_PERIOD = 60000;
protected static class WeightedRoundRobin {
private int weight;
private AtomicLong current = new AtomicLong(0);
private long lastUpdate;
public int getWeight() {
return weight;
}
public void setWeight(int weight) {
this.weight = weight;
current.set(0);
}
public long increaseCurrent() {
return current.addAndGet(weight);
}
public void sel(int total) {
current.addAndGet(-1 * total);
}
public long getLastUpdate() {
return lastUpdate;
}
public void setLastUpdate(long lastUpdate) {
this.lastUpdate = lastUpdate;
}
}private ConcurrentMap<String, ConcurrentMap<String, WeightedRoundRobin>> methodWeightMap = new ConcurrentHashMap<String, ConcurrentMap<String, WeightedRoundRobin>>();
private AtomicBoolean updateLock = new AtomicBoolean();/**
- 獲取爲指定invocation緩存的invocation地址列表
- for unit test only
*/
protected <T> Collection<String> getInvokerAddrList(List<Invoker<T>> invokers, Invocation invocation) {
String key = invokers.get(0).getUrl().getServiceKey() + “.” + invocation.getMethodName();
Map<String, WeightedRoundRobin> map = methodWeightMap.get(key);
if (map != null) {
return map.keySet();
}
return null;
}
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
String key = invokers.get(0).getUrl().getServiceKey() + “.” + invocation.getMethodName();
ConcurrentMap<String, WeightedRoundRobin> map = methodWeightMap.get(key);
if (map == null) {
methodWeightMap.putIfAbsent(key, new ConcurrentHashMap<String, WeightedRoundRobin>());
map = methodWeightMap.get(key);
}
int totalWeight = 0;
long maxCurrent = Long.MIN_VALUE;
long now = System.currentTimeMillis();
Invoker<T> selectedInvoker = null;
WeightedRoundRobin selectedWRR = null;
for (Invoker<T> invoker : invokers) {
String identifyString = invoker.getUrl().toIdentityString();
WeightedRoundRobin weightedRoundRobin = map.get(identifyString);
int weight = getWeight(invoker, invocation);<span class="hljs-keyword">if</span> (weightedRoundRobin == <span class="hljs-keyword">null</span>) { weightedRoundRobin = <span class="hljs-keyword">new</span> WeightedRoundRobin(); weightedRoundRobin.setWeight(weight); map.putIfAbsent(identifyString, weightedRoundRobin); } <span class="hljs-keyword">if</span> (weight != weightedRoundRobin.getWeight()) { <span class="hljs-comment">//weight changed</span> weightedRoundRobin.setWeight(weight); } <span class="hljs-keyword">long</span> cur = weightedRoundRobin.increaseCurrent(); weightedRoundRobin.setLastUpdate(now); <span class="hljs-keyword">if</span> (cur > maxCurrent) { maxCurrent = cur; selectedInvoker = invoker; selectedWRR = weightedRoundRobin; } totalWeight += weight; } <span class="hljs-keyword">if</span> (!updateLock.get() && invokers.size() != map.size()) { <span class="hljs-keyword">if</span> (updateLock.compareAndSet(<span class="hljs-keyword">false</span>, <span class="hljs-keyword">true</span>)) { <span class="hljs-keyword">try</span> { <span class="hljs-comment">// copy -> modify -> update reference</span> ConcurrentMap<String, WeightedRoundRobin> newMap = <span class="hljs-keyword">new</span> ConcurrentHashMap<>(map); newMap.entrySet().removeIf(item -> now - item.getValue().getLastUpdate() > RECYCLE_PERIOD); methodWeightMap.put(key, newMap); } <span class="hljs-keyword">finally</span> { updateLock.set(<span class="hljs-keyword">false</span>); } } } <span class="hljs-keyword">if</span> (selectedInvoker != <span class="hljs-keyword">null</span>) { selectedWRR.sel(totalWeight); <span class="hljs-keyword">return</span> selectedInvoker; } <span class="hljs-comment">// should not happen here</span> <span class="hljs-keyword">return</span> invokers.get(<span class="hljs-number">0</span>);}
}
Nginx的負載均衡默認就是輪詢
6 LeastActiveLoadBalance(最少活躍數)
- 最少活躍調用數，相同活躍數的隨機，活躍數指調用前後計數差
- 使慢的提供者收到更少請求，因爲越慢的提供者的調用前後計數差會越大。
舉個例子.每個服務有一個活躍計數器
那麼我們假如有A,B兩個提供者.計數初始均爲0
當A提供者開始處理請求,該計數+1,此時A還沒處理完,當處理完後則計數-1
而B請求接收到請求處理得很快.B處理完後A還沒處理完,所以此時A,B的計數爲1,0
那麼當有新的請求來的時候,就會選擇B提供者(B的活躍計數比A小)
這就是文檔說的,使慢的提供者收到更少請求
package org.apache.dubbo.rpc.cluster.loadbalance;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.rpc.Invocation;
import org.apache.dubbo.rpc.Invoker;
import org.apache.dubbo.rpc.RpcStatus;
import java.util.List;
import java.util.concurrent.ThreadLocalRandom;
/**
-
過濾活動調用次數最少的調用者數量，並計算這些調用者的權重和數量。
-
如果只有一個調用程序，則直接使用該調用程序；
-
如果有多個調用者並且權重不相同，則根據總權重隨機；
-
如果有多個調用者且權重相同，則將其隨機調用。
*/
public class LeastActiveLoadBalance extends AbstractLoadBalance {public static final String NAME = “leastactive”;
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
// invoker的總個數
int length = invokers.size();
// invoker最小的活躍數
int leastActive = -1;
// 相同最小活躍數(leastActive)的invoker個數
int leastCount = 0;
// 相同最小活躍數(leastActive)的下標
int[] leastIndexes = new int[length];
// the weight of every invokers
int[] weights = new int[length];
// 所有最不活躍invoker的預熱權重之和
int totalWeight = 0;
// 第一個最不活躍的invoker的權重, 用於於計算是否相同
int firstWeight = 0;
// 每個最不活躍的調用者都具有相同的權重值？
boolean sameWeight = true;<span class="hljs-comment">// Filter out all the least active invokers</span> <span class="hljs-keyword">for</span> (<span class="hljs-keyword">int</span> i = <span class="hljs-number">0</span>; i < length; i++) { Invoker<T> invoker = invokers.get(i); <span class="hljs-comment">// Get the active number of the invoker</span> <span class="hljs-keyword">int</span> active = RpcStatus.getStatus(invoker.getUrl(), invocation.getMethodName()).getActive(); <span class="hljs-comment">// Get the weight of the invoker's configuration. The default value is 100.</span> <span class="hljs-keyword">int</span> afterWarmup = getWeight(invoker, invocation); <span class="hljs-comment">// save for later use</span> weights[i] = afterWarmup; <span class="hljs-comment">// If it is the first invoker or the active number of the invoker is less than the current least active number</span> <span class="hljs-keyword">if</span> (leastActive == -<span class="hljs-number">1</span> || active < leastActive) { <span class="hljs-comment">// Reset the active number of the current invoker to the least active number</span> leastActive = active; <span class="hljs-comment">// Reset the number of least active invokers</span> leastCount = <span class="hljs-number">1</span>; <span class="hljs-comment">// Put the first least active invoker first in leastIndexes</span> leastIndexes[<span class="hljs-number">0</span>] = i; <span class="hljs-comment">// Reset totalWeight</span> totalWeight = afterWarmup; <span class="hljs-comment">// Record the weight the first least active invoker</span> firstWeight = afterWarmup; <span class="hljs-comment">// Each invoke has the same weight (only one invoker here)</span> sameWeight = <span class="hljs-keyword">true</span>; <span class="hljs-comment">// If current invoker's active value equals with leaseActive, then accumulating.</span> } <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> (active == leastActive) { <span class="hljs-comment">// 記錄leastIndexes order最小活躍數下標</span> leastIndexes[leastCount++] = i; <span class="hljs-comment">// 累計總權重</span> totalWeight += afterWarmup; <span class="hljs-comment">// If every invoker has the same weight?</span> <span class="hljs-keyword">if</span> (sameWeight && i > <span class="hljs-number">0</span> && afterWarmup != firstWeight) { sameWeight = <span class="hljs-keyword">false</span>; } } } <span class="hljs-comment">// Choose an invoker from all the least active invokers</span> <span class="hljs-keyword">if</span> (leastCount == <span class="hljs-number">1</span>) { <span class="hljs-comment">// 如果只有一個最小則直接返回</span> <span class="hljs-keyword">return</span> invokers.get(leastIndexes[<span class="hljs-number">0</span>]); } <span class="hljs-keyword">if</span> (!sameWeight && totalWeight > <span class="hljs-number">0</span>) { <span class="hljs-comment">// 如果權重不相同且權重大於0則按總權重數隨機</span> <span class="hljs-keyword">int</span> offsetWeight = ThreadLocalRandom.current().nextInt(totalWeight); <span class="hljs-comment">// 並確定隨機值落在哪個片斷上</span> <span class="hljs-keyword">for</span> (<span class="hljs-keyword">int</span> i = <span class="hljs-number">0</span>; i < leastCount; i++) { <span class="hljs-keyword">int</span> leastIndex = leastIndexes[i]; offsetWeight -= weights[leastIndex]; <span class="hljs-keyword">if</span> (offsetWeight < <span class="hljs-number">0</span>) { <span class="hljs-keyword">return</span> invokers.get(leastIndex); } } } <span class="hljs-comment">// 如果權重相同或權重爲0則均等隨機</span> <span class="hljs-keyword">return</span> invokers.get(leastIndexes[ThreadLocalRandom.current().nextInt(leastCount)]);}
}
分析
這部分代碼概括起來就兩部分
- 活躍數和權重的統計
- 選擇invoker.也就是他把最小活躍數的invoker統計到leastIndexs數組中,如果權重一致(這個一致的規則參考上面的隨機算法)或者總權重爲0,則均等隨機調用,如果不同,則從leastIndexs數組中按照權重比例調用(還是和隨機算法中的那個依次相減的思路一樣).還不明白沒關係,看下面的流程圖和數學分析
- 流程圖
![]()
理論
假設A,B,C,D節點的最小活躍數分別是1,1,2,3,權重爲1,2,3,4.則leastIndexs(該數組是最小活躍數組,因爲A,B的活躍數是1,均爲最小)數組內容爲[A,B].A,B的權重是1和2,所以調用A的概率爲 1/(1+2) = 1/3,B的概率爲 2/(1+2) = 2/3
活躍數的變化是在org.apache.dubbo.rpc.filter.ActiveLimitFilter中
如果沒有配置dubbo:reference的actives屬性,默認是調用前活躍數+1,調用結束-1
鑑於很多人可能沒用過這個屬性,所以我把文檔截圖貼出來
另外如果使用該種負載均衡算法,則dubbo:service中還需要配置filter="activelimit"
7 ConsistentHashLoadBalance(一致性哈希)
- 一致性 Hash，相同參數的請求總是發到同一提供者
- 當某一臺提供者掛時，原本發往該提供者的請求，基於虛擬節點，平攤到其它提供者，不會引起劇烈變動。
- 推薦閱讀
http://en.wikipedia.org/wiki/Consistent_hashing
缺省只對第一個參數 Hash，如果要修改，請配置
<dubbo:parameter key="hash.arguments" value="0,1" />缺省用 160 份虛擬節點，如果要修改，請配置
<dubbo:parameter key="hash.nodes" value="320" />package org.apache.dubbo.rpc.cluster.loadbalance;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.rpc.Invocation;
import org.apache.dubbo.rpc.Invoker;
import org.apache.dubbo.rpc.support.RpcUtils;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import static org.apache.dubbo.common.constants.CommonConstants.COMMA_SPLIT_PATTERN;
public class ConsistentHashLoadBalance extends AbstractLoadBalance {
public static final String NAME = “consistenthash”;
<span class="hljs-comment">/**
* Hash nodes name
*/</span>
<span class="hljs-keyword">public</span> <span class="hljs-keyword">static</span> <span class="hljs-keyword">final</span> String HASH_NODES = <span class="hljs-string">"hash.nodes"</span>;
<span class="hljs-comment">/**
* Hash arguments name
*/</span>
<span class="hljs-keyword">public</span> <span class="hljs-keyword">static</span> <span class="hljs-keyword">final</span> String HASH_ARGUMENTS = <span class="hljs-string">"hash.arguments"</span>;
<span class="hljs-keyword">private</span> <span class="hljs-keyword">final</span> ConcurrentMap<String, ConsistentHashSelector<?>> selectors = <span class="hljs-keyword">new</span> ConcurrentHashMap<String, ConsistentHashSelector<?>>();
<span class="hljs-meta">@SuppressWarnings</span>(<span class="hljs-string">"unchecked"</span>)
<span class="hljs-meta">@Override</span>
<span class="hljs-keyword">protected</span> <T> <span class="hljs-function">Invoker<T> <span class="hljs-title">doSelect</span><span class="hljs-params">(List<Invoker<T>> invokers, URL url, Invocation invocation)</span> </span>{
String methodName = RpcUtils.getMethodName(invocation);
String key = invokers.get(<span class="hljs-number">0</span>).getUrl().getServiceKey() + <span class="hljs-string">"."</span> + methodName;
<span class="hljs-keyword">int</span> identityHashCode = System.identityHashCode(invokers);
ConsistentHashSelector<T> selector = (ConsistentHashSelector<T>) selectors.get(key);
<span class="hljs-keyword">if</span> (selector == <span class="hljs-keyword">null</span> || selector.identityHashCode != identityHashCode) {
selectors.put(key, <span class="hljs-keyword">new</span> ConsistentHashSelector<T>(invokers, methodName, identityHashCode));
selector = (ConsistentHashSelector<T>) selectors.get(key);
}
<span class="hljs-keyword">return</span> selector.select(invocation);
}
<span class="hljs-keyword">private</span> <span class="hljs-keyword">static</span> <span class="hljs-keyword">final</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">ConsistentHashSelector</span><<span class="hljs-title">T</span>> </span>{
<span class="hljs-keyword">private</span> <span class="hljs-keyword">final</span> TreeMap<Long, Invoker<T>> virtualInvokers;
<span class="hljs-keyword">private</span> <span class="hljs-keyword">final</span> <span class="hljs-keyword">int</span> replicaNumber;
<span class="hljs-keyword">private</span> <span class="hljs-keyword">final</span> <span class="hljs-keyword">int</span> identityHashCode;
<span class="hljs-keyword">private</span> <span class="hljs-keyword">final</span> <span class="hljs-keyword">int</span>[] argumentIndex;
ConsistentHashSelector(List<Invoker<T>> invokers, String methodName, <span class="hljs-keyword">int</span> identityHashCode) {
<span class="hljs-keyword">this</span>.virtualInvokers = <span class="hljs-keyword">new</span> TreeMap<Long, Invoker<T>>();
<span class="hljs-keyword">this</span>.identityHashCode = identityHashCode;
URL url = invokers.get(<span class="hljs-number">0</span>).getUrl();
<span class="hljs-keyword">this</span>.replicaNumber = url.getMethodParameter(methodName, HASH_NODES, <span class="hljs-number">160</span>);
String[] index = COMMA_SPLIT_PATTERN.split(url.getMethodParameter(methodName, HASH_ARGUMENTS, <span class="hljs-string">"0"</span>));
argumentIndex = <span class="hljs-keyword">new</span> <span class="hljs-keyword">int</span>[index.length];
<span class="hljs-keyword">for</span> (<span class="hljs-keyword">int</span> i = <span class="hljs-number">0</span>; i < index.length; i++) {
argumentIndex[i] = Integer.parseInt(index[i]);
}
<span class="hljs-keyword">for</span> (Invoker<T> invoker : invokers) {
String address = invoker.getUrl().getAddress();
<span class="hljs-keyword">for</span> (<span class="hljs-keyword">int</span> i = <span class="hljs-number">0</span>; i < replicaNumber / <span class="hljs-number">4</span>; i++) {
<span class="hljs-keyword">byte</span>[] digest = md5(address + i);
<span class="hljs-keyword">for</span> (<span class="hljs-keyword">int</span> h = <span class="hljs-number">0</span>; h < <span class="hljs-number">4</span>; h++) {
<span class="hljs-keyword">long</span> m = hash(digest, h);
virtualInvokers.put(m, invoker);
}
}
}
}
<span class="hljs-function"><span class="hljs-keyword">public</span> Invoker<T> <span class="hljs-title">select</span><span class="hljs-params">(Invocation invocation)</span> </span>{
String key = toKey(invocation.getArguments());
<span class="hljs-keyword">byte</span>[] digest = md5(key);
<span class="hljs-keyword">return</span> selectForKey(hash(digest, <span class="hljs-number">0</span>));
}
<span class="hljs-function"><span class="hljs-keyword">private</span> String <span class="hljs-title">toKey</span><span class="hljs-params">(Object[] args)</span> </span>{
StringBuilder buf = <span class="hljs-keyword">new</span> StringBuilder();
<span class="hljs-keyword">for</span> (<span class="hljs-keyword">int</span> i : argumentIndex) {
<span class="hljs-keyword">if</span> (i >= <span class="hljs-number">0</span> && i < args.length) {
buf.append(args[i]);
}
}
<span class="hljs-keyword">return</span> buf.toString();
}
<span class="hljs-function"><span class="hljs-keyword">private</span> Invoker<T> <span class="hljs-title">selectForKey</span><span class="hljs-params">(<span class="hljs-keyword">long</span> hash)</span> </span>{
Map.Entry<Long, Invoker<T>> entry = virtualInvokers.ceilingEntry(hash);
<span class="hljs-keyword">if</span> (entry == <span class="hljs-keyword">null</span>) {
entry = virtualInvokers.firstEntry();
}
<span class="hljs-keyword">return</span> entry.getValue();
}
<span class="hljs-function"><span class="hljs-keyword">private</span> <span class="hljs-keyword">long</span> <span class="hljs-title">hash</span><span class="hljs-params">(<span class="hljs-keyword">byte</span>[] digest, <span class="hljs-keyword">int</span> number)</span> </span>{
<span class="hljs-keyword">return</span> (((<span class="hljs-keyword">long</span>) (digest[<span class="hljs-number">3</span> + number * <span class="hljs-number">4</span>] & <span class="hljs-number">0xFF</span>) << <span class="hljs-number">24</span>)
| ((<span class="hljs-keyword">long</span>) (digest[<span class="hljs-number">2</span> + number * <span class="hljs-number">4</span>] & <span class="hljs-number">0xFF</span>) << <span class="hljs-number">16</span>)
| ((<span class="hljs-keyword">long</span>) (digest[<span class="hljs-number">1</span> + number * <span class="hljs-number">4</span>] & <span class="hljs-number">0xFF</span>) << <span class="hljs-number">8</span>)
| (digest[number * <span class="hljs-number">4</span>] & <span class="hljs-number">0xFF</span>))
& <span class="hljs-number">0xFFFFFFFFL</span>;
}
<span class="hljs-keyword">private</span> <span class="hljs-keyword">byte</span>[] md5(String value) {
MessageDigest md5;
<span class="hljs-keyword">try</span> {
md5 = MessageDigest.getInstance(<span class="hljs-string">"MD5"</span>);
} <span class="hljs-keyword">catch</span> (NoSuchAlgorithmException e) {
<span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> IllegalStateException(e.getMessage(), e);
}
md5.reset();
<span class="hljs-keyword">byte</span>[] bytes = value.getBytes(StandardCharsets.UTF_8);
md5.update(bytes);
<span class="hljs-keyword">return</span> md5.digest();
}
}
}
該算法的代碼實現拿出來講的話篇幅較大,主要講三個關鍵詞,原理,down機影響,虛擬節點
原理
簡單講就是,假設我們有個時鐘,各服務器節點映射放在鐘錶的時刻上,把key也映射到鐘錶的某個時刻上,然後key順時針走,碰到的第一個節點則爲我們需要找的服務器節點
還是假如我們有a,b,c,d四個節點(感覺整篇文章都在做這個假如....),把他們通過某種規則轉成整數,分別爲0,3,6,9.所以按照時鐘分佈如下圖
假設這個key通過某種規則轉化成1,那麼他順時針碰到的第一個節點就是b,也就是b是我們要找的節點
這個規則你可以自己設計,但是要注意的是,不同的節點名,轉換爲相同的整數的概率就是衡量這個規則的好壞,如果你能做到不同的節點名唯一對應一個整數,那就是棒棒噠.當然java裏面的CRC32這個類你可以瞭解一下.
說到這裏可能又會有另個疑問,時鐘點數有限,萬一裝不下怎麼辦
其實這個時鐘只是方便大家理解做的比喻而已,在實際中,我們可以在圓環上分佈[0,2^32-1]的數字,這量級全世界的服務器都可以裝得下.
down機影響
通過上圖我們可以看出,當b節點掛了之後,根據順時針的規則,那麼目標節點就是c,也就是說,隻影響了一個節點,其他節點不受影響.
如果是輪詢的取模算法,假設從N臺服務器變成了N-1臺,那麼命中率就變成1/(N-1),因此服務器越多,影響也就越大.
虛擬節點
爲什麼要有虛擬節點的概念呢?我們還是回到第一個假設,我們還是有a,b,c,d四個節點,他們通過某個規則轉化成0,3,6,9這種自然是均勻的.但是萬一是0,1,2,3這樣,那就是非常不均勻了.事實上, 一般的Hash函數對於節點在圓環上的映射,並不均勻.所以我們需要引入虛擬節點,那麼什麼是虛擬節點呢?
假如有N個真實節點,把每個真實節點映射成M個虛擬節點,再把 M*N 個虛擬節點, 散列在圓環上. 各真實節點對應的虛擬節點相互交錯分佈這樣,某真實節點down後,則把其影響平均分擔到其他所有節點上.
也就是a,b,c,d的虛擬節點a0,a1,a2,b0,b1,b2,c0,c1,c2,d0,d1,d2散落在圓環上,假設C號節點down,則c0,c1,c2的壓力分別傳給d0,a1,b1,如下圖