zookeeper-curator分佈式鎖實現及源碼分析
zookeeper-curator分佈式鎖實現
1. curator分佈式鎖類型
1.1. InterProcessMutex
特性：分佈式可重入排它鎖，此鎖可以重入，但是重入幾次需要釋放幾次
原理：InterProcessMutex通過在zookeeper的某路徑節點下創建臨時序列節點來實現分佈式鎖，即每個線程（跨進程的線程）獲取同一把鎖前，都需要在同樣的路徑下創建一個節點，節點名字由uuid + 遞增序列組成。而通過對比自身的序列數是否在所有子節點的第一位，來判斷是否成功獲取到了鎖。當獲取鎖失敗時，它會添加watcher來監聽前一個節點的變動情況，然後進行等待狀態。直到watcher的事件生效將自己喚醒，或者超時時間異常返回。
1.2. InterProcessSemaphoreMutex
InterProcessSemaphoreMutex是一種不可重入的互斥鎖，也就意味着即使是同一個線程也無法在持有鎖的情況下再次獲得鎖，所以需要注意，不可重入的鎖很容易在一些情況導致死鎖
1.3. InterProcessReadWriteLock
分佈式讀寫鎖
1.4. InterProcessMultiLock
將多個鎖作爲單個實體管理的容器,創建多重鎖對象
例如
// 可重入鎖
final InterProcessLock interProcessLock1 = new InterProcessMutex(client, lockPath);
// 不可重入鎖
final InterProcessLock interProcessLock2 = new InterProcessSemaphoreMutex(client2, lockPath);
// 創建多重鎖對象
final InterProcessLock lock = new InterProcessMultiLock(Arrays.asList(interProcessLock1, interProcessLock2));
1.5 InterProcessSemaphoreV2
共享信號量
2. InterProcessMutex實現分佈式鎖
public class DistributedLockDemo {
public static void main(String[] args) {
CuratorFramework curatorFramework = CuratorFrameworkFactory.builder().
connectString("114.55.254.108:2181,114.55.254.108:2182,114.55.254.108:2183").
sessionTimeoutMs(5000).
// ExponentialBackoffRetry 重試指定次數
// ReTryOneTime 僅重試一次
retryPolicy(new ExponentialBackoffRetry(1000,3)).
build();
curatorFramework.start(); //啓動
InterProcessLock interProcessLock = new InterProcessMutex(curatorFramework,"/path");
for (int i = 0; i < 5; i++) {
new Thread(()->{
System.out.println(Thread.currentThread().getName() + " 嘗試獲取鎖");
try {
interProcessLock.acquire();
System.out.println(Thread.currentThread().getName() + " 成功獲取到鎖");
Thread.sleep(1000);
} catch (Exception e) {
e.printStackTrace();
}finally {
try {
interProcessLock.release();
System.out.println(Thread.currentThread().getName() + " 成功釋放鎖");
} catch (Exception e) {
e.printStackTrace();
}
}
},"thread-" + i).start();
}
}
}
執行結果
thread-1 嘗試獲取鎖
thread-0 嘗試獲取鎖
thread-2 嘗試獲取鎖
thread-3 嘗試獲取鎖
thread-4 嘗試獲取鎖
thread-4 成功獲取到鎖
thread-4 成功釋放鎖
thread-1 成功獲取到鎖
thread-1 成功釋放鎖
thread-3 成功獲取到鎖
thread-3 成功釋放鎖
thread-2 成功獲取到鎖
thread-2 成功釋放鎖
thread-0 成功獲取到鎖
thread-0 成功釋放鎖
3. InterProcessMutex源碼分析
3.1 鎖的初始化
InterProcessLock interProcessLock = new InterProcessMutex(curatorFramework,"/path");
public InterProcessMutex(CuratorFramework client, String path)
{
this(client, path, new StandardLockInternalsDriver());
}
LockInternalsDriver的實現類StandardLockInternalsDriver後面加鎖的時候會用到
public InterProcessMutex(CuratorFramework client, String path, LockInternalsDriver driver)
{
this(client, path, LOCK_NAME, 1, driver);
}
maxLeases爲1，在後面進行節點監聽會用到
3.2 獲取鎖
//獲取鎖
public void acquire() throws Exception
{
if ( !internalLock(-1, null) )
{
throw new IOException("Lost connection while trying to acquire lock: " + basePath);
}
}
private boolean internalLock(long time, TimeUnit unit) throws Exception
{
// 同一個線程可以多次獲取鎖
Thread currentThread = Thread.currentThread();
LockData lockData = threadData.get(currentThread);
if ( lockData != null )
{
// re-entering
lockData.lockCount.incrementAndGet();
return true;
}
// 如果是不同線程嘗試去獲取鎖
String lockPath = internals.attemptLock(time, unit, getLockNodeBytes());
// 如果lockPath 不爲空代表獲取鎖成功
if ( lockPath != null )
{
LockData newLockData = new LockData(currentThread, lockPath);
threadData.put(currentThread, newLockData);
return true;
}
return false;
}
LockData lockData = threadData.get(currentThread);
上面的代碼可以看出如果是同一線程可以多次直接返回獲取鎖成功，只是計數會增加，可見InterProcessMutex是支持可重入的
String attemptLock(long time, TimeUnit unit, byte[] lockNodeBytes) throws Exception
{
final long startMillis = System.currentTimeMillis();
final Long millisToWait = (unit != null) ? unit.toMillis(time) : null;
final byte[] localLockNodeBytes = (revocable.get() != null) ? new byte[0] : lockNodeBytes;
int retryCount = 0;
String ourPath = null;
boolean hasTheLock = false;
boolean isDone = false;
while ( !isDone )
{
isDone = true;
try
{
ourPath = driver.createsTheLock(client, path, localLockNodeBytes);
hasTheLock = internalLockLoop(startMillis, millisToWait, ourPath);
}
catch ( KeeperException.NoNodeException e )
{
// gets thrown by StandardLockInternalsDriver when it can't find the lock node
// this can happen when the session expires, etc. So, if the retry allows, just try it all again
if ( client.getZookeeperClient().getRetryPolicy().allowRetry(retryCount++, System.currentTimeMillis() - startMillis, RetryLoop.getDefaultRetrySleeper()) )
{
isDone = false;
}
else
{
throw e;
}
}
}
if ( hasTheLock )
{
return ourPath;
}
return null;
}
driver.createsTheLock(client, path, localLockNodeBytes);
去創建節點
// 創建臨時有序節點
public String createsTheLock(CuratorFramework client, String path, byte[] lockNodeBytes) throws Exception
{
String ourPath;
if ( lockNodeBytes != null )
{
ourPath = client.create().creatingParentContainersIfNeeded().withProtection().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(path, lockNodeBytes);
}
else
{
ourPath = client.create().creatingParentContainersIfNeeded().withProtection().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(path);
}
return ourPath;
}
client.create().creatingParentContainersIfNeeded().withProtection().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(path);
創建節點的類型是臨時有序節點
private boolean internalLockLoop(long startMillis, Long millisToWait, String ourPath) throws Exception
{
boolean haveTheLock = false;
boolean doDelete = false;
try
{
if ( revocable.get() != null )
{
client.getData().usingWatcher(revocableWatcher).forPath(ourPath);
}
while ( (client.getState() == CuratorFrameworkState.STARTED) && !haveTheLock )
{
List<String> children = getSortedChildren();
String sequenceNodeName = ourPath.substring(basePath.length() + 1); // +1 to include the slash
PredicateResults predicateResults = driver.getsTheLock(client, children, sequenceNodeName, maxLeases);
if ( predicateResults.getsTheLock() )
{
//超線程獲取到鎖直接返回
haveTheLock = true;
}
else
{
//該線程未獲取到鎖
String previousSequencePath = basePath + "/" + predicateResults.getPathToWatch();
synchronized(this)
{
try
{
// 把當前線程對應的子節點綁定到上一個子節點對應的監聽器上，該線程被阻塞，當上一個子節點對應的線程結束後，對應的臨時節點就會消失，此時會觸發監聽事件，在監聽事件中，喚醒該線程，則獲取鎖成功
client.getData().usingWatcher(watcher).forPath(previousSequencePath);
// 超時等待
if ( millisToWait != null )
{
millisToWait -= (System.currentTimeMillis() - startMillis);
startMillis = System.currentTimeMillis();
if ( millisToWait <= 0 )
{
// 超時刪除節點標誌位
doDelete = true; // timed out - delete our node
break;
}
// 線程未獲取鎖進行超時等待
wait(millisToWait);
}
else
{
// 線程未獲取鎖進行阻塞
wait();
}
}
catch ( KeeperException.NoNodeException e )
{
// it has been deleted (i.e. lock released). Try to acquire again
}
}
}
}
}
catch ( Exception e )
{
ThreadUtils.checkInterrupted(e);
doDelete = true;
throw e;
}
finally
{
// 如果線程等待時間超時則會刪除該臨時節點
if ( doDelete )
{
deleteOurPath(ourPath);
}
}
return haveTheLock;
}
getSortedChildren()
對path下的所有的臨時有序子節點進行排序
public PredicateResults getsTheLock(CuratorFramework client, List<String> children, String sequenceNodeName, int maxLeases) throws Exception
{
int ourIndex = children.indexOf(sequenceNodeName);
validateOurIndex(sequenceNodeName, ourIndex);
boolean getsTheLock = ourIndex < maxLeases;
String pathToWatch = getsTheLock ? null : children.get(ourIndex - maxLeases);
return new PredicateResults(pathToWatch, getsTheLock);
}
上面的方法在類StandardLockInternalsDriver中從初始化中可以看出來，根據children.indexOf(sequenceNodeName)獲取本節點的排序位置，
maxLeases在初始化時賦值爲1
例如：如果該節點爲0號節點，此時ourIndex=0 ， maxLeases=1，則getsTheLock=true；獲取到鎖，直接返回；
r如果此時再有1號節點進行爭搶鎖時，此時此時ourIndex=1，maxLeases=1，則getsTheLock=false；獲取鎖失敗，此時
pathToWatch=children.get(0)；則pathToWatch爲0號節點，也即是1號對應的線程的PredicateResults對象中的節點是0號節點的信息；
在後面進行綁定監聽器，在就會出現1號節點綁定在0號監聽器，以此類推，2號綁定1號上，3號綁定的2號號上，
爲何如此設計？
原因是：防止出現驚羣效應
//初始化StandardLockInternalsDriver代碼
public InterProcessMutex(CuratorFramework client, String path)
{
this(client, path, new StandardLockInternalsDriver());
}
3.3 釋放鎖
鎖是通過release進行釋放的
public void release() throws Exception
{
/*
Note on concurrency: a given lockData instance
can be only acted on by a single thread so locking isn't necessary
*/
Thread currentThread = Thread.currentThread();
LockData lockData = threadData.get(currentThread);
if ( lockData == null )
{
throw new IllegalMonitorStateException("You do not own the lock: " + basePath);
}
// 同一線程多次獲取鎖，計數進行減一
int newLockCount = lockData.lockCount.decrementAndGet();
if ( newLockCount > 0 )
{
return;
}
if ( newLockCount < 0 )
{
throw new IllegalMonitorStateException("Lock count has gone negative for lock: " + basePath);
}
try
{
// 釋放鎖
internals.releaseLock(lockData.lockPath);
}
finally
{
threadData.remove(currentThread);
}
}
釋放鎖也就是刪除該線程對應的臨時有序子節點
void releaseLock(String lockPath) throws Exception
{
revocable.set(null);
deleteOurPath(lockPath);
}
private void deleteOurPath(String ourPath) throws Exception
{
try
{
client.delete().guaranteed().forPath(ourPath);
}
catch ( KeeperException.NoNodeException e )
{
// ignore - already deleted (possibly expired session, etc.)
}
}