本源碼使用的Kafka Client是0.10.0.1
NetworkClient是一個通用的網絡客戶端實現，Kafka生產者和消費者都使用NetworkClient組件和服務端Broker之間進行通訊。
public class NetworkClient implements KafkaClient {
private static final Logger log = LoggerFactory.getLogger(NetworkClient.class);
/* the selector used to perform network i/o */
//網絡I/O，發送和接受消息
private final Selectable selector;
......
}
NetworkClient中負責網絡I/O的是Selectable selector接口，接下來主要分析下Selectable接口的實現Selector類。
Selector類
Selector類(在org.apache.kafka.common.network包下)，Selector底層封裝了Java NIO，使用一個單獨的線程可以管理多條網絡連接上的鏈接、讀、寫等操作。該類的核心屬性如下：
核心屬性及作用
//java.nio.channels.Selector類型，用來監聽網絡I/O事件。
private final java.nio.channels.Selector nioSelector;
//維護了NodeId與KafkaChannel之間的映射關係，表示生產者客戶端與各個Node之間的網絡鏈接。
//KafkaChannel是在SocketChannel上的又一層封裝。其中Send和NetworkReceive分別表示讀和寫時用的緩存，此等通過ByteBuffer實現，
//TransportLayer封裝了SocketChannel及SelectionKey，TransportLayer根據網絡協議的不同，提供不同的子類，而對KafkaChannel提供統一的接口
private final Map<String, KafkaChannel> channels;
//記錄已經完全發送出去的請求
private final List<Send> completedSends;
//記錄已經完全接受到的請求
private final List<NetworkReceive> completedReceives;
//記錄從連接中讀取到的消息
//暫停一次OP_READ事件處理完成之後，會將stagedReceives集合中的請求保存到completeReceives集合中
private final Map<KafkaChannel, Deque<NetworkReceive>> stagedReceives;
//記錄剛剛創建的連接SelectionKey，因爲是異步的，所以不知道該連接是否連接完成
private final Set<SelectionKey> immediatelyConnectedKeys;
//記錄一次poll過程中發現的斷開鏈接
private final List<String> disconnected;
//記錄一次poll過程中新建立的連接
private final List<String> connected;
//記錄向哪些Node發送的請求失敗了
private final List<String> failedSends;
//用於創建KafkaChannel的Builder。根據不同配置創建不同的TransportLayer的子類，然後創建KafkaChannel。
private final ChannelBuilder channelBuilder;
//LinkedHashMap類型，用來記錄各個鏈接的使用情況，並根據此關閉空閒時間超過connectionsMaxIdleNanos的鏈接
private final Map<String, Long> lruConnections;
//連接最大空閒的時間，單位：ns
private final long connectionsMaxIdleNanos;
//最大接受的消息大小
private final int maxReceiveSize;
接下來主要看一下Selector類中的常用方法
構造器
public Selector(int maxReceiveSize, long connectionMaxIdleMs, Metrics metrics, Time time, String metricGrpPrefix, Map<String, String> metricTags, boolean metricsPerConnection, ChannelBuilder channelBuilder) {
try {
this.nioSelector = java.nio.channels.Selector.open(); //創建一個新的nioSelector
} catch (IOException e) {
throw new KafkaException(e);
}
this.maxReceiveSize = maxReceiveSize;
this.connectionsMaxIdleNanos = connectionMaxIdleMs * 1000 * 1000;
this.time = time;
this.metricGrpPrefix = metricGrpPrefix;
this.metricTags = metricTags;
this.channels = new HashMap<>();
this.completedSends = new ArrayList<>();
this.completedReceives = new ArrayList<>();
this.stagedReceives = new HashMap<>();
this.immediatelyConnectedKeys = new HashSet<>();
this.connected = new ArrayList<>();
this.disconnected = new ArrayList<>();
this.failedSends = new ArrayList<>();
this.sensors = new SelectorMetrics(metrics);
this.channelBuilder = channelBuilder;
// initial capacity and load factor are default, we set them explicitly because we want to set accessOrder = true
this.lruConnections = new LinkedHashMap<>(16, .75F, true);
currentTimeNanos = time.nanoseconds();
nextIdleCloseCheckTime = currentTimeNanos + connectionsMaxIdleNanos;
this.metricsPerConnection = metricsPerConnection;
}
構造器中主要對nioSelector和一些屬性進行初始化操作
connect方法
public void connect(String id, InetSocketAddress address, int sendBufferSize, int receiveBufferSize) throws IOException {
if (this.channels.containsKey(id)) //已經包含該node的id，則拋出異常
throw new IllegalStateException("There is already a connection for id " + id);
//創建SocketChannel
SocketChannel socketChannel = SocketChannel.open();
socketChannel.configureBlocking(false); //配置成非阻塞模式
Socket socket = socketChannel.socket();
socket.setKeepAlive(true); //設置爲長連接
if (sendBufferSize != Selectable.USE_DEFAULT_BUFFER_SIZE)
socket.setSendBufferSize(sendBufferSize); //設置SO_SNDBUF大小
if (receiveBufferSize != Selectable.USE_DEFAULT_BUFFER_SIZE)
socket.setReceiveBufferSize(receiveBufferSize);//設置SO_RCVBUF大小
socket.setTcpNoDelay(true); //Tcp無延遲
boolean connected;
try {
/*
* 因爲是非阻塞方式，所以SocketChannel.connect()方法是發起一個連接，
* connect方法在連接正式建立之前就可能返回，在後面會通過Selector.finishConnect()方法確認連接
* 是否真正建立了。
* */
connected = socketChannel.connect(address); //發起連接到kafka服務端
} catch (UnresolvedAddressException e) {
socketChannel.close();
throw new IOException("Can't resolve address: " + address, e);
} catch (IOException e) {
socketChannel.close();
throw e;
}
//將這個socketChannel註冊到nioSelector上，並關注OP_CONNECT事件
SelectionKey key = socketChannel.register(nioSelector, SelectionKey.OP_CONNECT);
//創建KafkaChannel
KafkaChannel channel = channelBuilder.buildChannel(id, key, maxReceiveSize);
key.attach(channel); //將KafkaChannel註冊到key上
this.channels.put(id, channel);//將NodeId和KafkaChannel綁定，放到channels中管理
if (connected) {
// OP_CONNECT won't trigger for immediately connected channels
log.debug("Immediately connected to node {}", channel.id());
immediatelyConnectedKeys.add(key);
key.interestOps(0);
}
}
創建SocketChannel並配置成非阻塞模式，將SocketChannel關聯的Socket設置成長連接，SocketChannel連接到遠程地址，並將SocketChannel註冊到nio的Selector對象上，註冊的事件爲SelectionKey.OP_CONNECT，並返回SelectionKey key
創建KafkaChannel對象，並將該對象附加(attach)到key上，後續有事件發生時會獲取該KafkaChannel對象。KafkaChannel中維護了當前key，用於處理最終的讀寫
由於java NIO是異步的，所以無法知道該連接是否完成，所以先將該key添加到immediatelyConnectedKeys中，稍後處理。
select方法
private int select(long ms) throws IOException {
if (ms < 0L)
throw new IllegalArgumentException("timeout should be >= 0");
if (ms == 0L)
return this.nioSelector.selectNow();
else
return this.nioSelector.select(ms);
}
我看到select方法主要調用java NIO的Selector對象獲取就緒的事件，
selectNow()不會阻塞，如果沒有事件就緒也會直接返回
select(ms) 會阻塞直到事件產生或者超時
poll方法
public void poll(long timeout) throws IOException {
if (timeout < 0)
throw new IllegalArgumentException("timeout should be >= 0");
clear(); //將上一次poll()方法的結果全部清除掉
if (hasStagedReceives() || !immediatelyConnectedKeys.isEmpty())
timeout = 0;
/* check ready keys */
long startSelect = time.nanoseconds();
//調用nioSelector.select()方法，等待I/O事件發送
int readyKeys = select(timeout);
long endSelect = time.nanoseconds();
currentTimeNanos = endSelect;
this.sensors.selectTime.record(endSelect - startSelect, time.milliseconds()); //統計select阻塞的時間
//有事件發生 或者 immediatelyConnectedKeys集合不爲空
if (readyKeys > 0 || !immediatelyConnectedKeys.isEmpty()) {
//處理I/O事件
pollSelectionKeys(this.nioSelector.selectedKeys(), false);
pollSelectionKeys(immediatelyConnectedKeys, true);
}
//將stagedReceives複製到completedReceives集合中
addToCompletedReceives();
long endIo = time.nanoseconds();
this.sensors.ioTime.record(endIo - endSelect, time.milliseconds());
maybeCloseOldestConnection(); //關閉長期空閒的連接
}
首先將上一次poll()方法的結果全部清除掉，並獲取select的參數timeout，即最長阻塞時間。調用java nio Selector的select方法獲取就緒的SelectionKey，即readyKeys。
如果有事件發生或immediatelyConnectedKeys集合(剛剛創建連接，還不確定是否連接完成的SelectionKey)不爲空，則調用pollSelectionKeys方法進行處理。
pollSelectionKeys方法
pollSelectionKeys方法主要處理已經就緒的Key 和 immediatelyConnectedKeys集合不爲空的情況
private void pollSelectionKeys(Iterable<SelectionKey> selectionKeys, boolean isImmediatelyConnected) {
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while (iterator.hasNext()) {
SelectionKey key = iterator.next();
iterator.remove();
//之前創建連接時，將KafkaChannel註冊到key上，就是爲了在這裏獲取
KafkaChannel channel = channel(key);
// register all per-connection metrics at once
sensors.maybeRegisterConnectionMetrics(channel.id());
lruConnections.put(channel.id(), currentTimeNanos); //更新lru信息
try {
/* complete any connections that have finished their handshake (either normally or immediately) */
//對connect方法返回true或OP_CONNECTION事件的處理
if (isImmediatelyConnected || key.isConnectable()) {
//finishConnect方法會先檢測socktChannel是否建立完成，完成後，
//會取消OP_CONNECT事件關注，開始關注OP_READ事件
if (channel.finishConnect()) {
this.connected.add(channel.id());//添加到"已連接"的集合中
this.sensors.connectionCreated.record();
} else
continue; //連接未完成，則跳過對此Channel的後續處理
}
/* if channel is not ready finish prepare */
//調用KafkaChannel.prepare()方法進行身份驗證，
if (channel.isConnected() && !channel.ready())
channel.prepare();
/* if channel is ready read from any connections that have readable data */
if (channel.ready() && key.isReadable() && !hasStagedReceive(channel)) {
//OP_READ事件處理
NetworkReceive networkReceive;
while ((networkReceive = channel.read()) != null)
/*
* 上面channel.read()讀取到一個完整的NetworkReceive，則將其添加到stagedReceives中保存
* 若讀取不到一個完整的NetworkReceive，則返回null，下次處理OP_READ事件時，繼續讀取，
* 直至讀取到一個完整的NetworkReceive
* 將讀取到的消息記錄到stagedReceives中
* */
addToStagedReceives(channel, networkReceive);
}
/* if channel is ready write to any sockets that have space in their buffer and for which we have data */
if (channel.ready() && key.isWritable()) { //OP_WRITE事件處理
//上面的channel.write()方法將KafkaChannel.send字段發送出去，如果未完成發送，則返回null
//如果發送完成，則返回send，並添加到completeSends集合中，待後續處理
Send send = channel.write();
if (send != null) {
this.completedSends.add(send); //添加到completedSends集合
this.sensors.recordBytesSent(channel.id(), send.size());
}
}
/*
* completedSends和completedReceives分別表示在Selector端已經發送的和接受到的請求，它們會在NetworkClient的poll調用之後被不同的
* handleCompleteXXX()方法處理
* */
/* cancel any defunct sockets */
if (!key.isValid()) {
close(channel);
this.disconnected.add(channel.id());
}
/*
* 通過isValid()的返回值以及執行過程中是否拋出異常來判斷連接的狀態，
* 並將斷開的連接收集到disconnected集合，並在後續操作中進行重連。
* */
} catch (Exception e) {
//拋出異常，則任務連接關閉，將對應NodeId添加到disconnected集合
String desc = channel.socketDescription();
if (e instanceof IOException)
log.debug("Connection with {} disconnected", desc, e);
else
log.warn("Unexpected error from {}; closing connection", desc, e);
close(channel);
this.disconnected.add(channel.id());
}
}
}
遍歷已經就緒的SelectionKey，首先從集合中刪除當前key。調用channel方法獲取當前key對應的KafkaChannel。
private KafkaChannel channel(SelectionKey key) {
return (KafkaChannel) key.attachment();
}
解析來分別處理OP_CONNECTION、OP_READ和OP_WRITE事件
處理OP_ CONNECTION事件
調用KafkaChannel的finishConnect方法判斷連接是否完成
finishConnect方法底層調用KafkaChannel通訊層TransportLayer的finishConnect進行判斷，我們看TransportLayer其中一個實現類PlaintextTransportLayer的finishConnect方法
public boolean finishConnect() throws IOException {
boolean connected = socketChannel.finishConnect();
if (connected)
key.interestOps(key.interestOps() & ~SelectionKey.OP_CONNECT | SelectionKey.OP_READ);
return connected;
}
KafkaChannel的通訊層TransportLayer封裝了SocketChannel通道，首先調用socketChannel的finishConnect()判斷當前SocketChannel對應的Socket連接是否完成，如果已完成，會取消OP_CONNECT事件關注，開始關注OP_READ事件。
如果已完成則將該KafkaChannel對應的Node Id添加到屬性connected集合中
處理OP_READ事件
將讀取到的完整消息，寫入到stagedReceives
處理OP_WRITE事件
如果有緩存的等待發送的消息，則發送緩存的消息。待發送的消息緩存在KafkaChannel的Send send字段中。如果消息發送成功，則將已發送的消息添加到completedSends集合。
send方法
send方法主要對外提供發送消息的功能
//並沒有網絡I/O
public void send(Send send) {
KafkaChannel channel = channelOrFail(send.destination());
try {
channel.setSend(send);
} catch (CancelledKeyException e) {
this.failedSends.add(send.destination());
close(channel);
}
}
將之前創建的RequestSend對象緩存到KafkaChannel的send字段中，並開始關注此連接的OP_WRITE事件。
wakeup方法
喚醒阻塞在I/O的線程
public void wakeup() {
this.nioSelector.wakeup();
}
其他方法就不在累述，以上是Selector中主要使用Java NIO進行處理的方法。當然KafkaChannel中的TransportLayer對象中存儲了SocketChannel socketChannel，以上的pollSelectionKeys方法處理讀寫事件都是調用該socketChannel進行讀寫，有興趣可自行閱讀。