Netty源碼學習系列①JavaNIO概覽

在正式開始Netty相關的學習之前，我決定還是要先回顧一下Java NIO，至少要對Java NIO相關的概念有一個瞭解，如Channel、ByteBuffer、Selector等。要自己動手寫一寫相關的demo實例、並且要儘可能地去了解其後面是如何實現的，也就是稍微看看相關jdk的源代碼。

Java NIO 由以下幾個核心部分組成：Buffer, Channel, Selector。傳統的IO操作面向數據流，面向流的 I/O 系統一次一個字節地處理數據，意味着每次從流中讀一個或多個字節，直至完成，數據沒有被緩存在任何地方；NIO操作面向緩衝區（面向塊），數據從Channel讀取到Buffer緩衝區，隨後在Buffer中處理數據。

Buffer

可以理解成煤礦裏面挖煤的小車，把煤從井底運地面上面。它的屬性與子類如下：

Buffer是一個抽象類，繼承自Object，擁有多個子類。此類在JDK源碼中的註釋如下：

A container for data of a specific primitive type.

A buffer is a linear, finite sequence of elements of a specific primitive type. Aside from its content, the essential properties of a buffer are its capacity, limit, and position:

A buffer’s capacity is the number of elements it contains. The capacity of a buffer is never negative and never changes.

A buffer’s limit is the index of the first element that should not be read or written. A buffer’s limit is never negative and is never greater than its capacity.

寫模式下，limit表示最多能往Buffer裏寫多少數據，等於capacity值；讀模式下，limit表示最多可以讀取多少數據，小於等於 capacity 值。

A buffer’s position is the index of the next element to be read or written. A buffer’s position is never negative and is never greater than its limit.

There is one subclass of this class for each non-boolean primitive type.

Transferring data

Each subclass of this class defines two categories of get and put operations:

Relative operations read or write one or more elements starting at the current position and then increment the position by the number of elements transferred. If the requested transfer exceeds the limit then a relative get operation throws a BufferUnderflowException and a relative put operation throws a BufferOverflowException; in either case, no data is transferred.

Absolute operations take an explicit element index and do not affect the position. Absolute get and put operations throw an IndexOutOfBoundsException if the index argument exceeds the limit.

Data may also, of course, be transferred in to or out of a buffer by the I/O operations of an appropriate channel, which are always relative to the current position.

Marking and resetting

A buffer’s mark is the index to which its position will be reset when the reset method is invoked. The mark is not always defined, but when it is defined it is never negative and is never greater than the position. If the mark is defined then it is discarded when the position or the limit is adjusted to a value smaller than the mark. If the mark is not defined then invoking the reset method causes an InvalidMarkException to be thrown.

Invariants

The following invariant holds for the mark, position, limit, and capacity values:

0 <= mark <= position <= limit <= capacity

A newly-created buffer always has a position of zero and a mark that is undefined. The initial limit may be zero, or it may be some other value that depends upon the type of the buffer and the manner in which it is constructed. Each element of a newly-allocated buffer is initialized to zero.

Clearing, flipping, and rewinding

In addition to methods for accessing the position, limit, and capacity values and for marking and resetting, this class also defines the following operations upon buffers:

clear() makes a buffer ready for a new sequence of channel-read or relative put operations: It sets the limit to the capacity and the position to zero.

flip() makes a buffer ready for a new sequence of channel-write or relative get operations: It sets the limit to the current position and then sets the position to zero.

rewind() makes a buffer ready for re-reading the data that it already contains: It leaves the limit unchanged and sets the position to zero.

Read-only buffers

Every buffer is readable, but not every buffer is writable. The mutation methods of each buffer class are specified as optional operations that will throw a ReadOnlyBufferException when invoked upon a read-only buffer. A read-only buffer does not allow its content to be changed, but its mark, position, and limit values are mutable. Whether or not a buffer is read-only may be determined by invoking its isReadOnly method.

Thread safety

Buffers are not safe for use by multiple concurrent threads. If a buffer is to be used by more than one thread then access to the buffer should be controlled by appropriate synchronization.

Invocation chaining

Methods in this class that do not otherwise have a value to return are specified to return the buffer upon which they are invoked. This allows method invocations to be chained; for example, the sequence of statements
b.flip();
b.position(23);
b.limit(42);
can be replaced by the single, more compact statement
b.flip().position(23).limit(42);

clear()方法

// 清除Buffer中的信息，只將參數恢復成默認
public final Buffer clear() {
    position = 0;
    limit = capacity;
    mark = -1;
    return this;
}

flip()方法

// 將limit記錄成當前的位置，指針指向頭部，爲讀取做準備
public final Buffer flip() {
    limit = position;
    position = 0;
    mark = -1;
    return this;
}

rewind()方法

// 指針指向頭部，可以用於再次讀取
public final Buffer rewind() {
    position = 0;
    mark = -1;
    return this;
}

如何使用Java NIO讀取文件內容

遇到坑了，但是感覺可以透過這個問題更加深入理解Java NIO的這些概念。出現問題的代碼：

public static void fileChannel() throws IOException {
    FileInputStream fis = new FileInputStream("/Users/yangyu/Documents/data.json");
    FileChannel fileChannel = fis.getChannel();
    ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
    while ((fileChannel.read(byteBuffer)) != -1) {
        while (byteBuffer.hasRemaining()) {
            System.out.print((char) byteBuffer.get());
        }
    }
}

上面的代碼讀取不到數據，一直在做循環，但是不輸出數據。爲什麼？因爲hasRemaining()是以position和limit作對比，如下：

public final boolean hasRemaining() {
		return position < limit;
}

當從fileChannel中讀取數據到byteBuffer中之後，limit與capacity相等（初始化既如此），此時的position也與capacity相同，導致hasRemaining()爲false，無法向控制檯輸出。

所以需要將position設置成從0開始，讓讀取從0開始，直到讀到之前的容量，所以使用flip()來完成這個目的，即：

此時卻發現控制檯無限打印東西，爲了弄明白這是爲什麼，我把byteBuffer的大小調成了8，跑起來之後的輸出如下：

這是爲什麼呢？這個問題應該與byteBuffer裏面的那幾個參數有關係：

猜測應該是與fileChannel.read(byteBuffer)中的具體實現有關。粗略看了看fileChannel.read(byteBuffer)的實現，大致流程如下：

計算byteBuffer的剩餘量，即limit - position。對於上面的情況，剩餘量爲0。
找出緩存buffer，此時緩存buffer爲上次read得到的，第一次爲空會直接分配；第二次read的時候，其3大屬性全部爲8，也即上次讀取的結果。
將緩存的buffer進行rewind()、flip(剩餘量)，得到一個[pos=0, limit=0, capacity=8]的buffer。

進行讀取的時候回根據緩存buffer的pos、limit來確定能讀取的數量，也即：

// 其中var1爲緩存buffer
int var5 = var1.position();
int var6 = var1.limit();

assert var5 <= var6;
// var6 - var5 = limit - position = 0
int var7 = var5 <= var6 ? var6 - var5 : 0;
// var7 = 0
if (var7 == 0) {
  // 0 即讀取的字節數
  return 0;
}

如果能讀到數據，會將緩存buffer裏面的的內容再轉移到byteBuffer（也就是我們read()裏面傳的ByteBuffer）中：

// var5即緩存buffer，讀取內容到var5中
int var6 = readIntoNativeBuffer(var0, var5, var2, var4);
// 準備用來讀取
var5.flip();
// var1是我們傳入的byteBuffer，如果讀取到的字節數大於0，
if (var6 > 0) {
  // 將var5中的內容拷貝到var1中
  var1.put(var5);
}

直到發現flip()的註釋裏面有這樣一段註釋：

Compacts this buffer (optional operation).
The bytes between the buffer’s current position and its limit, if any, are copied to the beginning of the buffer. That is, the byte at index p = position() is copied to index zero, the byte at index p + 1 is copied to index one, and so forth until the byte at index limit() - 1 is copied to index n = limit() - 1 - p. The buffer’s position is then set to n+1 and its limit is set to its capacity. The mark, if defined, is discarded.

The buffer’s position is set to the number of bytes copied, rather than to zero, so that an invocation of this method can be followed immediately by an invocation of another relative put method.

Invoke this method after writing data from a buffer in case the write was incomplete. The following loop, for example, copies bytes from one channel to another via the buffer buf:
buf.clear();          // Prepare buffer for use
while (in.read(buf) >= 0 || buf.position != 0) {
  buf.flip();
  out.write(buf);
  buf.compact();    // In case of partial write
}

加上這段代碼buf.compact()便可以正常讀取文件內容。到這裏就有點心累了，爲什麼寫個讀取都這麼多坑。感覺有問題的時候往這三個參數上面想就行了。

Channel

煤礦廠裏面運煤的通道，需要看的子類總共有4個，分別爲：

FileChannel：文件通道，用於文件的讀和寫。不支持非阻塞
DatagramChannel：用於 UDP 連接的接收和發送
SocketChannel：把它理解爲 TCP 連接通道，簡單理解就是 TCP 客戶端
ServerSocketChannel：TCP 對應的服務端，用於監聽某個端口進來的請求

Selector

只有自己寫過的代碼纔會有更深刻的印象，哪怕是從別的地方抄來的，自己慢慢debug一下，找出自己對代碼的疑問，然後再去搞清楚這些問題，我覺得這樣讓我對它的瞭解會更深。這兩段代碼基本和網上的教程類似，大部分是抄的，但是自己有一定的加工，也遇到了1個問題，外加一個疑問。

客戶端代碼

客戶端的代碼很簡單：①讀標準輸入。②發送給Server端。

// Client端的代碼很像八股文，這樣弄就行了。
public static void main(String[] args) throws Exception {
    SocketChannel sc = SocketChannel.open();
    sc.configureBlocking(false);
    sc.connect(new InetSocketAddress("127.0.0.1", 8086));

    Scanner scanner = new Scanner(System.in);
    if (sc.finishConnect()) {
        ByteBuffer buffer = ByteBuffer.allocate(1024);
        while (scanner.hasNextLine()) {
          	// 讀標準輸入
            String info = scanner.nextLine();
            buffer.clear();
            buffer.put(info.getBytes());
            buffer.flip();
            while (buffer.hasRemaining()) {
                System.out.println(buffer);
              	// 發送給Server端
                sc.write(buffer);
            }
        }
    }
}

服務端代碼

主要參考了一篇CSDN上的博客和一篇簡書上的博客，簡書上面的這邊對我的幫助很大，十分感謝。我的問題主要有兩點，第一個是少了it.remove();，第二個是關於如何觸發SelectionKey.OP_WRITE事件。

// 八股文的感覺。
public static void startServer() throws IOException {
    ServerSocketChannel ssc = ServerSocketChannel.open();
    ssc.configureBlocking(false);
    ssc.socket().bind(new InetSocketAddress(8086));

    Selector selector = Selector.open();
    ssc.register(selector, SelectionKey.OP_ACCEPT);

    while (true) {
        selector.select();
        Iterator<SelectionKey> it = selector.selectedKeys().iterator();
        while (it.hasNext()) {
            SelectionKey key = it.next();
            // 一定要remove掉，不然上次的事件會累積。
            // 也就是對同一事件會處理兩次，這樣可能會導致報錯。
            it.remove();
            if (key.isAcceptable()) {
                System.out.println("ACCEPT");
                ServerSocketChannel ssChannel = (ServerSocketChannel)key.channel();
                SocketChannel sc = ssChannel.accept();
                sc.configureBlocking(false);
                sc.register(key.selector(), SelectionKey.OP_READ, ByteBuffer.allocateDirect(BUF_SIZE));
            } else if (key.isReadable()) {
                System.out.print("READ：");
                SocketChannel sc = (SocketChannel)key.channel();
                ByteBuffer buf = (ByteBuffer)key.attachment();
                long bytesRead = sc.read(buf);
                while(bytesRead>0){
                    buf.flip();
                    while(buf.hasRemaining()){
                        System.out.print((char)buf.get());
                    }
                    System.out.println();
                    buf.clear();
                    bytesRead = sc.read(buf);
                }
                if(bytesRead == -1){
                    sc.close();
                }
            } else if (key.isWritable()) {
                // OP_WRITE事件如何觸發？
                System.out.print("WRITE：");
                ByteBuffer buf = (ByteBuffer) key.attachment();
                buf.flip();
                SocketChannel sc = (SocketChannel) key.channel();
                while(buf.hasRemaining()){
                    sc.write(buf);
                }
                buf.compact();
            } else if (key.isConnectable()) {
                System.out.println("CONNECT");
            } else {
                System.out.println("UNKNOWN");
            }
        }
    }
}

如果缺少it.remove()方法的調用，那麼會導致事件會堆積在Selector的Set<SelectionKey> publicSelectedKeys中，引發對同一事件會處理兩次，這樣可能會導致報錯。
如何觸發SelectionKey.OP_WRITE？因爲我看到大部分關於selector的博客，都沒有寫如何觸發該事件，並且也未對讀事件做出說明。

首先肯定要在調用ssChannel.accept()之後，將得到的SocketChannel多註冊一個OP_WRITE事件。即修改成：

SocketChannel sc = ssChannel.accept();
sc.configureBlocking(false);
sc.register(key.selector(), SelectionKey.OP_READ | SelectionKey.OP_WRITE,ByteBuffer.allocateDirect(BUF_SIZE));

然後會發現程序卡死，屏幕一直輸出Write。爲什麼會有這麼多OP_WRITE事件？因爲Java NIO的事件觸發是水平觸發，即只要滿足條件，就觸發一個事件，所以只要內核緩衝區還不滿，就一直髮出OP_WRITE事件。

與水平觸發對應的還有一個叫做邊緣觸發，即每當狀態變化時，觸發一個事件。對之前的Netty的事件是邊緣觸發又有了一個認識。

Netty源碼學習系列①JavaNIO概覽

Buffer

Transferring data

Marking and resetting

Invariants

Clearing, flipping, and rewinding

Read-only buffers

Thread safety

Invocation chaining

如何使用Java NIO讀取文件內容

Channel

Selector

客戶端代碼

服務端代碼

Netty源碼學習系列①JavaNIO概覽

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結