Netty(一)：IO通信模型

[TOC]

BIO:同步阻塞模型

在傳統的同步阻塞模型中，ServerSocket負責綁定IP地址，監聽端口；Socket負責發起連接操作。連接成功後，雙方通過輸入和輸出流進行同步阻塞式通信。 採用BIO通信模型的Server端，由一個Acceptor線程負責監聽客戶端的連接，它接收到客戶端的連接後，爲每一個客戶端創建一個新的線程進行鏈路處理，完成後，通過輸出流返回客戶端，線程銷燬。該模型最大的缺點就是缺乏伸縮能力，當客戶端併發量增加的時候，服務端的線程和客戶端的線程成1:1的正比關係，當線程數過多系統的性能將急劇下降，最終導致宕機，不能對外提供服務的嚴重情況。

Server:

package netty.bio; 

import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;

/**
 * @author wang.xw
 * @date 2018/3/13 13:44.
 */
public class TimeServer {
    private static final int DEFAULT_SERVER_PORT = 8081;

    public static void main(String[] args) {
        ServerSocket server = null;
        try {
            server = new ServerSocket(DEFAULT_SERVER_PORT);
            System.out.println("The time server is start in port ：" + DEFAULT_SERVER_PORT);
            while (true) {
                System.out.println("...");
                Socket socket = server.accept();
                new Thread(new TimerServerHadler(socket)).start();
            }

        } catch (IOException e) {
        } finally {
            if (server != null) {
                try {
                    server.close();
                    System.out.println("server close");
                } catch (IOException e) {
                    e.printStackTrace();
                }

            }
        }
    }

}

package netty.bio;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.Socket;

/**
 * @author wang.xw
 * @date 2018/3/13 13:54.
 */
public class TimerServerHadler implements Runnable {
    private Socket socket;

    public TimerServerHadler(Socket socket) {
        this.socket = socket;
    }

    public void run() {
        BufferedReader in = null;
        PrintWriter out = null;
        String currentTime = null;
        try {
            in = new BufferedReader(new InputStreamReader(this.socket.getInputStream()));
            out = new PrintWriter(this.socket.getOutputStream(), true);
            String body = null;
            while (true) {

                body = in.readLine();
                System.out.println("The TimeServer receive order : " + body);

                currentTime = "query time".equalsIgnoreCase(body) ? System.currentTimeMillis() + "" : "bad order";
                // 返回客戶端
                out.println(currentTime);

            }

        } catch (IOException e) {
            if (in != null) {
                try {
                    in.close();
                } catch (IOException e1) {
                    e.printStackTrace();
                }
            }
            if (out != null) {
                out.close();
            }
            if (socket != null) {
                try {
                    socket.close();
                } catch (IOException e1) {
                    e.printStackTrace();
                }
            }
        }
    }

}

client:

package netty.bio;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.Socket;
import java.util.Scanner;

/**
 * @author wang.xw
 * @date 2018/3/13 14:28.
 */
public class TimeClient {
    private static final String DEFAULT_SERVER_IP = "localhost";
    private static final int DEFAULT_SERVER_PORT = 8081;

    public static void main(String[] args) {
        BufferedReader in = null;
        PrintWriter out = null;
        Socket socket = null;
        try {
            socket = new Socket(DEFAULT_SERVER_IP, DEFAULT_SERVER_PORT);
            in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
            out = new PrintWriter(socket.getOutputStream(), true);
            // 讀取控制檯消息
            Scanner scanner = new Scanner(System.in);
            while (scanner.hasNext()){
                out.println(scanner.nextLine());
                System.out.println("send to server success !");
                String res = in.readLine();
                System.out.println("server return : " + res);
            }



        } catch (IOException e) {
            //不處理
        } finally {
            if (in != null) {
                try {
                    in.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (out != null) {
                out.close();
            }
            if (socket != null) {
                try {
                    socket.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

僞異步I/O模型：

僞異步I/O採用線程池管理管理這些線程，因爲線程池和消息隊列都是有界的，因此避免了由於客戶端併發數量大，導致內存溢出或者宕機。而對Socket的輸入流進行讀取時，會一直阻塞，直到發生：
- 有數據可讀
- 可用數據以及讀取完畢
- 發生空指針或I/O異常
所以在讀取數據較慢時（比如數據量大、網絡傳輸慢等），大量併發的情況下，其他接入的消息，只能一直排入到消息隊列等待。嚴重情況下會導致所有客戶端都連接超時。
Server:

package netty.pesudo;


import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;

/**
 * @author wang.xw
 * @date 2018/3/13 13:44.
 */
public class TimeServer {
    private static final int DEFAULT_SERVER_PORT = 8081;

    public static void main(String[] args) {
        ServerSocket server = null;
        try {
            server = new ServerSocket(DEFAULT_SERVER_PORT);
            System.out.println("The time server is start in port ：" + DEFAULT_SERVER_PORT);
            System.out.println("corePoolSize ：" +  Runtime.getRuntime().availableProcessors());
            while (true) {
                System.out.println("...");
                Socket socket = server.accept();
                TimerServerHadlerExecutePool executePool = new TimerServerHadlerExecutePool(50, 1000);
                executePool.execute(new TimerServerHadler(socket));
            }

        } catch (IOException e) {
        } finally {
            if (server != null) {
                try {
                    server.close();
                    System.out.println("server close");
                } catch (IOException e) {
                    e.printStackTrace();
                }

            }
        }
    }

}

自定義線程池：

package netty.pesudo;

import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/**
 * @author wang.xw
 * @date 2018/3/14 10:22.
 */
public class TimerServerHadlerExecutePool {
    private ExecutorService executor;

    public TimerServerHadlerExecutePool(int maxPoolSize, int queueSize) {

        executor = new ThreadPoolExecutor(
                Runtime.getRuntime().availableProcessors(),
                maxPoolSize,
                60L,
                TimeUnit.SECONDS,
                new ArrayBlockingQueue<>(queueSize)
        );
    }

    public void execute(Runnable task) {
        executor.execute(task);
    }
}

NIO:同步非阻塞I/O模型

NIO是基於緩衝區編程的，先要了解幾個重要的概念
1. 緩衝區Buffer
Buffer是一個對象，包含一些要寫入或者讀出的數據,在NIO庫中，所有數據都是用緩衝區處理的。在讀取數據時，它是直接讀到緩衝區中的；在寫入數據時，也是寫入到緩衝區中。任何時候訪問NIO中的數據，都是通過緩衝區進行操作。提供了對數據結構化訪問以及維護讀寫位置(limit)等信息。具體的緩存區有這些：ByteBuffe、CharBuffer、 ShortBuffer、IntBuffer、LongBuffer、FloatBuffer、DoubleBuffer。他們實現了相同的接口：Buffer。
2. 通道Channel
Channel是一個通道，它像自來水管一樣，網絡數據通過Channel讀取和寫入，通道和流的不同之處在於通道是雙向的而流是單向的（一個流必須是InputStream或者OutputStream的子類），而通道可以同時用於讀和寫。因爲Channel是全雙工的所以它比流能更好的映射底層的操作系統的api.
3. 多路複用器Selector
Selector提供選擇已經就緒的任務的能力：Selector會不斷輪詢註冊在其上的Channel，如果某個Channel上面發生讀或者寫事件，這個Channel就處於就緒狀態，會被Selector輪詢出來，然後通過SelectionKey可以獲取就緒Channel的集合，進行後續的I/O操作。一個Selector可以同時輪詢多個Channel，因爲JDK使用了epoll()代替傳統的select實現，所以沒有最大連接句柄1024/2048的限制。所以，只需要一個線程負責Selector的輪詢，就可以接入成千上萬的客戶端。
Server

public class TimeServer {
    private static final int DEFAULT_SERVER_PORT = 8081;
    public static void main(String[] args) {
        TimeServerHandler timeServer = new TimeServerHandler(DEFAULT_SERVER_PORT);
        new Thread(timeServer).start();
        System.out.println("main thread id: " + Thread.currentThread().getId());
    }
}


--
package netty.nio;

import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Set;

/**
 * @author wang.xw
 * @date 2018/3/15 11:21.
 */
public class TimeServerHandler implements Runnable {
    private Selector selector;
    private ServerSocketChannel channel;
    private volatile boolean start = false;

    /**
     * 初始化多路複用器，綁定監聽端口號
     *
     * @param port
     */
    public TimeServerHandler(int port) {
        try {
            selector = Selector.open();
            // 1.打開ServerSocketChannel，監聽客戶端連接，是所有客戶端連接的父管道
            channel = ServerSocketChannel.open();
            // 2.監聽端口號，設置爲非阻塞模式
            channel.socket().bind(new InetSocketAddress(port), 1024);
            channel.configureBlocking(false);
            // 3.監聽客戶端連接請求
            channel.register(selector, SelectionKey.OP_ACCEPT);
            System.out.println("The time server is start in port ：" + port);
            System.out.println("The Selector Thread id ：" + Thread.currentThread().getId());
        } catch (IOException e) {
            e.printStackTrace();
            // 資源初始化失敗系統退出
            System.exit(1);
        }
    }

    @Override
    public void run() {
        while (!start) {
            if (!selector.isOpen()) {
                System.out.println("selector is closed");
                break;
            }

            try {
                // 每隔一秒喚醒一次
                selector.select();
                Set<SelectionKey> selectionKeys = selector.selectedKeys();
                Iterator<SelectionKey> it = selectionKeys.iterator();
                SelectionKey key = null;
                while (it.hasNext()) {
                    key = it.next();
                    it.remove();
                    try {
                        handleInput(key);
                    } catch (Exception e) {
                        if (key != null) {
                            key.cancel();
                            if (key.channel() != null) {
                                key.channel().close();
                            }
                        }
                    }

                }

            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        //selector關閉後會自動釋放裏面管理的資源
        if (selector != null)
            try {
                selector.close();
            } catch (Exception e) {
                e.printStackTrace();
            }
    }

    private void handleInput(SelectionKey key) throws IOException {
        if (key.isValid()) {
            if (key.isAcceptable()) {
                ServerSocketChannel ssc = (ServerSocketChannel) key.channel();
                //通過ServerSocketChannel的accept創建SocketChannel實例
                //完成該操作意味着完成TCP三次握手，TCP物理鏈路正式建立
                SocketChannel sc = ssc.accept();
                //設置爲非阻塞的
                sc.configureBlocking(false);
                //註冊爲讀操作
                sc.register(selector, SelectionKey.OP_READ);
            }
            // 讀操作
            if (key.isReadable()) {
                SocketChannel sc = (SocketChannel) key.channel();
                //創建ByteBuffer，並開闢一個1M的緩衝區
                ByteBuffer buffer = ByteBuffer.allocate(1024);
                //讀取請求碼流，返回讀取到的字節數
                int readBytes = sc.read(buffer);
                if (readBytes > 0) {
                    //將緩衝區當前的limit設置爲position=0，用於後續對緩衝區的讀取操作
                    buffer.flip();
                    //根據緩衝區可讀字節數創建字節數組
                    byte[] bytes = new byte[buffer.remaining()];
                    //將緩衝區可讀字節數組複製到新建的數組中
                    buffer.get(bytes);
                    String body = new String(bytes, "UTF-8");
                    System.out.println("The TimeServer receive order : " + body);
                    String currentTime = "query time".equalsIgnoreCase(body) ? System.currentTimeMillis() + "" : "bad order";
                    //發送應答消息
                    doWrite(sc, currentTime);
                } else if (readBytes < 0) {
                    key.cancel();
                    sc.close();
                }
            }
        }
    }
    // 寫回客戶端
    private void doWrite(SocketChannel channel, String response) throws IOException {
        if (response != null && response.trim().length() > 0) {
            byte[] bytes = response.getBytes();
            ByteBuffer writeBuffer = ByteBuffer.allocate(bytes.length);
            writeBuffer.put(bytes);
            writeBuffer.flip();
            channel.write(writeBuffer);
        }
    }
}

Client

package netty.nio;

/**
 * @author wang.xw
 * @date 2018/3/15 14:32.
 */
public class TimeClient {
    private static final String DEFAULT_SERVER_IP = "localhost";
    private static final int DEFAULT_SERVER_PORT = 8081;

    public static void main(String[] args) {
        new Thread(new TimeClientHandler(DEFAULT_SERVER_IP, DEFAULT_SERVER_PORT)).start();
        System.out.println("main thread id: " + Thread.currentThread().getId());
    }
}
--
package netty.nio;

import java.io.IOException;
import java.io.PrintWriter;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Scanner;
import java.util.Set;

/**
 * @author wang.xw
 * @date 2018/3/15 14:34.
 */
public class TimeClientHandler implements Runnable {
    private transient String host;
    private transient int port;
    private Selector selector;
    private SocketChannel socketChannel;
    private volatile boolean start = true;

    public TimeClientHandler(String host, int port) {
        this.host = host;
        this.port = port;
        try {
            // 創建選擇器
            selector = Selector.open();
            // 打開SocketChannel,綁定客戶端本地地址
            socketChannel = SocketChannel.open();
            // 設置爲非阻塞模式
            socketChannel.configureBlocking(false);
            System.out.println("The client Thread id ：" + Thread.currentThread().getId());
        } catch (IOException e) {
            e.printStackTrace();
            System.exit(1);
        }

    }

    @Override
    public void run() {
        try {
            doConnect();
        } catch (IOException e) {
            e.printStackTrace();
            System.exit(1);
        }
        while (start) {
            if (!selector.isOpen()) {
                System.out.println("selector is closed");
                break;
            }
            System.out.println("...");
            try {
                // 每隔一秒喚醒一次
                selector.select();
                Set<SelectionKey> selectionKeys = selector.selectedKeys();
                Iterator<SelectionKey> it = selectionKeys.iterator();
                SelectionKey key = null;
                while (it.hasNext()) {
                    key = it.next();
                    it.remove();
                    try {
                        handleInput(key);
                    } catch (Exception e) {
                        if (key != null) {
                            key.cancel();
                            if (key.channel() != null) {
                                key.channel().close();
                            }
                        }
                    }

                }

            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        if (selector != null) {
            try {
                selector.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    private void handleInput(SelectionKey key) throws IOException {

        if (key.isValid()) {
            SocketChannel sc = (SocketChannel) key.channel();
            // 判斷是否處於連接狀態,服務端是否返回ack消息
            if (key.isConnectable()) {
                // 判斷客戶端是否連接成功
                if (sc.finishConnect()) {
                    //註冊爲讀操作
                    sc.register(selector, SelectionKey.OP_READ);
                    doWrite(sc);
                } else {
                    System.exit(1);
                }
            }
            if (key.isReadable()) {
                //創建ByteBuffer，並開闢一個1M的緩衝區
                ByteBuffer buffer = ByteBuffer.allocate(1024);
                //讀取請求碼流，返回讀取到的字節數
                int readBytes = sc.read(buffer);
                if (readBytes > 0) {
                    //將緩衝區當前的limit設置爲position=0，用於後續對緩衝區的讀取操作
                    buffer.flip();
                    //根據緩衝區可讀字節數創建字節數組
                    byte[] bytes = new byte[buffer.remaining()];
                    //將緩衝區可讀字節數組複製到新建的數組中
                    buffer.get(bytes);
                    String res = new String(bytes, "UTF-8");
                    System.out.println("server return : " + res);
                    this.start = false;
                } else if (readBytes < 0) {
                    key.cancel();
                    sc.close();
                }
            }
        }

    }

    private void doWrite(SocketChannel sc) throws IOException {
        Scanner scanner = new Scanner(System.in);
        byte[] req = scanner.next().getBytes();
        ByteBuffer buffer = ByteBuffer.allocate(req.length);
        buffer.put(req);
        buffer.flip();
        sc.write(buffer);
        System.out.println("send to server success !");


    }

    private void doConnect() throws IOException {
        // 如果直接連接成功，則直接註冊到多路複用器上，進行讀操作
        if (socketChannel.connect(new InetSocketAddress(host, port))) {
            //註冊爲讀操作
            socketChannel.register(selector, SelectionKey.OP_READ);
            doWrite(socketChannel);
        } else {
            // 等待服務器返回TCP syn-ack
            socketChannel.register(selector, SelectionKey.OP_CONNECT);
        }
    }
}

JDK原生的NIO類庫編程十分複雜，而且還有一個嚴重epollbug.所以不推薦被直接拿來使用。在NIO2.0的時候引入了AIO的概念，AIO真的實現了異步非阻塞I/O,他不需要通過多路複用器對註冊的管道進行輪詢操作就能夠實現異步讀寫。只有瞭解了基本的I/O模型我們才能更深入的研究netty。

Netty(一)：IO通信模型

BIO:同步阻塞模型

僞異步I/O模型：

NIO:同步非阻塞I/O模型

數據庫進階（文件的組織方式和索引）

Spring隨筆（IOC/DI 及注入方式）

MyBaits緩存原理（源碼級跟蹤）

spring隨筆（AOP）

spring隨筆(Junit 和web)

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結