Netty服務端的啓動源碼分析

ServerBootstrap的構造:

public class ServerBootstrap extends AbstractBootstrap<ServerBootstrap, ServerChannel> {
    private static final InternalLogger logger = InternalLoggerFactory.getInstance(ServerBootstrap.class);
    private final Map<ChannelOption<?>, Object> childOptions = new LinkedHashMap();
    private final Map<AttributeKey<?>, Object> childAttrs = new LinkedHashMap();
    private final ServerBootstrapConfig config = new ServerBootstrapConfig(this);
    private volatile EventLoopGroup childGroup;
    private volatile ChannelHandler childHandler;

    public ServerBootstrap() {
    }
	......
}

隱式地執行了父類的無參構造:

public abstract class AbstractBootstrap<B extends AbstractBootstrap<B, C>, C extends Channel> implements Cloneable {
    volatile EventLoopGroup group;
    private volatile ChannelFactory<? extends C> channelFactory;
    private volatile SocketAddress localAddress;
    private final Map<ChannelOption<?>, Object> options = new LinkedHashMap();
    private final Map<AttributeKey<?>, Object> attrs = new LinkedHashMap();
    private volatile ChannelHandler handler;

    AbstractBootstrap() {
    }
    ......
}

只是初始化了幾個容器成員

在ServerBootstrap創建後,需要調用group方法,綁定EventLoopGroup,有關EventLoopGroup的創建在我之前博客中寫過:Netty中NioEventLoopGroup的創建源碼分析

ServerBootstrap的group方法:

public ServerBootstrap group(EventLoopGroup group) {
    return this.group(group, group);
}

public ServerBootstrap group(EventLoopGroup parentGroup, EventLoopGroup childGroup) {
    super.group(parentGroup);
    if (childGroup == null) {
        throw new NullPointerException("childGroup");
    } else if (this.childGroup != null) {
        throw new IllegalStateException("childGroup set already");
    } else {
        this.childGroup = childGroup;
        return this;
    }
}

首先調用父類的group方法綁定parentGroup:

public B group(EventLoopGroup group) {
    if (group == null) {
        throw new NullPointerException("group");
    } else if (this.group != null) {
        throw new IllegalStateException("group set already");
    } else {
        this.group = group;
        return this.self();
    }
}

private B self() {
    return this;
}

將傳入的parentGroup綁定給AbstractBootstrap的group成員,將childGroup綁定給ServerBootstrap的childGroup成員。
group的綁定僅僅是交給了成員保存。

再來看看ServerBootstrap的channel方法,,是在AbstractBootstrap中實現的:

public B channel(Class<? extends C> channelClass) {
    if (channelClass == null) {
        throw new NullPointerException("channelClass");
    } else {
        return this.channelFactory((io.netty.channel.ChannelFactory)(new ReflectiveChannelFactory(channelClass)));
    }
}

使用channelClass構建了一個ReflectiveChannelFactory對象:

public class ReflectiveChannelFactory<T extends Channel> implements ChannelFactory<T> {
    private final Class<? extends T> clazz;

    public ReflectiveChannelFactory(Class<? extends T> clazz) {
        if (clazz == null) {
            throw new NullPointerException("clazz");
        } else {
            this.clazz = clazz;
        }
    }

    public T newChannel() {
        try {
            return (Channel)this.clazz.getConstructor().newInstance();
        } catch (Throwable var2) {
            throw new ChannelException("Unable to create Channel from class " + this.clazz, var2);
        }
    }

    public String toString() {
        return StringUtil.simpleClassName(this.clazz) + ".class";
    }
}

可以看到ReflectiveChannelFactory的作用就是通過反射機制,產生clazz的實例(這裏以NioServerSocketChannel爲例)。

在創建完ReflectiveChannelFactory對象後, 調用channelFactory方法:

public B channelFactory(io.netty.channel.ChannelFactory<? extends C> channelFactory) {
    return this.channelFactory((ChannelFactory)channelFactory);
}

public B channelFactory(ChannelFactory<? extends C> channelFactory) {
    if (channelFactory == null) {
        throw new NullPointerException("channelFactory");
    } else if (this.channelFactory != null) {
        throw new IllegalStateException("channelFactory set already");
    } else {
        this.channelFactory = channelFactory;
        return this.self();
    }
}

將剛纔創建的ReflectiveChannelFactory對象交給channelFactory成員,用於後續服務端NioServerSocketChannel的創建。

再來看ServerBootstrap的childHandler方法:

public ServerBootstrap childHandler(ChannelHandler childHandler) {
    if (childHandler == null) {
        throw new NullPointerException("childHandler");
    } else {
        this.childHandler = childHandler;
        return this;
    }
}

還是交給了childHandler成員保存,可以看到上述這一系列的操作,都是爲了填充ServerBootstrap,而ServerBootstrap真正的啓動是在bind時:
ServerBootstrap的bind方法,在AbstractBootstrap中實現:

public ChannelFuture bind(int inetPort) {
	return this.bind(new InetSocketAddress(inetPort));
}

public ChannelFuture bind(String inetHost, int inetPort) {
return this.bind(SocketUtils.socketAddress(inetHost, inetPort));
}

public ChannelFuture bind(InetAddress inetHost, int inetPort) {
	return this.bind(new InetSocketAddress(inetHost, inetPort));
}

public ChannelFuture bind(SocketAddress localAddress) {
	this.validate();
	if (localAddress == null) {
		throw new NullPointerException("localAddress");
	} else {
		return this.doBind(localAddress);
	}
}

可以看到首先調用了ServerBootstrap的validate方法,:

public ServerBootstrap validate() {
	super.validate();
	if (this.childHandler == null) {
		throw new IllegalStateException("childHandler not set");
	} else {
		if (this.childGroup == null) {
			logger.warn("childGroup is not set. Using parentGroup instead.");
			this.childGroup = this.config.group();
		}
	
		return this;
	}
}

先調用了AbstractBootstrap的validate方法:

public B validate() {
    if (this.group == null) {
        throw new IllegalStateException("group not set");
    } else if (this.channelFactory == null) {
        throw new IllegalStateException("channel or channelFactory not set");
    } else {
        return this.self();
    }
}

這個方法就是用來檢查是否綁定了group和channel以及childHandler,所以在執行bind方法前,無論如何都要執行group、channel和childHandler方法。

實際的bind交給了doBind來完成:

private ChannelFuture doBind(final SocketAddress localAddress) {
    final ChannelFuture regFuture = this.initAndRegister();
    final Channel channel = regFuture.channel();
    if (regFuture.cause() != null) {
        return regFuture;
    } else if (regFuture.isDone()) {
        ChannelPromise promise = channel.newPromise();
        doBind0(regFuture, channel, localAddress, promise);
        return promise;
    } else {
        final AbstractBootstrap.PendingRegistrationPromise promise = new AbstractBootstrap.PendingRegistrationPromise(channel);
        regFuture.addListener(new ChannelFutureListener() {
            public void operationComplete(ChannelFuture future) throws Exception {
                Throwable cause = future.cause();
                if (cause != null) {
                    promise.setFailure(cause);
                } else {
                    promise.registered();
                    AbstractBootstrap.doBind0(regFuture, channel, localAddress, promise);
                }
            }
        });
        return promise;
    }
}

首先調用initAndRegister,完成ServerSocketChannel的創建以及註冊:

final ChannelFuture initAndRegister() {
    Channel channel = null;

    try {
        channel = this.channelFactory.newChannel();
        this.init(channel);
    } catch (Throwable var3) {
        if (channel != null) {
            channel.unsafe().closeForcibly();
            return (new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE)).setFailure(var3);
        }

        return (new DefaultChannelPromise(new FailedChannel(), GlobalEventExecutor.INSTANCE)).setFailure(var3);
    }

    ChannelFuture regFuture = this.config().group().register(channel);
    if (regFuture.cause() != null) {
        if (channel.isRegistered()) {
            channel.close();
        } else {
            channel.unsafe().closeForcibly();
        }
    }

    return regFuture;
}

首先調用channelFactory的newChannel通過反射機制構建Channel實例,也就是NioServerSocketChannel,

NioServerSocketChannel的無參構造:

public class NioServerSocketChannel extends AbstractNioMessageChannel implements ServerSocketChannel {
	private static final SelectorProvider DEFAULT_SELECTOR_PROVIDER = SelectorProvider.provider();
	
	public NioServerSocketChannel() {
		this(newSocket(DEFAULT_SELECTOR_PROVIDER));
	}
	......
}

SelectorProvider 是JDK的,關於SelectorProvider在我之前的博客中有介紹:【Java】NIO中Selector的創建源碼分析
在Windows系統下默認產生WindowsSelectorProvider,即DEFAULT_SELECTOR_PROVIDER,再來看看newSocket方法:

private static java.nio.channels.ServerSocketChannel newSocket(SelectorProvider provider) {
	try {
		return provider.openServerSocketChannel();
	} catch (IOException var2) {
		throw new ChannelException("Failed to open a server socket.", var2);
	}
}

使用WindowsSelectorProvider創建了一個ServerSocketChannelImpl,其實看到這裏就明白了,NioServerSocketChannel是爲了封裝JDK的ServerSocketChannel

接着調用另一個重載的構造:

public NioServerSocketChannel(java.nio.channels.ServerSocketChannel channel) {
    super((Channel)null, channel, 16);
    this.config = new NioServerSocketChannel.NioServerSocketChannelConfig(this, this.javaChannel().socket());
}

首先調用父類的三參構造,其中16對應的是JDK中SelectionKey的ACCEPT狀態:

public static final int OP_ACCEPT = 1 << 4;

其父類的構造處於一條繼承鏈上:

AbstractNioMessageChannel:

protected AbstractNioMessageChannel(Channel parent, SelectableChannel ch, int readInterestOp) {
    super(parent, ch, readInterestOp);
}

AbstractNioChannel:

protected AbstractNioChannel(Channel parent, SelectableChannel ch, int readInterestOp) {
    super(parent);
    this.ch = ch;
    this.readInterestOp = readInterestOp;

    try {
        ch.configureBlocking(false);
    } catch (IOException var7) {
        try {
            ch.close();
        } catch (IOException var6) {
            if (logger.isWarnEnabled()) {
                logger.warn("Failed to close a partially initialized socket.", var6);
            }
        }

        throw new ChannelException("Failed to enter non-blocking mode.", var7);
    }
}

AbstractChannel:

private final ChannelId id;
private final Channel parent;
private final Unsafe unsafe;
private final DefaultChannelPipeline pipeline;

protected AbstractChannel(Channel parent) {
    this.parent = parent;
    this.id = this.newId();
    this.unsafe = this.newUnsafe();
    this.pipeline = this.newChannelPipeline();
}

在AbstractChannel中使用newUnsafe和newChannelPipeline分別創建了一個Unsafe和一個DefaultChannelPipeline對象,
在前面的博客介紹NioEventLoopGroup時候,在NioEventLoop的run方法中,每次輪詢完調用processSelectedKeys方法時,都是通過這個unsafe根據SelectedKey來完成數據的讀或寫,unsafe是處理基礎的數據讀寫
(unsafe在NioServerSocketChannel創建時,產生NioMessageUnsafe實例,在NioSocketChannel創建時產生NioSocketChannelUnsafe實例)
而pipeline的實現是一條雙向責任鏈,負責處理unsafe提供的數據,進而進行用戶的業務邏輯(Netty中的ChannelPipeline源碼分析

在AbstractNioChannel中調用configureBlocking方法給JDK的ServerSocketChannel設置爲非阻塞模式,且讓readInterestOp成員賦值爲16用於未來註冊ACCEPT事件。

在調用完繼承鏈後回到NioServerSocketChannel構造,調用了javaChannel方法:

protected java.nio.channels.ServerSocketChannel javaChannel() {
    return (java.nio.channels.ServerSocketChannel)super.javaChannel();
}

其實這個javaChannel就是剛纔出傳入到AbstractNioChannel中的ch成員:

protected SelectableChannel javaChannel() {
    return this.ch;
}

也就是剛纔創建的JDK的ServerSocketChannelImpl,用其socket方法,得到一個ServerSocket對象,然後產生了一個NioServerSocketChannelConfig對象,用於封裝相關信息。

NioServerSocketChannel構建完畢,回到initAndRegister方法,使用剛創建的NioServerSocketChannel調用init方法,這個方法是在ServerBootstrap中實現的:

void init(Channel channel) throws Exception {
    Map<ChannelOption<?>, Object> options = this.options0();
    synchronized(options) {
        setChannelOptions(channel, options, logger);
    }

    Map<AttributeKey<?>, Object> attrs = this.attrs0();
    synchronized(attrs) {
        Iterator var5 = attrs.entrySet().iterator();

        while(true) {
            if (!var5.hasNext()) {
                break;
            }

            Entry<AttributeKey<?>, Object> e = (Entry)var5.next();
            AttributeKey<Object> key = (AttributeKey)e.getKey();
            channel.attr(key).set(e.getValue());
        }
    }

    ChannelPipeline p = channel.pipeline();
    final EventLoopGroup currentChildGroup = this.childGroup;
    final ChannelHandler currentChildHandler = this.childHandler;
    Map var9 = this.childOptions;
    final Entry[] currentChildOptions;
    synchronized(this.childOptions) {
        currentChildOptions = (Entry[])this.childOptions.entrySet().toArray(newOptionArray(0));
    }

    var9 = this.childAttrs;
    final Entry[] currentChildAttrs;
    synchronized(this.childAttrs) {
        currentChildAttrs = (Entry[])this.childAttrs.entrySet().toArray(newAttrArray(0));
    }

    p.addLast(new ChannelHandler[]{new ChannelInitializer<Channel>() {
        public void initChannel(final Channel ch) throws Exception {
            final ChannelPipeline pipeline = ch.pipeline();
            ChannelHandler handler = ServerBootstrap.this.config.handler();
            if (handler != null) {
                pipeline.addLast(new ChannelHandler[]{handler});
            }

            ch.eventLoop().execute(new Runnable() {
                public void run() {
                    pipeline.addLast(new ChannelHandler[]{new ServerBootstrap.ServerBootstrapAcceptor(ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs)});
                }
            });
        }
    }});
}

首先對attrs和options這兩個成員進行了填充屬性配置,這不是重點,然後獲取剛纔創建的NioServerSocketChannel的責任鏈pipeline,通過addLast將ChannelInitializer加入責任鏈,在ChannelInitializer中重寫了initChannel方法,首先根據handler是否是null(這個handler是ServerBootstrap調用handler方法添加的,和childHandler方法不一樣),若是handler不是null,將handler加入責任鏈,無論如何,都會異步將一個ServerBootstrapAcceptor對象加入責任鏈(後面會說爲什麼是異步)

這個ChannelInitializer的initChannel方法的執行需要等到後面註冊時纔會被調用,在後面pipeline處理channelRegistered請求時,此initChannel方法纔會被執行(Netty中的ChannelPipeline源碼分析

ChannelInitializer的channelRegistered方法:

public final void channelRegistered(ChannelHandlerContext ctx) throws Exception {
    if (initChannel(ctx)) {
        ctx.pipeline().fireChannelRegistered();
    } else {
        ctx.fireChannelRegistered();
    }
}

首先調用initChannel方法(和上面的initChannel不是一個):

private boolean initChannel(ChannelHandlerContext ctx) throws Exception {
    if (initMap.putIfAbsent(ctx, Boolean.TRUE) == null) { 
        try {
            initChannel((C) ctx.channel());
        } catch (Throwable cause) {
            exceptionCaught(ctx, cause);
        } finally {
            remove(ctx);
        }
        return true;
    }
    return false;
}

可以看到,這個ChannelInitializer只會在pipeline中初始化一次,僅用於Channel的註冊,在完成註冊後,會調用remove方法將其從pipeline中移除:
remove方法:

private void remove(ChannelHandlerContext ctx) {
    try {
        ChannelPipeline pipeline = ctx.pipeline();
        if (pipeline.context(this) != null) {
            pipeline.remove(this);
        }
    } finally {
        initMap.remove(ctx);
    }
}

在移除前,就會回調用剛纔覆蓋的initChannel方法,異步向pipeline添加了ServerBootstrapAcceptor,用於後續的NioServerSocketChannel偵聽到客戶端連接後,完成在服務端的NioSocketChannel的註冊。

回到initAndRegister,在對NioServerSocketChannel初始化完畢,接下來就是註冊邏輯

ChannelFuture regFuture = this.config().group().register(channel);

首先調用config().group(),這個就得到了一開始在ServerBootstrap的group方法傳入的parentGroup,調用parentGroup的register方法,parentGroup是NioEventLoopGroup,這個方法是在子類MultithreadEventLoopGroup中實現的:

public ChannelFuture register(Channel channel) {
	return this.next().register(channel);
}

首先調用next方法:

public EventLoop next() {
    return (EventLoop)super.next();
}

實際上調用父類MultithreadEventExecutorGroup的next方法:

public EventExecutor next() {
	return this.chooser.next();
}

關於chooser在我之前博客:Netty中NioEventLoopGroup的創建源碼分析介紹過,在NioEventLoopGroup創建時,默認會根據cpu個數創建二倍個NioEventLoop,而chooser就負責通過取模,每次選擇一個NioEventLoop使用
所以在MultithreadEventLoopGroup的register方法實際調用了NioEventLoop的register方法:

NioEventLoop的register方法在子類SingleThreadEventLoop中實現:

public ChannelFuture register(Channel channel) {
    return this.register((ChannelPromise)(new DefaultChannelPromise(channel, this)));
}

public ChannelFuture register(ChannelPromise promise) {
   ObjectUtil.checkNotNull(promise, "promise");
    promise.channel().unsafe().register(this, promise);
    return promise;
}

先把channel包裝成ChannelPromise,默認是DefaultChannelPromise ( Netty中的ChannelFuture和ChannelPromise),用於處理異步操作
調用重載方法,而在重載方法裏,可以看到,實際上的register操作交給了channel的unsafe來實現:
unsafe的register方法在AbstractUnsafe中實現:

public final void register(EventLoop eventLoop, final ChannelPromise promise) {
    if (eventLoop == null) {
        throw new NullPointerException("eventLoop");
    } else if (AbstractChannel.this.isRegistered()) {
        promise.setFailure(new IllegalStateException("registered to an event loop already"));
    } else if (!AbstractChannel.this.isCompatible(eventLoop)) {
        promise.setFailure(new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName()));
    } else {
        AbstractChannel.this.eventLoop = eventLoop;
        if (eventLoop.inEventLoop()) {
            this.register0(promise);
        } else {
            try {
                eventLoop.execute(new Runnable() {
                    public void run() {
                        AbstractUnsafe.this.register0(promise);
                    }
                });
            } catch (Throwable var4) {
                AbstractChannel.logger.warn("Force-closing a channel whose registration task was not accepted by an event loop: {}", AbstractChannel.this, var4);
                this.closeForcibly();
                AbstractChannel.this.closeFuture.setClosed();
                this.safeSetFailure(promise, var4);
            }
        }

    }
}

前面的判斷做了一些檢查就不細說了,直接看到else塊
首先給當前Channel綁定了eventLoop,即通過剛纔chooser選擇的eventLoop,該Channel也就是NioServerSocketChannel
由於Unsafe的操作是在輪詢線程中異步執行的,所裏,這裏需要判斷inEventLoop是否處於輪詢中
在之前介紹NioEventLoopGroup的時候說過,NioEventLoop在沒有調用doStartThread方法時並沒有啓動輪詢的,所以inEventLoop判斷不成立

那麼就調用eventLoop的execute方法,實際上的註冊方法可以看到調用了AbstractUnsafe的register0方法,而將這個方法封裝爲Runnable交給eventLoop作爲一個task去異步執行
先來看eventLoop的execute方法實現,是在NioEventLoop的子類SingleThreadEventExecutor中實現的:

public void execute(Runnable task) {
    if (task == null) {
        throw new NullPointerException("task");
    } else {
        boolean inEventLoop = this.inEventLoop();
        this.addTask(task);
        if (!inEventLoop) {
            this.startThread();
            if (this.isShutdown() && this.removeTask(task)) {
                reject();
            }
        }

        if (!this.addTaskWakesUp && this.wakesUpForTask(task)) {
            this.wakeup(inEventLoop);
        }

    }
}

這裏首先將task,即剛纔的註冊事件放入阻塞任務隊列中,然後調用startThread方法:

private void startThread() {
    if (this.state == 1 && STATE_UPDATER.compareAndSet(this, 1, 2)) {
        try {
            this.doStartThread();
        } catch (Throwable var2) {
            STATE_UPDATER.set(this, 1);
            PlatformDependent.throwException(var2);
        }
    }

}

NioEventLoop此時還沒有輪詢,所以狀態是1,對應ST_NOT_STARTED,此時利用CAS操作,將狀態修改爲2,即ST_STARTED ,標誌着NioEventLoop要啓動輪詢了,果然,接下來就調用了doStartThread開啓輪詢線程:

private void doStartThread() {
    assert this.thread == null;

    this.executor.execute(new Runnable() {
        public void run() {
            SingleThreadEventExecutor.this.thread = Thread.currentThread();
            if (SingleThreadEventExecutor.this.interrupted) {
                SingleThreadEventExecutor.this.thread.interrupt();
            }

            boolean success = false;
            SingleThreadEventExecutor.this.updateLastExecutionTime();
            boolean var112 = false;

            int oldState;
            label1907: {
                try {
                    var112 = true;
                    SingleThreadEventExecutor.this.run();
                    success = true;
                    var112 = false;
                    break label1907;
                } catch (Throwable var119) {
                    SingleThreadEventExecutor.logger.warn("Unexpected exception from an event executor: ", var119);
                    var112 = false;
                } finally {
                    if (var112) {
                        int oldStatex;
                        do {
                            oldStatex = SingleThreadEventExecutor.this.state;
                        } while(oldStatex < 3 && !SingleThreadEventExecutor.STATE_UPDATER.compareAndSet(SingleThreadEventExecutor.this, oldStatex, 3));

                        if (success && SingleThreadEventExecutor.this.gracefulShutdownStartTime == 0L && SingleThreadEventExecutor.logger.isErrorEnabled()) {
                            SingleThreadEventExecutor.logger.error("Buggy " + EventExecutor.class.getSimpleName() + " implementation; " + SingleThreadEventExecutor.class.getSimpleName() + ".confirmShutdown() must be called before run() implementation terminates.");
                        }

                        try {
                            while(!SingleThreadEventExecutor.this.confirmShutdown()) {
                                ;
                            }
                        } finally {
                            try {
                                SingleThreadEventExecutor.this.cleanup();
                            } finally {
                                SingleThreadEventExecutor.STATE_UPDATER.set(SingleThreadEventExecutor.this, 5);
                                SingleThreadEventExecutor.this.threadLock.release();
                                if (!SingleThreadEventExecutor.this.taskQueue.isEmpty() && SingleThreadEventExecutor.logger.isWarnEnabled()) {
                                    SingleThreadEventExecutor.logger.warn("An event executor terminated with non-empty task queue (" + SingleThreadEventExecutor.this.taskQueue.size() + ')');
                                }

                                SingleThreadEventExecutor.this.terminationFuture.setSuccess((Object)null);
                            }
                        }

                    }
                }

                do {
                    oldState = SingleThreadEventExecutor.this.state;
                } while(oldState < 3 && !SingleThreadEventExecutor.STATE_UPDATER.compareAndSet(SingleThreadEventExecutor.this, oldState, 3));

                if (success && SingleThreadEventExecutor.this.gracefulShutdownStartTime == 0L && SingleThreadEventExecutor.logger.isErrorEnabled()) {
                    SingleThreadEventExecutor.logger.error("Buggy " + EventExecutor.class.getSimpleName() + " implementation; " + SingleThreadEventExecutor.class.getSimpleName() + ".confirmShutdown() must be called before run() implementation terminates.");
                }

                try {
                    while(!SingleThreadEventExecutor.this.confirmShutdown()) {
                        ;
                    }

                    return;
                } finally {
                    try {
                        SingleThreadEventExecutor.this.cleanup();
                    } finally {
                        SingleThreadEventExecutor.STATE_UPDATER.set(SingleThreadEventExecutor.this, 5);
                        SingleThreadEventExecutor.this.threadLock.release();
                        if (!SingleThreadEventExecutor.this.taskQueue.isEmpty() && SingleThreadEventExecutor.logger.isWarnEnabled()) {
                            SingleThreadEventExecutor.logger.warn("An event executor terminated with non-empty task queue (" + SingleThreadEventExecutor.this.taskQueue.size() + ')');
                        }

                        SingleThreadEventExecutor.this.terminationFuture.setSuccess((Object)null);
                    }
                }
            }

            do {
                oldState = SingleThreadEventExecutor.this.state;
            } while(oldState < 3 && !SingleThreadEventExecutor.STATE_UPDATER.compareAndSet(SingleThreadEventExecutor.this, oldState, 3));

            if (success && SingleThreadEventExecutor.this.gracefulShutdownStartTime == 0L && SingleThreadEventExecutor.logger.isErrorEnabled()) {
                SingleThreadEventExecutor.logger.error("Buggy " + EventExecutor.class.getSimpleName() + " implementation; " + SingleThreadEventExecutor.class.getSimpleName() + ".confirmShutdown() must be called before run() implementation terminates.");
            }

            try {
                while(!SingleThreadEventExecutor.this.confirmShutdown()) {
                    ;
                }
            } finally {
                try {
                    SingleThreadEventExecutor.this.cleanup();
                } finally {
                    SingleThreadEventExecutor.STATE_UPDATER.set(SingleThreadEventExecutor.this, 5);
                    SingleThreadEventExecutor.this.threadLock.release();
                    if (!SingleThreadEventExecutor.this.taskQueue.isEmpty() && SingleThreadEventExecutor.logger.isWarnEnabled()) {
                        SingleThreadEventExecutor.logger.warn("An event executor terminated with non-empty task queue (" + SingleThreadEventExecutor.this.taskQueue.size() + ')');
                    }

                    SingleThreadEventExecutor.this.terminationFuture.setSuccess((Object)null);
                }
            }

        }
    });
}

關於doStartThread方法,我在Netty中NioEventLoopGroup的創建源碼分析 中已經說的很細了,這裏就不再一步一步分析了

因爲此時還沒真正意義上的啓動輪詢,所以thread等於null成立的,然後調用executor的execute方法,這裏的executor是一個線程池,在之前說過的,所以裏面的run方法是處於一個線程裏面的,然後給thread成員賦值爲當前線程,表明正式進入了輪詢。
而SingleThreadEventExecutor.this.run()纔是真正的輪詢邏輯,這在之前也說過,這個run的實現是在父類NioEventLoop中:

protected void run() {
    while(true) {
        while(true) {
            try {
                switch(this.selectStrategy.calculateStrategy(this.selectNowSupplier, this.hasTasks())) {
                case -2:
                    continue;
                case -1:
                    this.select(this.wakenUp.getAndSet(false));
                    if (this.wakenUp.get()) {
                        this.selector.wakeup();
                    }
                default:
                    this.cancelledKeys = 0;
                    this.needsToSelectAgain = false;
                    int ioRatio = this.ioRatio;
                    if (ioRatio == 100) {
                        try {
                            this.processSelectedKeys();
                        } finally {
                            this.runAllTasks();
                        }
                    } else {
                        long ioStartTime = System.nanoTime();
                        boolean var13 = false;

                        try {
                            var13 = true;
                            this.processSelectedKeys();
                            var13 = false;
                        } finally {
                            if (var13) {
                                long ioTime = System.nanoTime() - ioStartTime;
                                this.runAllTasks(ioTime * (long)(100 - ioRatio) / (long)ioRatio);
                            }
                        }

                        long ioTime = System.nanoTime() - ioStartTime;
                        this.runAllTasks(ioTime * (long)(100 - ioRatio) / (long)ioRatio);
                    }
                }
            } catch (Throwable var21) {
                handleLoopException(var21);
            }

            try {
                if (this.isShuttingDown()) {
                    this.closeAll();
                    if (this.confirmShutdown()) {
                        return;
                    }
                }
            } catch (Throwable var18) {
                handleLoopException(var18);
            }
        }
    }
}

首先由於task已經有一個了,就是剛纔的註冊事件,所以選擇策略calculateStrategy最終調用selectNow(也是之前說過的):

private final IntSupplier selectNowSupplier = new IntSupplier() {
    public int get() throws Exception {
        return NioEventLoop.this.selectNow();
    }
};

int selectNow() throws IOException {
    int var1;
    try {
        var1 = this.selector.selectNow();
    } finally {
        if (this.wakenUp.get()) {
            this.selector.wakeup();
        }

    }

    return var1;
}

使用JDK原生Selector進行selectNow,由於此時沒有任何Channel的註冊,所以selectNow會立刻返回0,此時就進入default邏輯,由於沒有任何註冊,processSelectedKeys方法也做不了什麼,所以在這一次的輪詢實質上只進行了runAllTasks方法,此方法會執行阻塞隊列中的task的run方法(還是在之前博客中介紹過),由於輪詢是在線程池中的一個線程中運行的,所以task的執行是一個異步操作。(在執行完task,將task移除阻塞對立,線程繼續輪詢)

這時就可以回到AbstractChannel的register方法中了,由上面可以知道task實際上異步執行了:

AbstractUnsafe.this.register0(promise);

register0方法:

private void register0(ChannelPromise promise) {
    try {
        if (!promise.setUncancellable() || !this.ensureOpen(promise)) {
            return;
        }

        boolean firstRegistration = this.neverRegistered;
        AbstractChannel.this.doRegister();
        this.neverRegistered = false;
        AbstractChannel.this.registered = true;
        AbstractChannel.this.pipeline.invokeHandlerAddedIfNeeded();
        this.safeSetSuccess(promise);
        AbstractChannel.this.pipeline.fireChannelRegistered();
        if (AbstractChannel.this.isActive()) {
            if (firstRegistration) {
                AbstractChannel.this.pipeline.fireChannelActive();
            } else if (AbstractChannel.this.config().isAutoRead()) {
                this.beginRead();
            }
        }
    } catch (Throwable var3) {
        this.closeForcibly();
        AbstractChannel.this.closeFuture.setClosed();
        this.safeSetFailure(promise, var3);
    }

}

可以看到實際上的註冊邏輯又交給了AbstractChannel的doRegister,而這個方法在AbstractNioChannel中實現:

protected void doRegister() throws Exception {
    boolean selected = false;

    while(true) {
        try {
            this.selectionKey = this.javaChannel().register(this.eventLoop().unwrappedSelector(), 0, this);
            return;
        } catch (CancelledKeyException var3) {
            if (selected) {
                throw var3;
            }

            this.eventLoop().selectNow();
            selected = true;
        }
    }
}

javaChannel就是之前產生的JDK的ServerSocketChannel,unwrappedSelector在之前說過,就是未經修改的JDK原生Selector,這個Selector和eventLoop是一對一綁定的,可以看到調用JDK原生的註冊方法,完成了對ServerSocketChannel的註冊,但是註冊的是一個0狀態(缺省值),而傳入的this,即AbstractNioChannel對象作爲了一個附件,用於以後processSelectedKeys方法從SelectionKey中得到對應的Netty的Channel(還是之前博客說過)
關於缺省值,是由於AbstractNioChannel不僅用於NioServerSocketChannel的註冊,還用於NioSocketChannel的註冊,只有都使用缺省值註冊纔不會產生異常(【Java】NIO中Channel的註冊源碼分析),並且,在以後processSelectedKeys方法會對0狀態判斷,再使用unsafe進行相應的邏輯處理。

在完成JDK的註冊後,調用pipeline的invokeHandlerAddedIfNeeded方法,處理ChannelHandler的handlerAdded的回調(Netty中的ChannelPipeline源碼分析),即調用用戶添加的ChannelHandler的handlerAdded方法。
調用safeSetSuccess,標誌異步操作完成:

protected final void safeSetSuccess(ChannelPromise promise) {
    if (!(promise instanceof VoidChannelPromise) && !promise.trySuccess()) {
        logger.warn("Failed to mark a promise as success because it is done already: {}", promise);
    }
}

關於異步操作我在之前的博客中說的很清楚了:Netty中的ChannelFuture和ChannelPromise

接着調用pipeline的fireChannelRegistered方法,也就是在責任鏈上調用channelRegistered方法,這時,就會調用之在ServerBootstrap中向pipeline添加的ChannelInitializer的channelRegistered,進而回調initChannel方法,完成對ServerBootstrapAcceptor的添加。

回到register0方法,在處理完pipeline的責任鏈後,根據當前AbstractChannel即NioServerSocketChannel的isActive:

public boolean isActive() {
    return this.javaChannel().socket().isBound();
}

獲得NioServerSocketChannel綁定的JDK的ServerSocketChannel,進而獲取ServerSocket,判斷isBound:

public boolean isBound() {
   // Before 1.3 ServerSockets were always bound during creation
    return bound || oldImpl;
}

這裏實際上就是判斷ServerSocket是否調用了bind方法,前面說過register0方法是一個異步操作,在多線程環境下不能保證執行順序,若是此時已經完成了ServerSocket的bind,根據firstRegistration判斷是否需要pipeline傳遞channelActive請求,首先會執行pipeline的head即HeadContext的channelActive方法:

@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
    ctx.fireChannelActive();

    readIfIsAutoRead();
}

在HeadContext通過ChannelHandlerContext 傳遞完channelActive請求後,會調用readIfIsAutoRead方法:

private void readIfIsAutoRead() {
    if (channel.config().isAutoRead()) {
        channel.read();
    }
}

此時調用AbstractChannel的read方法:

public Channel read() {
	pipeline.read();
	return this;
}

最終在請求鏈由HeadContext執行read方法:

public void read(ChannelHandlerContext ctx) {
    unsafe.beginRead();
}

終於可以看到此時調用unsafe的beginRead方法:

public final void beginRead() {
    assertEventLoop();

    if (!isActive()) {
        return;
    }

    try {
        doBeginRead();
    } catch (final Exception e) {
        invokeLater(new Runnable() {
            @Override
            public void run() {
                pipeline.fireExceptionCaught(e);
            }
        });
        close(voidPromise());
    }
}

最終執行了doBeginRead方法,由AbstractNioChannel實現:

protected void doBeginRead() throws Exception {
	final SelectionKey selectionKey = this.selectionKey;
	if (!selectionKey.isValid()) {
	    return;
	}
	
	readPending = true;
	
	final int interestOps = selectionKey.interestOps();
	if ((interestOps & readInterestOp) == 0) {
	    selectionKey.interestOps(interestOps | readInterestOp);
	}
}

這裏,就完成了向Selector註冊readInterestOp事件,從前面來看就是ACCEPT事件

回到AbstractBootstrap的doBind方法,在initAndRegister邏輯結束後,由上面可以知道,實際上的register註冊邏輯是一個異步操作,在register0中完成
根據ChannelFuture來判斷異步操作是否完成,如果isDone,則表明異步操作先完成,即完成了safeSetSuccess方法,
然後調用newPromise方法:

public ChannelPromise newPromise() {
    return pipeline.newPromise();
}

給channel的pipeline綁定異步操作ChannelPromise
然後調用doBind0方法完成ServerSocket的綁定,若是register0這個異步操作還沒完成,就需要給ChannelFuture產生一個異步操作的偵聽ChannelFutureListener對象,等到register0方法調用safeSetSuccess時,在promise的trySuccess中會回調ChannelFutureListener的operationComplete方法,進而調用doBind0方法

doBind0方法:

private static void doBind0(
        final ChannelFuture regFuture, final Channel channel,
        final SocketAddress localAddress, final ChannelPromise promise) {
    channel.eventLoop().execute(new Runnable() {
        @Override
        public void run() {
            if (regFuture.isSuccess()) {
                channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
            } else {
                promise.setFailure(regFuture.cause());
            }
        }
    });
}

向輪詢線程提交了一個任務,異步處理bind,可以看到,只有在regFuture異步操作成功結束後,調用channel的bind方法:

public ChannelFuture bind(SocketAddress localAddress, ChannelPromise promise) {
   return pipeline.bind(localAddress, promise);
}

實際上的bind又交給pipeline,去完成,pipeline中就會交給責任鏈去完成,最終會交給HeadContext完成:

public void bind(
                ChannelHandlerContext ctx, SocketAddress localAddress, ChannelPromise promise)
                throws Exception {
    unsafe.bind(localAddress, promise);
}

可以看到,繞了一大圈,交給了unsafe完成:

public final void bind(final SocketAddress localAddress, final ChannelPromise promise) {
    assertEventLoop();

    if (!promise.setUncancellable() || !ensureOpen(promise)) {
        return;
    }
    
    if (Boolean.TRUE.equals(config().getOption(ChannelOption.SO_BROADCAST)) &&
        localAddress instanceof InetSocketAddress &&
        !((InetSocketAddress) localAddress).getAddress().isAnyLocalAddress() &&
        !PlatformDependent.isWindows() && !PlatformDependent.maybeSuperUser()) {
        logger.warn(
                "A non-root user can't receive a broadcast packet if the socket " +
                "is not bound to a wildcard address; binding to a non-wildcard " +
                "address (" + localAddress + ") anyway as requested.");
    }

    boolean wasActive = isActive();
    try {
        doBind(localAddress);
    } catch (Throwable t) {
        safeSetFailure(promise, t);
        closeIfClosed();
        return;
    }

    if (!wasActive && isActive()) {
        invokeLater(new Runnable() {
            @Override
            public void run() {
                pipeline.fireChannelActive();
            }
        });
    }

    safeSetSuccess(promise);
}

然而,真正的bind還是回調了doBind方法,最終是由NioServerSocketChannel來實現:

@Override
protected void doBind(SocketAddress localAddress) throws Exception {
    if (PlatformDependent.javaVersion() >= 7) {
        javaChannel().bind(localAddress, config.getBacklog());
    } else {
        javaChannel().socket().bind(localAddress, config.getBacklog());
    }
}

在這裏終於完成了對JDK的ServerSocketChannel的bind操作

在上面的

if (!wasActive && isActive()) {
    invokeLater(new Runnable() {
        @Override
        public void run() {
            pipeline.fireChannelActive();
        }
    });
}

這個判斷,就是確保在register0中isActive時,還沒完成綁定,也就沒有beginRead操作來向Selector註冊ACCEPT事件,那麼就在這裏進行註冊,進而讓ServerSocket去偵聽客戶端的連接

在服務端ACCEPT到客戶端的連接後,在NioEventLoop輪詢中,就會調用processSelectedKeys處理ACCEPT的事件就緒,然後交給unsafe的read去處理 Netty中NioEventLoopGroup的創建源碼分析

在服務端,由NioMessageUnsafe實現:

public void read() {
        assert eventLoop().inEventLoop();
        final ChannelConfig config = config();
        final ChannelPipeline pipeline = pipeline();
        final RecvByteBufAllocator.Handle allocHandle = unsafe().recvBufAllocHandle();
        allocHandle.reset(config);

        boolean closed = false;
        Throwable exception = null;
        try {
            try {
                do {
                    int localRead = doReadMessages(readBuf);
                    if (localRead == 0) {
                        break;
                    }
                    if (localRead < 0) {
                        closed = true;
                        break;
                    }

                    allocHandle.incMessagesRead(localRead);
                } while (allocHandle.continueReading());
            } catch (Throwable t) {
                exception = t;
            }

            int size = readBuf.size();
            for (int i = 0; i < size; i ++) {
                readPending = false;
                pipeline.fireChannelRead(readBuf.get(i));
            }
            readBuf.clear();
            allocHandle.readComplete();
            pipeline.fireChannelReadComplete();

            if (exception != null) {
                closed = closeOnReadError(exception);

                pipeline.fireExceptionCaught(exception);
            }

            if (closed) {
                inputShutdown = true;
                if (isOpen()) {
                    close(voidPromise());
                }
            }
        } finally {
            if (!readPending && !config.isAutoRead()) {
                removeReadOp();
            }
        }
    }
}

核心在doReadMessages方法,由NioServerSocketChannel實現:

protected int doReadMessages(List<Object> buf) throws Exception {
    SocketChannel ch = SocketUtils.accept(javaChannel());

    try {
        if (ch != null) {
            buf.add(new NioSocketChannel(this, ch));
            return 1;
        }
    } catch (Throwable t) {
        logger.warn("Failed to create a new channel from an accepted socket.", t);

        try {
            ch.close();
        } catch (Throwable t2) {
            logger.warn("Failed to close a socket.", t2);
        }
    }

    return 0;
}

SocketUtils的accept方法其實就是用來調用JDK中ServerSocketChannel原生的accept方法,來得到一個JDK的SocketChannel對象,然後通過這個SocketChannel對象,將其包裝成NioSocketChannel對象添加在buf這個List中

由此可以看到doReadMessages用來偵聽所有就緒的連接,包裝成NioSocketChannel將其放在List中
然後遍歷這個List,調用 NioServerSocketChannel的pipeline的fireChannelRead方法,傳遞channelRead請求,、
在前面向pipeline中添加了ServerBootstrapAcceptor這個ChannelHandler,此時,它也會響應這個請求,回調channelRead方法:

public void channelRead(ChannelHandlerContext ctx, Object msg) {
    final Channel child = (Channel) msg;

    child.pipeline().addLast(childHandler);

    setChannelOptions(child, childOptions, logger);

    for (Entry<AttributeKey<?>, Object> e: childAttrs) {
        child.attr((AttributeKey<Object>) e.getKey()).set(e.getValue());
    }

    try {
        childGroup.register(child).addListener(new ChannelFutureListener() {
            @Override
            public void operationComplete(ChannelFuture future) throws Exception {
                if (!future.isSuccess()) {
                    forceClose(child, future.cause());
                }
            }
        });
    } catch (Throwable t) {
        forceClose(child, t);
    }
}

msg就是偵聽到的NioSocketChannel對象,給該對象的pipeline添加childHandler,也就是我們在ServerBootstrap中通過childHandler方法添加的
然後通過register方法完成對NioSocketChannel的註冊(和NioServerSocketChannel註冊邏輯一樣)

至此Netty服務端的啓動結束。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章