我的架構夢：（十七）Tomcat 源碼構建以及源碼剖析

Tomcat 源碼構建以及源碼剖析

一、源碼構建

1、下載源碼

這裏博主下載的是apache-tomcat-8.5.50-src

http://archive.apache.org/dist/tomcat/tomcat-8/v8.5.50/src/

2、源碼導入IDE之前準備工作

解壓 tar.gz 壓縮包，得到目錄 apache-tomcat-8.5.50-src

進入 apache-tomcat-8.5.50-src 目錄，創建一個pom.xml文件，文件內容如下

<?xml version="1.0" encoding="utf-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">  
  <modelVersion>4.0.0</modelVersion>  
  <groupId>org.apache.tomcat</groupId>  
  <artifactId>apache-tomcat-8.5.50-src</artifactId>  
  <name>Tomcat8.5</name>  
  <version>8.5</version>  
  <build> 
    <!--指定源目錄-->  
    <finalName>Tomcat8.5</finalName>  
    <sourceDirectory>java</sourceDirectory>  
    <resources> 
      <resource> 
        <directory>java</directory> 
      </resource> 
    </resources>  
    <plugins> 
      <!--引入編譯插件-->  
      <plugin> 
        <groupId>org.apache.maven.plugins</groupId>  
        <artifactId>maven-compiler-plugin</artifactId>  
        <version>3.1</version>  
        <configuration> 
          <encoding>UTF-8</encoding>  
          <source>1.8</source>  
          <target>1.8</target> 
        </configuration> 
      </plugin> 
    </plugins> 
  </build>  
  <!--tomcat 依賴的基礎包-->  
  <dependencies> 
    <dependency> 
      <groupId>org.easymock</groupId>  
      <artifactId>easymock</artifactId>  
      <version>3.4</version> 
    </dependency>  
    <dependency> 
      <groupId>ant</groupId>  
      <artifactId>ant</artifactId>  
      <version>1.7.0</version> 
    </dependency>  
    <dependency> 
      <groupId>wsdl4j</groupId>  
      <artifactId>wsdl4j</artifactId>  
      <version>1.6.2</version> 
    </dependency>  
    <dependency> 
      <groupId>javax.xml</groupId>  
      <artifactId>jaxrpc</artifactId>  
      <version>1.1</version> 
    </dependency>  
    <dependency> 
      <groupId>org.eclipse.jdt.core.compiler</groupId>  
      <artifactId>ecj</artifactId>  
      <version>4.5.1</version> 
    </dependency>  
    <dependency> 
      <groupId>javax.xml.soap</groupId>  
      <artifactId>javax.xml.soap-api</artifactId>  
      <version>1.4.0</version> 
    </dependency> 
  </dependencies> 
</project>

在 apache-tomcat-8.5.50-src 目錄中創建 source 文件夾
將 conf、webapps 目錄移動到剛剛創建的 source 文件夾中

3、導入源碼工程到IDE並進行配置

將源碼工程導入到 IDEA 中

給 tomcat 的源碼程序啓動類 Bootstrap 配置 VM 參數，因爲 tomcat 源碼運行也需要加載配置文件等。

-Dcatalina.home=/Users/Riemann/Code/framework-source-code-analysis/apache-tomcat-8.5.50-src/source
-Dcatalina.base=/Users/Riemann/Code/framework-source-code-analysis/apache-tomcat-8.5.50-src/source
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 
-Djava.util.logging.config.file=/Users/Riemann/Code/framework-source-code-analysis/apache-tomcat-8.5.50-src/source/conf/logging.properties

運行 Bootstrap 類的 main 函數，此時就啓動了tomcat，啓動時候會去加載所配置的 conf 目錄下的server.xml等配置文件，所以訪問8080端口即可，但此時我們會遇到如下的一個錯誤。

原因是Jsp引擎Jasper沒有被初始化，從而無法編譯JSP，我們需要在tomcat的源碼ContextConfig類中的configureStart方法中增加一行代碼將 Jsp 引擎初始化，如下：

4、測試Tomcat是否構建成功

把前面的準備工作都搞好了以後，啓動一下：

來到了我們熟悉的頁面，這就說明我們的Tomcat源碼構建成功！

二、源碼剖析

源碼追蹤部分我們關注兩個流程：Tomcat啓動流程和Tomcat請求處理流程

1、Tomcat啓動流程

啓動流程時序圖如下：

2、Tomcat請求處理流程

2.1 請求處理流程分析

2.2 請求處理流程示意圖

3、源碼分析

3.1 第一步 main方法入口 Bootstrap#main()

// 時序圖中第一步 main方法入口
public static void main(String args[]) {

    synchronized (daemonLock) {
        if (daemon == null) {
            // Don't set daemon until init() has completed
            Bootstrap bootstrap = new Bootstrap();
            try {
                // 第二步 init() Bootstrap初始化
                bootstrap.init();
            } catch (Throwable t) {
                handleThrowable(t);
                t.printStackTrace();
                return;
            }
            daemon = bootstrap; // daemon表示當前Bootstrap類對象本身
        } else {
            // When running as a service the call to stop will be on a new
            // thread so make sure the correct class loader is used to
            // prevent a range of class not found exceptions.
            Thread.currentThread().setContextClassLoader(daemon.catalinaLoader);
        }
    }

    try {
        String command = "start";
        if (args.length > 0) {
            command = args[args.length - 1];
        }

        if (command.equals("startd")) {
            args[args.length - 1] = "start";
            daemon.load(args);
            daemon.start();
        } else if (command.equals("stopd")) {
            args[args.length - 1] = "stop";
            daemon.stop();
        } else if (command.equals("start")) {
            daemon.setAwait(true);
            // 第三步 load() Bootstrap加載
            daemon.load(args);
            // 第十四步 start() Bootstrap啓動
            daemon.start();
            if (null == daemon.getServer()) {
                System.exit(1);
            }
        } else if (command.equals("stop")) {
            daemon.stopServer(args);
        } else if (command.equals("configtest")) {
            daemon.load(args);
            if (null == daemon.getServer()) {
                System.exit(1);
            }
            System.exit(0);
        } else {
            log.warn("Bootstrap: command \"" + command + "\" does not exist.");
        }
    } catch (Throwable t) {
        // Unwrap the Exception for clearer error reporting
        if (t instanceof InvocationTargetException &&
                t.getCause() != null) {
            t = t.getCause();
        }
        handleThrowable(t);
        t.printStackTrace();
        System.exit(1);
    }
}

3.2 第二步 init() Bootstrap初始化 Bootstrap#init()

public void init() throws Exception {

    // 初始化類加載器
    initClassLoaders();

    Thread.currentThread().setContextClassLoader(catalinaLoader);

    SecurityClassLoad.securityClassLoad(catalinaLoader);

    // Load our startup class and call its process() method
    if (log.isDebugEnabled())
        log.debug("Loading startup class");
    Class<?> startupClass = catalinaLoader.loadClass("org.apache.catalina.startup.Catalina");
    Object startupInstance = startupClass.getConstructor().newInstance();

    // Set the shared extensions class loader
    if (log.isDebugEnabled())
        log.debug("Setting startup class properties");
    String methodName = "setParentClassLoader";
    Class<?> paramTypes[] = new Class[1];
    paramTypes[0] = Class.forName("java.lang.ClassLoader");
    Object paramValues[] = new Object[1];
    paramValues[0] = sharedLoader;
    Method method =
        startupInstance.getClass().getMethod(methodName, paramTypes);
    method.invoke(startupInstance, paramValues);

    catalinaDaemon = startupInstance; // catalinaDaemon 是一個 Catalina 的實例
}

3.3 第三步 load() Bootstrap加載 Bootstrap#load()

private void load(String[] arguments) throws Exception {

    // Call the load() method
    String methodName = "load";
    Object param[];
    Class<?> paramTypes[];
    if (arguments==null || arguments.length==0) {
        paramTypes = null;
        param = null;
    } else {
        paramTypes = new Class[1];
        paramTypes[0] = arguments.getClass();
        param = new Object[1];
        param[0] = arguments;
    }
    Method method =
        catalinaDaemon.getClass().getMethod(methodName, paramTypes);
    if (log.isDebugEnabled()) {
        log.debug("Calling startup class " + method);
    }
    // 第四步 load() Catalina加載
    method.invoke(catalinaDaemon, param); // Catalina.load()
}

3.4 第四步 load() Catalina加載反射調用 Catalina#load()

public void load() {

    if (loaded) {
        return;
    }
    loaded = true;

    long t1 = System.nanoTime();

    initDirs();

    // Before digester - it may be needed
    initNaming();

    // Create and execute our Digester
    // xml配置文件的解析器，比如解析server.xml
    Digester digester = createStartDigester();

    InputSource inputSource = null;
    InputStream inputStream = null;
    File file = null;
    try {
        try {
            file = configFile();
            inputStream = new FileInputStream(file);
            inputSource = new InputSource(file.toURI().toURL().toString());
        } catch (Exception e) {
            if (log.isDebugEnabled()) {
                log.debug(sm.getString("catalina.configFail", file), e);
            }
        }
        if (inputStream == null) {
            try {
                inputStream = getClass().getClassLoader()
                    .getResourceAsStream(getConfigFile());
                inputSource = new InputSource
                    (getClass().getClassLoader()
                     .getResource(getConfigFile()).toString());
            } catch (Exception e) {
                if (log.isDebugEnabled()) {
                    log.debug(sm.getString("catalina.configFail",
                            getConfigFile()), e);
                }
            }
        }

        // This should be included in catalina.jar
        // Alternative: don't bother with xml, just create it manually.
        if (inputStream == null) {
            try {
                inputStream = getClass().getClassLoader()
                        .getResourceAsStream("server-embed.xml");
                inputSource = new InputSource
                (getClass().getClassLoader()
                        .getResource("server-embed.xml").toString());
            } catch (Exception e) {
                if (log.isDebugEnabled()) {
                    log.debug(sm.getString("catalina.configFail",
                            "server-embed.xml"), e);
                }
            }
        }


        if (inputStream == null || inputSource == null) {
            if  (file == null) {
                log.warn(sm.getString("catalina.configFail",
                        getConfigFile() + "] or [server-embed.xml]"));
            } else {
                log.warn(sm.getString("catalina.configFail",
                        file.getAbsolutePath()));
                if (file.exists() && !file.canRead()) {
                    log.warn("Permissions incorrect, read permission is not allowed on the file.");
                }
            }
            return;
        }

        try {
            inputSource.setByteStream(inputStream);
            digester.push(this);
            digester.parse(inputSource);
        } catch (SAXParseException spe) {
            log.warn("Catalina.start using " + getConfigFile() + ": " +
                    spe.getMessage());
            return;
        } catch (Exception e) {
            log.warn("Catalina.start using " + getConfigFile() + ": " , e);
            return;
        }
    } finally {
        if (inputStream != null) {
            try {
                inputStream.close();
            } catch (IOException e) {
                // Ignore
            }
        }
    }

    getServer().setCatalina(this);
    getServer().setCatalinaHome(Bootstrap.getCatalinaHomeFile());
    getServer().setCatalinaBase(Bootstrap.getCatalinaBaseFile());

    // Stream redirection
    initStreams();

    // Start the new server
    try {
        // 第五步 getServer() 創建Server
        // 第六步 init() Server初始化
        getServer().init();
    } catch (LifecycleException e) {
        if (Boolean.getBoolean("org.apache.catalina.startup.EXIT_ON_INIT_FAILURE")) {
            throw new java.lang.Error(e);
        } else {
            log.error("Catalina.start", e);
        }
    }

    long t2 = System.nanoTime();
    if(log.isInfoEnabled()) {
        log.info("Initialization processed in " + ((t2 - t1) / 1000000) + " ms");
    }
}

3.5 第五步 getServer() 創建Server Catalina#load()

protected Server server = null;

public Server getServer() {
    return server;
}

3.6 第六步 init() Server初始化 Catalina#load()

3.6.1 LifecycleBase#init()

3.6.2 StandardServer#initInternal()

@Override
protected void initInternal() throws LifecycleException {

    super.initInternal();

    // Register global String cache
    // Note although the cache is global, if there are multiple Servers
    // present in the JVM (may happen when embedding) then the same cache
    // will be registered under multiple names
    onameStringCache = register(new StringCache(), "type=StringCache");

    // Register the MBeanFactory
    MBeanFactory factory = new MBeanFactory();
    factory.setContainer(this);
    onameMBeanFactory = register(factory, "type=MBeanFactory");

    // Register the naming resources
    globalNamingResources.init();

    // Populate the extension validator with JARs from common and shared
    // class loaders
    if (getCatalina() != null) {
        ClassLoader cl = getCatalina().getParentClassLoader();
        // Walk the class loader hierarchy. Stop at the system class loader.
        // This will add the shared (if present) and common class loaders
        while (cl != null && cl != ClassLoader.getSystemClassLoader()) {
            if (cl instanceof URLClassLoader) {
                URL[] urls = ((URLClassLoader) cl).getURLs();
                for (URL url : urls) {
                    if (url.getProtocol().equals("file")) {
                        try {
                            File f = new File (url.toURI());
                            if (f.isFile() &&
                                    f.getName().endsWith(".jar")) {
                                ExtensionValidator.addSystemResource(f);
                            }
                        } catch (URISyntaxException e) {
                            // Ignore
                        } catch (IOException e) {
                            // Ignore
                        }
                    }
                }
            }
            cl = cl.getParent();
        }
    }
    // Initialize our defined Services
    // 第七步 init() Service初始化
    for (int i = 0; i < services.length; i++) {
        services[i].init();
    }
}

3.7 第七步 init() Service初始化

3.7.1 LifecycleBase#init()

3.7.2 StandardService#initInternal()

@Override
protected void initInternal() throws LifecycleException {

    super.initInternal();

    if (engine != null) {
        // 第八步 init() Engine初始化
        engine.init();
    }

    // Initialize any Executors
    for (Executor executor : findExecutors()) {
        if (executor instanceof JmxEnabled) {
            ((JmxEnabled) executor).setDomain(getDomain());
        }
        // 第十一步 init() Executor初始化
        executor.init();
    }

    // Initialize mapper listener
    mapperListener.init();

    // Initialize our defined Connectors
    synchronized (connectorsLock) {
        for (Connector connector : connectors) {
            try {
                // 第十二步 init() Connector初始化
                connector.init();
            } catch (Exception e) {
                String message = sm.getString(
                        "standardService.connector.initFailed", connector);
                log.error(message, e);

                if (Boolean.getBoolean("org.apache.catalina.startup.EXIT_ON_INIT_FAILURE"))
                    throw new LifecycleException(message);
            }
        }
    }
}

3.8 第八步 init() Engine初始化

3.8.1 StandardEngine#initInternal()

3.8.2 ContainerBase#initInternal()

3.9 第十二步 init() Connector初始化 Connector#initInternal()

@Override
protected void initInternal() throws LifecycleException {

    super.initInternal();

    // Initialize adapter
    // CoyoteAdapter是連接器轉發的一個組件，把request對象轉換成ServletRequest對象 (適配器的設計模式)
    adapter = new CoyoteAdapter(this);
    // ProtocolHandler裏有EndPoint和Processor，需要使用adapter。
    protocolHandler.setAdapter(adapter);

    // Make sure parseBodyMethodsSet has a default
    if (null == parseBodyMethodsSet) {
        setParseBodyMethods(getParseBodyMethods());
    }

    if (protocolHandler.isAprRequired() && !AprLifecycleListener.isAprAvailable()) {
        throw new LifecycleException(sm.getString("coyoteConnector.protocolHandlerNoApr",
                getProtocolHandlerClassName()));
    }
    if (AprLifecycleListener.isAprAvailable() && AprLifecycleListener.getUseOpenSSL() &&
            protocolHandler instanceof AbstractHttp11JsseProtocol) {
        AbstractHttp11JsseProtocol<?> jsseProtocolHandler =
                (AbstractHttp11JsseProtocol<?>) protocolHandler;
        if (jsseProtocolHandler.isSSLEnabled() &&
                jsseProtocolHandler.getSslImplementationName() == null) {
            // OpenSSL is compatible with the JSSE configuration, so use it if APR is available
            jsseProtocolHandler.setSslImplementationName(OpenSSLImplementation.class.getName());
        }
    }

    try {
        // 第十三步 init() ProtocolHandler初始化
        protocolHandler.init();
    } catch (Exception e) {
        throw new LifecycleException(
                sm.getString("coyoteConnector.protocolHandlerInitializationFailed"), e);
    }
}

3.10 第十三步 init() ProtocolHandler初始化

綁定8080端口：

3.11 第十四步 start() Bootstrap啓動 Bootstrap#main()

public void start() throws Exception {
    if (catalinaDaemon == null) {
        init();
    }

    Method method = catalinaDaemon.getClass().getMethod("start", (Class [])null);
    // 第十五步 start() Catalina啓動
    method.invoke(catalinaDaemon, (Object [])null); // Catalina.start()
}

3.12 第十五步 start() Catalina啓動反射機制 Catalina#start()

3.13 第十六步 start() Server啓動

public void start() {

    if (getServer() == null) {
        load();
    }

    if (getServer() == null) {
        log.fatal("Cannot start server. Server instance is not configured.");
        return;
    }

    long t1 = System.nanoTime();

    // Start the new server
    try {
        // 第十六步 start() Server啓動
        getServer().start();
    } catch (LifecycleException e) {
        log.fatal(sm.getString("catalina.serverStartFail"), e);
        try {
            getServer().destroy();
        } catch (LifecycleException e1) {
            log.debug("destroy() failed for failed Server ", e1);
        }
        return;
    }

    long t2 = System.nanoTime();
    if(log.isInfoEnabled()) {
        log.info("Server startup in " + ((t2 - t1) / 1000000) + " ms");
    }

    // Register shutdown hook
    if (useShutdownHook) {
        if (shutdownHook == null) {
            shutdownHook = new CatalinaShutdownHook();
        }
        Runtime.getRuntime().addShutdownHook(shutdownHook);

        // If JULI is being used, disable JULI's shutdown hook since
        // shutdown hooks run in parallel and log messages may be lost
        // if JULI's hook completes before the CatalinaShutdownHook()
        LogManager logManager = LogManager.getLogManager();
        if (logManager instanceof ClassLoaderLogManager) {
            ((ClassLoaderLogManager) logManager).setUseShutdownHook(
                    false);
        }
    }

    if (await) {
        await();
        stop();
    }
}

3.14 第十七步 start() Service啓動

@Override
protected void startInternal() throws LifecycleException {

    fireLifecycleEvent(CONFIGURE_START_EVENT, null);
    setState(LifecycleState.STARTING);

    globalNamingResources.start();

    // Start our defined Services
    // 第十七步 start() Service啓動
    synchronized (servicesLock) {
        for (int i = 0; i < services.length; i++) {
            services[i].start();
        }
    }
}

engine和executor的啓動我們就不分析了哈，和前面的初始化類似，我們來看下connector的啓動方式，繼續往下分析。

3.15 第二十二步 start() Connector啓動

 @Override
protected void startInternal() throws LifecycleException {

    // Validate settings before starting
    if (getPort() < 0) {
        throw new LifecycleException(sm.getString(
                "coyoteConnector.invalidPort", Integer.valueOf(getPort())));
    }

    setState(LifecycleState.STARTING);

    try {
        // 第二十三步 start() ProtocolHandler啓動
        protocolHandler.start();
    } catch (Exception e) {
        throw new LifecycleException(
                sm.getString("coyoteConnector.protocolHandlerStartFailed"), e);
    }
}

3.16 第二十三步 start() ProtocolHandler啓動

public final void start() throws Exception {
    if (bindState == BindState.UNBOUND) {
        bind();
        bindState = BindState.BOUND_ON_START;
    }
    startInternal();
}

// 初始化只是監聽了端口，啓動start要啓動監聽，也就是startAcceptorThreads方法
protected final void startAcceptorThreads() {
    int count = getAcceptorThreadCount();
    acceptors = new Acceptor[count];

    for (int i = 0; i < count; i++) {
        acceptors[i] = createAcceptor();
        String threadName = getName() + "-Acceptor-" + i;
        acceptors[i].setThreadName(threadName);
        Thread t = new Thread(acceptors[i], threadName);
        t.setPriority(getAcceptorThreadPriority());
        t.setDaemon(getDaemon());
        t.start();
    }
}

@Override
protected AbstractEndpoint.Acceptor createAcceptor() {
   return new Acceptor();
}

protected class Acceptor extends AbstractEndpoint.Acceptor {

    @Override
    public void run() {

        int errorDelay = 0;

        // Loop until we receive a shutdown command
        while (running) {

            // Loop if endpoint is paused
            while (paused && running) {
                state = AcceptorState.PAUSED;
                try {
                    Thread.sleep(50);
                } catch (InterruptedException e) {
                    // Ignore
                }
            }

            if (!running) {
                break;
            }
            state = AcceptorState.RUNNING;

            try {
                //if we have reached max connections, wait
                countUpOrAwaitConnection();

                SocketChannel socket = null;
                try {
                    // Accept the next incoming connection from the server
                    // socket
                    // 啓動瀏覽器的監聽，獲取socket連接請求
                    socket = serverSock.accept();
                } catch (IOException ioe) {
                    // We didn't get a socket
                    countDownConnection();
                    if (running) {
                        // Introduce delay if necessary
                        errorDelay = handleExceptionWithDelay(errorDelay);
                        // re-throw
                        throw ioe;
                    } else {
                        break;
                    }
                }
                // Successful accept, reset the error delay
                errorDelay = 0;

                // Configure the socket
                if (running && !paused) {
                    // setSocketOptions() will hand the socket off to
                    // an appropriate processor if successful
                    if (!setSocketOptions(socket)) {
                        closeSocket(socket);
                    }
                } else {
                    closeSocket(socket);
                }
            } catch (Throwable t) {
                ExceptionUtils.handleThrowable(t);
                log.error(sm.getString("endpoint.accept.fail"), t);
            }
        }
        state = AcceptorState.ENDED;
    }


    private void closeSocket(SocketChannel socket) {
        countDownConnection();
        try {
            socket.socket().close();
        } catch (IOException ioe)  {
            if (log.isDebugEnabled()) {
                log.debug(sm.getString("endpoint.err.close"), ioe);
            }
        }
        try {
            socket.close();
        } catch (IOException ioe) {
            if (log.isDebugEnabled()) {
                log.debug(sm.getString("endpoint.err.close"), ioe);
            }
        }
    }
}

AbstractEndpoint.Acceptor

4、Mapper組件體系結構

至此，我們的Tomcat 源碼構建以及源碼剖析就到此結束了，其實這裏源碼分析的啓動流程機制的源碼，有時間的話可以再分析下請求處理機制的源碼。

我的架構夢：（十七）Tomcat 源碼構建以及源碼剖析

Tomcat 源碼構建以及源碼剖析

一、源碼構建

二、源碼剖析

電子科技大學計算機科學與技術就讀體驗

Golang爬蟲代理接入的技術與實踐

我的架構夢：（十九）Nginx搭建、反向代理、負載均衡、動靜分離以及底層進程機制詳解

我的架構夢：（十三）SpringBoot 原理深入以及源碼分析

我的架構夢：（十七）Tomcat 源碼構建以及源碼剖析

我的架構夢：（十八）Tomcat 性能優化策略

我的架構夢：（十六）手寫Tomcat服務器

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結