渣渣菜雞的 ElasticSearch 源碼解析 —— 啓動流程（上）

前提

上篇文章寫了 ElasticSearch 源碼解析 —— 環境搭建 ，其中裏面說了啓動 打開 server 模塊下的 Elasticsearch 類：org.elasticsearch.bootstrap.Elasticsearch，運行裏面的 main 函數就可以啓動 ElasticSearch 了，這篇文章講講啓動流程，因爲篇幅會很多，所以分了兩篇來寫。

啓動流程

main 方法入口

可以看到入口其實是一個 main 方法，方法裏面先是檢查權限，然後是一個錯誤日誌監聽器（確保在日誌配置之前狀態日誌沒有出現 error），然後是 Elasticsearch 對象的創建，然後調用了靜態方法 main 方法（18 行），並把創建的對象和參數以及 Terminal 默認值傳進去。靜態的 main 方法裏面調用 elasticsearch.main 方法。

 1public static void main(final String[] args) throws Exception {         //1、入口
 2    // we want the JVM to think there is a security manager installed so that if internal policy decisions that would be based on the
 3    // presence of a security manager or lack thereof act as if there is a security manager present (e.g., DNS cache policy)
 4    System.setSecurityManager(new SecurityManager() {
 5        @Override
 6        public void checkPermission(Permission perm) {
 7            // grant all permissions so that we can later set the security manager to the one that we want
 8        }
 9    });
10    LogConfigurator.registerErrorListener();                            //
11    final Elasticsearch elasticsearch = new Elasticsearch();
12    int status = main(args, elasticsearch, Terminal.DEFAULT); //2、調用Elasticsearch.main方法
13    if (status != ExitCodes.OK) {
14        exit(status);
15    }
16}
17
18static int main(final String[] args, final Elasticsearch elasticsearch, final Terminal terminal) throws Exception {
19    return elasticsearch.main(args, terminal);  //3、command main
20}

因爲 Elasticsearch 類是繼承了 EnvironmentAwareCommand 類，EnvironmentAwareCommand 類繼承了 Command 類，但是 Elasticsearch 類並沒有重寫 main 方法，所以上面調用的 elasticsearch.main 其實是調用了 Command 的 main 方法，代碼如下：

 1/** Parses options for this command from args and executes it. */
 2public final int main(String[] args, Terminal terminal) throws Exception {
 3    if (addShutdownHook()) {                                                //利用Runtime.getRuntime().addShutdownHook方法加入一個Hook，在程序退出時觸發該Hook
 4        shutdownHookThread = new Thread(() -> {
 5            try {
 6                this.close();
 7            } catch (final IOException e) {
 8                try (
 9                    StringWriter sw = new StringWriter();
10                    PrintWriter pw = new PrintWriter(sw)) {
11                    e.printStackTrace(pw);
12                    terminal.println(sw.toString());
13                } catch (final IOException impossible) {
14                    // StringWriter#close declares a checked IOException from the Closeable interface but the Javadocs for StringWriter
15                    // say that an exception here is impossible
16                    throw new AssertionError(impossible);
17                }
18            }
19        });
20        Runtime.getRuntime().addShutdownHook(shutdownHookThread);
21    }
22
23    beforeMain.run();
24
25    try {
26        mainWithoutErrorHandling(args, terminal);//4、mainWithoutErrorHandling
27    } catch (OptionException e) {
28        printHelp(terminal);
29        terminal.println(Terminal.Verbosity.SILENT, "ERROR: " + e.getMessage());
30        return ExitCodes.USAGE;
31    } catch (UserException e) {
32        if (e.exitCode == ExitCodes.USAGE) {
33            printHelp(terminal);
34        }
35        terminal.println(Terminal.Verbosity.SILENT, "ERROR: " + e.getMessage());
36        return e.exitCode;
37    }
38    return ExitCodes.OK;
39}

上面代碼一開始利用一個勾子函數，在程序退出時觸發該 Hook，該方法主要代碼是 mainWithoutErrorHandling() 方法，然後下面的是 catch 住方法拋出的異常，方法代碼如下：

 1/*** Executes the command, but all errors are thrown. */
 2void mainWithoutErrorHandling(String[] args, Terminal terminal) throws Exception {
 3    final OptionSet options = parser.parse(args);
 4    if (options.has(helpOption)) {
 5        printHelp(terminal);
 6        return;
 7    }
 8    if (options.has(silentOption)) {
 9        terminal.setVerbosity(Terminal.Verbosity.SILENT);
10    } else if (options.has(verboseOption)) {
11        terminal.setVerbosity(Terminal.Verbosity.VERBOSE);
12    } else {
13        terminal.setVerbosity(Terminal.Verbosity.NORMAL);
14    }
15    execute(terminal, options);//5、執行 EnvironmentAwareCommand 中的 execute()，（重寫了command裏面抽象的execute方法）
16}

上面的代碼從 3 ～ 14 行是解析傳進來的參數並配置 terminal，重要的 execute() 方法，執行的是 EnvironmentAwareCommand 中的 execute() （重寫了 Command 類裏面的抽象 execute 方法），從上面那個繼承圖可以看到 EnvironmentAwareCommand 繼承了 Command，重寫的 execute 方法代碼如下：

 1@Override
 2protected void execute(Terminal terminal, OptionSet options) throws Exception {
 3    final Map<String, String> settings = new HashMap<>();
 4    for (final KeyValuePair kvp : settingOption.values(options)) {
 5        if (kvp.value.isEmpty()) {
 6            throw new UserException(ExitCodes.USAGE, "setting [" + kvp.key + "] must not be empty");
 7        }
 8        if (settings.containsKey(kvp.key)) {
 9            final String message = String.format(
10                Locale.ROOT, "setting [%s] already set, saw [%s] and [%s]",
11                kvp.key, settings.get(kvp.key), kvp.value);
12            throw new UserException(ExitCodes.USAGE, message);
13        }
14        settings.put(kvp.key, kvp.value);
15    }
16    //6、根據我們ide配置的 vm options 進行設置path.data、path.home、path.logs
17    putSystemPropertyIfSettingIsMissing(settings, "path.data", "es.path.data");
18    putSystemPropertyIfSettingIsMissing(settings, "path.home", "es.path.home");
19    putSystemPropertyIfSettingIsMissing(settings, "path.logs", "es.path.logs");
20
21    execute(terminal, options, createEnv(terminal, settings));//7、先調用 createEnv 創建環境
22    //9、執行elasticsearch的execute方法，elasticsearch中重寫了EnvironmentAwareCommand中的抽象execute方法
23}

方法前面是根據傳參去判斷配置的，如果配置爲空，就會直接跳到執行 putSystemPropertyIfSettingIsMissing 方法，這裏會配置三個屬性：path.data、path.home、path.logs 設置 es 的 data、home、logs 目錄，它這裏是根據我們 ide 配置的 vm options 進行設置的，這也是爲什麼我們上篇文章說的配置信息，如果不配置的話就會直接報錯。下面看看 putSystemPropertyIfSettingIsMissing 方法代碼裏面怎麼做到的：

 1/** Ensure the given setting exists, reading it from system properties if not already set. */
 2private static void putSystemPropertyIfSettingIsMissing(final Map<String, String> settings, final String setting, final String key) {
 3    final String value = System.getProperty(key);//獲取key（es.path.data）找系統設置
 4    if (value != null) {
 5        if (settings.containsKey(setting)) {
 6            final String message =
 7                String.format(
 8                Locale.ROOT,
 9                "duplicate setting [%s] found via command-line [%s] and system property [%s]",
10                setting, settings.get(setting), value);
11            throw new IllegalArgumentException(message);
12        } else {
13            settings.put(setting, value);
14        }
15    }
16}

執行這三個方法後：

跳出此方法，繼續看會發現 execute 方法調用了方法，

1 execute(terminal, options, createEnv(terminal, settings));

這裏我們先看看 createEnv(terminal, settings) 方法：

1protected Environment createEnv(final Terminal terminal, final Map<String, String> settings) throws UserException {
2    final String esPathConf = System.getProperty("es.path.conf");//8、讀取我們 vm options 中配置的 es.path.conf
3    if (esPathConf == null) {
4        throw new UserException(ExitCodes.CONFIG, "the system property [es.path.conf] must be set");
5    }
6    return InternalSettingsPreparer.prepareEnvironment(Settings.EMPTY, terminal, settings, getConfigPath(esPathConf));  //8、準備環境 prepareEnvironment
7}

讀取我們 ide vm options 中配置的 es.path.conf，同上篇文章也講了這個一定要配置的，因爲 es 啓動的時候會加載我們的配置和一些插件。這裏繼續看下上面代碼第 6 行的 prepareEnvironment 方法：

 1public static Environment prepareEnvironment(Settings input, Terminal terminal, Map<String, String> properties, Path configPath) {
 2    // just create enough settings to build the environment, to get the config dir
 3    Settings.Builder output = Settings.builder();
 4    initializeSettings(output, input, properties);
 5    Environment environment = new Environment(output.build(), configPath);
 6
 7    //查看 es.path.conf 目錄下的配置文件是不是 yml 格式的，如果不是則拋出一個異常
 8    if (Files.exists(environment.configFile().resolve("elasticsearch.yaml"))) {
 9        throw new SettingsException("elasticsearch.yaml was deprecated in 5.5.0 and must be renamed to elasticsearch.yml");
10    }
11
12    if (Files.exists(environment.configFile().resolve("elasticsearch.json"))) {
13        throw new SettingsException("elasticsearch.json was deprecated in 5.5.0 and must be converted to elasticsearch.yml");
14    }
15
16    output = Settings.builder(); // start with a fresh output
17    Path path = environment.configFile().resolve("elasticsearch.yml");
18    if (Files.exists(path)) {
19        try {
20            output.loadFromPath(path);  //加載文件並讀取配置文件內容
21        } catch (IOException e) {
22            throw new SettingsException("Failed to load settings from " + path.toString(), e);
23        }
24    }
25
26    // re-initialize settings now that the config file has been loaded
27    initializeSettings(output, input, properties);          //再一次初始化設置
28    finalizeSettings(output, terminal);
29
30    environment = new Environment(output.build(), configPath);
31
32    // we put back the path.logs so we can use it in the logging configuration file
33    output.put(Environment.PATH_LOGS_SETTING.getKey(), environment.logsFile().toAbsolutePath().normalize().toString());
34    return new Environment(output.build(), configPath);
35}

準備的環境如上圖，通過構建的環境查看配置文件 elasticsearch.yml 是不是以 yml 結尾，如果是 yaml 或者 json 結尾的則拋出異常（在 5.5.0 版本其他兩種格式過期了，只能使用 yml 格式），然後加載該配置文件並讀取裏面的內容（KV結構）。

跳出 createEnv 方法，我們繼續看 execute 方法吧。

EnvironmentAwareCommand 類的 execute 方法代碼如下：

1protected abstract void execute(Terminal terminal, OptionSet options, Environment env) throws Exception;

這是個抽象方法，那麼它的實現方法在 Elasticsearch 類中，代碼如下：

 1@Override
 2protected void execute(Terminal terminal, OptionSet options, Environment env) throws UserException {
 3    if (options.nonOptionArguments().isEmpty() == false) {
 4        throw new UserException(ExitCodes.USAGE, "Positional arguments not allowed, found " + options.nonOptionArguments());
 5    }
 6    if (options.has(versionOption)) {
 7        final String versionOutput = String.format(
 8            Locale.ROOT,
 9            "Version: %s, Build: %s/%s/%s/%s, JVM: %s",
10            Version.displayVersion(Version.CURRENT, Build.CURRENT.isSnapshot()),
11            Build.CURRENT.flavor().displayName(),
12            Build.CURRENT.type().displayName(),
13            Build.CURRENT.shortHash(),
14            Build.CURRENT.date(),
15            JvmInfo.jvmInfo().version());
16        terminal.println(versionOutput);
17        return;
18    }
19
20    final boolean daemonize = options.has(daemonizeOption);
21    final Path pidFile = pidfileOption.value(options);
22    final boolean quiet = options.has(quietOption);
23
24    // a misconfigured java.io.tmpdir can cause hard-to-diagnose problems later, so reject it immediately
25    try {
26        env.validateTmpFile();
27    } catch (IOException e) {
28        throw new UserException(ExitCodes.CONFIG, e.getMessage());
29    }
30    try {
31        init(daemonize, pidFile, quiet, env);    //10、初始化
32    } catch (NodeValidationException e) {
33        throw new UserException(ExitCodes.CONFIG, e.getMessage());
34    }
35}

上面代碼裏主要還是看看 init(daemonize, pidFile, quiet, env); 初始化方法吧。

 1void init(final boolean daemonize, final Path pidFile, final boolean quiet, Environment initialEnv)
 2    throws NodeValidationException, UserException {
 3    try {
 4        Bootstrap.init(!daemonize, pidFile, quiet, initialEnv); //11、執行 Bootstrap 中的 init 方法
 5    } catch (BootstrapException | RuntimeException e) {
 6        // format exceptions to the console in a special way
 7        // to avoid 2MB stacktraces from guice, etc.
 8        throw new StartupException(e);
 9    }
10}

init 方法

Bootstrap 中的靜態 init 方法如下：

  1static void init(
  2    final boolean foreground,
  3    final Path pidFile,
  4    final boolean quiet,
  5    final Environment initialEnv) throws BootstrapException, NodeValidationException, UserException {
  6    // force the class initializer for BootstrapInfo to run before
  7    // the security manager is installed
  8    BootstrapInfo.init();
  9
 10    INSTANCE = new Bootstrap();   //12、創建一個 Bootstrap 實例
 11
 12    final SecureSettings keystore = loadSecureSettings(initialEnv);//如果註冊了安全模塊則將相關配置加載進來
 13    final Environment environment = createEnvironment(foreground, pidFile, keystore, initialEnv.settings(), initialEnv.configFile());   //幹之前幹過的事情
 14    try {
 15        LogConfigurator.configure(environment);   //13、log 配置環境
 16    } catch (IOException e) {
 17        throw new BootstrapException(e);
 18    }
 19    if (environment.pidFile() != null) {
 20        try {
 21            PidFile.create(environment.pidFile(), true);
 22        } catch (IOException e) {
 23            throw new BootstrapException(e);
 24        }
 25    }
 26
 27    final boolean closeStandardStreams = (foreground == false) || quiet;
 28    try {
 29        if (closeStandardStreams) {
 30            final Logger rootLogger = ESLoggerFactory.getRootLogger();
 31            final Appender maybeConsoleAppender = Loggers.findAppender(rootLogger, ConsoleAppender.class);
 32            if (maybeConsoleAppender != null) {
 33                Loggers.removeAppender(rootLogger, maybeConsoleAppender);
 34            }
 35            closeSystOut();
 36        }
 37
 38        // fail if somebody replaced the lucene jars
 39        checkLucene();             //14、檢查Lucene版本
 40
 41// install the default uncaught exception handler; must be done before security is initialized as we do not want to grant the runtime permission setDefaultUncaughtExceptionHandler
 42        Thread.setDefaultUncaughtExceptionHandler(
 43            new ElasticsearchUncaughtExceptionHandler(() -> Node.NODE_NAME_SETTING.get(environment.settings())));
 44
 45        INSTANCE.setup(true, environment);      //15、調用 setup 方法
 46
 47        try {
 48            // any secure settings must be read during node construction
 49            IOUtils.close(keystore);
 50        } catch (IOException e) {
 51            throw new BootstrapException(e);
 52        }
 53
 54        INSTANCE.start();         //26、調用 start 方法
 55
 56        if (closeStandardStreams) {
 57            closeSysError();
 58        }
 59    } catch (NodeValidationException | RuntimeException e) {
 60        // disable console logging, so user does not see the exception twice (jvm will show it already)
 61        final Logger rootLogger = ESLoggerFactory.getRootLogger();
 62        final Appender maybeConsoleAppender = Loggers.findAppender(rootLogger, ConsoleAppender.class);
 63        if (foreground && maybeConsoleAppender != null) {
 64            Loggers.removeAppender(rootLogger, maybeConsoleAppender);
 65        }
 66        Logger logger = Loggers.getLogger(Bootstrap.class);
 67        if (INSTANCE.node != null) {
 68            logger = Loggers.getLogger(Bootstrap.class, Node.NODE_NAME_SETTING.get(INSTANCE.node.settings()));
 69        }
 70        // HACK, it sucks to do this, but we will run users out of disk space otherwise
 71        if (e instanceof CreationException) {
 72            // guice: log the shortened exc to the log file
 73            ByteArrayOutputStream os = new ByteArrayOutputStream();
 74            PrintStream ps = null;
 75            try {
 76                ps = new PrintStream(os, false, "UTF-8");
 77            } catch (UnsupportedEncodingException uee) {
 78                assert false;
 79                e.addSuppressed(uee);
 80            }
 81            new StartupException(e).printStackTrace(ps);
 82            ps.flush();
 83            try {
 84                logger.error("Guice Exception: {}", os.toString("UTF-8"));
 85            } catch (UnsupportedEncodingException uee) {
 86                assert false;
 87                e.addSuppressed(uee);
 88            }
 89        } else if (e instanceof NodeValidationException) {
 90            logger.error("node validation exception\n{}", e.getMessage());
 91        } else {
 92            // full exception
 93            logger.error("Exception", e);
 94        }
 95        // re-enable it if appropriate, so they can see any logging during the shutdown process
 96        if (foreground && maybeConsoleAppender != null) {
 97            Loggers.addAppender(rootLogger, maybeConsoleAppender);
 98        }
 99        throw e;
100    }
101}

該方法主要有：

1、創建 Bootstrap 實例

2、如果註冊了安全模塊則將相關配置加載進來

3、創建 Elasticsearch 運行的必須環境以及相關配置, 如將 config、scripts、plugins、modules、logs、lib、bin 等配置目錄加載到運行環境中

4、log 配置環境，創建日誌上下文

5、檢查是否存在 PID 文件，如果不存在，創建 PID 文件

6、檢查 Lucene 版本

7、調用 setup 方法（用當前環境來創建一個節點）

setup 方法

 1private void setup(boolean addShutdownHook, Environment environment) throws BootstrapException {
 2    Settings settings = environment.settings();//根據環境得到配置
 3    try {
 4        spawner.spawnNativeControllers(environment);
 5    } catch (IOException e) {
 6        throw new BootstrapException(e);
 7    }
 8    initializeNatives(
 9        environment.tmpFile(),
10        BootstrapSettings.MEMORY_LOCK_SETTING.get(settings),
11        BootstrapSettings.SYSTEM_CALL_FILTER_SETTING.get(settings),
12        BootstrapSettings.CTRLHANDLER_SETTING.get(settings));
13    // initialize probes before the security manager is installed
14    initializeProbes();
15    if (addShutdownHook) {
16        Runtime.getRuntime().addShutdownHook(new Thread() {
17            @Override
18            public void run() {
19                try {
20                    IOUtils.close(node, spawner);
21                    LoggerContext context = (LoggerContext) LogManager.getContext(false);
22                    Configurator.shutdown(context);
23                } catch (IOException ex) {
24                    throw new ElasticsearchException("failed to stop node", ex);
25                }
26            }
27        });
28    }
29    try {
30        // look for jar hell
31        final Logger logger = ESLoggerFactory.getLogger(JarHell.class);
32        JarHell.checkJarHell(logger::debug);
33    } catch (IOException | URISyntaxException e) {
34        throw new BootstrapException(e);
35    }
36    // Log ifconfig output before SecurityManager is installed
37    IfConfig.logIfNecessary();
38    // install SM after natives, shutdown hooks, etc.
39    try {
40        Security.configure(environment, BootstrapSettings.SECURITY_FILTER_BAD_DEFAULTS_SETTING.get(settings));
41    } catch (IOException | NoSuchAlgorithmException e) {
42        throw new BootstrapException(e);
43    }
44    node = new Node(environment) {              //16、新建節點
45        @Override
46        protected void validateNodeBeforeAcceptingRequests(
47            final BootstrapContext context,
48            final BoundTransportAddress boundTransportAddress, List<BootstrapCheck> checks) throws NodeValidationException {
49            BootstrapChecks.check(context, boundTransportAddress, checks);
50        }
51    };
52}

上面代碼最後就是 Node 節點的創建，這篇文章就不講 Node 的創建了，下篇文章會好好講一下 Node 節點的創建和正式啓動 ES 節點的。

總結

這篇文章主要先把大概啓動流程串通，因爲篇幅較多所以拆開成兩篇，先不扣細節了，後面流程啓動文章寫完後我們再單一的扣細節。

轉載請務必註明原創地址爲：http://www.54tianzhisheng.cn/2018/08/11/es-code02/