前言
項目需要,需要搭建job調度中心,現在用的比較多的是xxxl-job,友情提示:
百度搜索時,請搜索“java xxl-job調度中心”,如果單獨搜索,【xxxl-job】會嘿嘿哈哈
XXL-JOB
官方文檔
- 源代碼地址:
https://github.com/xuxueli/xxl-job
如下所示的我下載的,代碼是從maser下載的,上面是releases下載的,但是不影響分享:
修改數據庫鏈接地址:
- 執行sql
- 啓動測試:
訪問地址:
http://localhost:9080/xxl-job-admin
賬號:admin
密碼:123456
首先創建執行器
創建任務:
參數介紹
執行器:任務的綁定的執行器,任務觸發調度時將會自動發現註冊成功的執行器, 實現任務自動發現功能; 另一方面也可以方便的進行任務分組。每個任務必須綁定一個執行器, 可在 “執行器管理” 進行設置。
任務描述:任務的描述信息,便於任務管理;
路由策略:當執行器集羣部署時,提供豐富的路由策略,包括;
FIRST(第一個):固定選擇第一個機器;
LAST(最後一個):固定選擇最後一個機器;
ROUND(輪詢):;
RANDOM(隨機):隨機選擇在線的機器;
CONSISTENT_HASH(一致性HASH):每個任務按照Hash算法固定選擇某一臺機器,且所有任務均勻散列在不同機器上。
LEAST_FREQUENTLY_USED(最不經常使用):使用頻率最低的機器優先被選舉;
LEAST_RECENTLY_USED(最近最久未使用):最久爲使用的機器優先被選舉;
FAILOVER(故障轉移):按照順序依次進行心跳檢測,第一個心跳檢測成功的機器選定爲目標執行器併發起調度;
BUSYOVER(忙碌轉移):按照順序依次進行空閒檢測,第一個空閒檢測成功的機器選定爲目標執行器併發起調度;
SHARDING_BROADCAST(分片廣播):廣播觸發對應集羣中所有機器執行一次任務,同時系統自動傳遞分片參數;可根據分片參數開發分片任務;
Cron:觸發任務執行的Cron表達式;
運行模式:
BEAN模式:任務以JobHandler方式維護在執行器端;需要結合 “JobHandler” 屬性匹配執行器中任務;
GLUE模式(Java):任務以源碼方式維護在調度中心;該模式的任務實際上是一段繼承自IJobHandler的Java類代碼並 “groovy” 源碼方式維護,它在執行器項目中運行,可使用 @Resource/@Autowire注入執行器裏中的其他服務;
GLUE模式(Shell):任務以源碼方式維護在調度中心;該模式的任務實際上是一段 “shell” 腳本;
GLUE模式(Python):任務以源碼方式維護在調度中心;該模式的任務實際上是一段 “python” 腳本;
GLUE模式(PHP):任務以源碼方式維護在調度中心;該模式的任務實際上是一段 “php” 腳本;
GLUE模式(NodeJS):任務以源碼方式維護在調度中心;該模式的任務實際上是一段 “nodejs” 腳本;
GLUE模式(PowerShell):任務以源碼方式維護在調度中心;該模式的任務實際上是一段 “PowerShell” 腳本;
JobHandler:運行模式爲 “BEAN模式” 時生效,對應執行器中新開發的JobHandler類“@JobHandler”註解自定義的value值;
阻塞處理策略:調度過於密集執行器來不及處理時的處理策略;
單機串行(默認):調度請求進入單機執行器後,調度請求進入FIFO隊列並以串行方式運行;
丟棄後續調度:調度請求進入單機執行器後,發現執行器存在運行的調度任務,本次請求將會被丟棄並標記爲失敗;
覆蓋之前調度:調度請求進入單機執行器後,發現執行器存在運行的調度任務,將會終止運行中的調度任務並清空隊列,然後運行本地調度任務;
子任務:每個任務都擁有一個唯一的任務ID(任務ID可以從任務列表獲取),當本任務執行結束並且執行成功時,將會觸發子任務ID所對應的任務的一次主動調度。
任務超時時間:支持自定義任務超時時間,任務運行超時將會主動中斷任務;
失敗重試次數;支持自定義任務失敗重試次數,當任務失敗時將會按照預設的失敗重試次數主動進行重試;
報警郵件:任務調度失敗時郵件通知的郵箱地址,支持配置多郵箱地址,配置多個郵箱地址時用逗號分隔;
負責人:任務的負責人;
執行參數:任務執行所需的參數,多個參數時用逗號分隔,任務執行時將會把多個參數轉換成數組傳入;
編寫執行器:就是我們的業務代碼(項目)
- 配置文件:
# web port
server.port=8081
# no web
#spring.main.web-environment=false
# log config
logging.config=classpath:logback.xml
### xxl-job admin address list, such as "http://address" or "http://address01,http://address02"
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin
### xxl-job, access token
xxl.job.accessToken=
### xxl-job executor appname
xxl.job.executor.appname=xxl-job-executor-sample
### xxl-job executor registry-address: default use address to registry , otherwise use ip:port if address is null
xxl.job.executor.address=
### xxl-job executor server-info
xxl.job.executor.ip=
xxl.job.executor.port=9999
### xxl-job executor log-path
xxl.job.executor.logpath=/data/applogs/xxl-job/jobhandler
### xxl-job executor log-retention-days
xxl.job.executor.logretentiondays=30
- XxlJobConfig
package com.xxl.job.executor.core.config;
import com.xxl.job.core.executor.impl.XxlJobSpringExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* xxl-job config
*
* @author xuxueli 2017-04-28
*/
@Configuration
public class XxlJobConfig {
private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);
@Value("${xxl.job.admin.addresses}")
private String adminAddresses;
@Value("${xxl.job.accessToken}")
private String accessToken;
@Value("${xxl.job.executor.appname}")
private String appname;
@Value("${xxl.job.executor.address}")
private String address;
@Value("${xxl.job.executor.ip}")
private String ip;
@Value("${xxl.job.executor.port}")
private int port;
@Value("${xxl.job.executor.logpath}")
private String logPath;
@Value("${xxl.job.executor.logretentiondays}")
private int logRetentionDays;
@Bean
public XxlJobSpringExecutor xxlJobExecutor() {
logger.info(">>>>>>>>>>> xxl-job config init.");
XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
xxlJobSpringExecutor.setAppname(appname);
xxlJobSpringExecutor.setAddress(address);
xxlJobSpringExecutor.setIp(ip);
xxlJobSpringExecutor.setPort(port);
xxlJobSpringExecutor.setAccessToken(accessToken);
xxlJobSpringExecutor.setLogPath(logPath);
xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);
return xxlJobSpringExecutor;
}
/**
* 針對多網卡、容器內部署等情況,可藉助 "spring-cloud-commons" 提供的 "InetUtils" 組件靈活定製註冊IP;
*
* 1、引入依賴:
* <dependency>
* <groupId>org.springframework.cloud</groupId>
* <artifactId>spring-cloud-commons</artifactId>
* <version>${version}</version>
* </dependency>
*
* 2、配置文件,或者容器啓動變量
* spring.cloud.inetutils.preferred-networks: 'xxx.xxx.xxx.'
*
* 3、獲取IP
* String ip_ = inetUtils.findFirstNonLoopbackHostInfo().getIpAddress();
*/
}
- JobHandler
package com.xxl.job.executor.service.jobhandler;
import com.xxl.job.core.biz.model.ReturnT;
import com.xxl.job.core.handler.IJobHandler;
import com.xxl.job.core.handler.annotation.XxlJob;
import com.xxl.job.core.log.XxlJobLogger;
import com.xxl.job.core.util.ShardingUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import java.io.BufferedInputStream;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Arrays;
import java.util.concurrent.TimeUnit;
/**
* XxlJob開發示例(Bean模式)
*
* 開發步驟:
* 1、在Spring Bean實例中,開發Job方法,方式格式要求爲 "public ReturnT<String> execute(String param)"
* 2、爲Job方法添加註解 "@XxlJob(value="自定義jobhandler名稱", init = "JobHandler初始化方法", destroy = "JobHandler銷燬方法")",註解value值對應的是調度中心新建任務的JobHandler屬性的值。
* 3、執行日誌:需要通過 "XxlJobLogger.log" 打印執行日誌;
*
* @author xuxueli 2019-12-11 21:52:51
*/
@Component
public class SampleXxlJob {
private static Logger logger = LoggerFactory.getLogger(SampleXxlJob.class);
/**
* 1、簡單任務示例(Bean模式)
*/
@XxlJob("demoJobHandler")
public ReturnT<String> demoJobHandler(String param) throws Exception {
XxlJobLogger.log("XXL-JOB, Hello World.");
for (int i = 0; i < 5; i++) {
XxlJobLogger.log("beat at:" + i);
TimeUnit.SECONDS.sleep(2);
}
return ReturnT.SUCCESS;
}
/**
* 2、分片廣播任務
*/
@XxlJob("shardingJobHandler")
public ReturnT<String> shardingJobHandler(String param) throws Exception {
// 分片參數
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
XxlJobLogger.log("分片參數:當前分片序號 = {}, 總分片數 = {}", shardingVO.getIndex(), shardingVO.getTotal());
// 業務邏輯
for (int i = 0; i < shardingVO.getTotal(); i++) {
if (i == shardingVO.getIndex()) {
XxlJobLogger.log("第 {} 片, 命中分片開始處理", i);
} else {
XxlJobLogger.log("第 {} 片, 忽略", i);
}
}
return ReturnT.SUCCESS;
}
/**
* 3、命令行任務
*/
@XxlJob("commandJobHandler")
public ReturnT<String> commandJobHandler(String param) throws Exception {
String command = param;
int exitValue = -1;
BufferedReader bufferedReader = null;
try {
// command process
Process process = Runtime.getRuntime().exec(command);
BufferedInputStream bufferedInputStream = new BufferedInputStream(process.getInputStream());
bufferedReader = new BufferedReader(new InputStreamReader(bufferedInputStream));
// command log
String line;
while ((line = bufferedReader.readLine()) != null) {
XxlJobLogger.log(line);
}
// command exit
process.waitFor();
exitValue = process.exitValue();
} catch (Exception e) {
XxlJobLogger.log(e);
} finally {
if (bufferedReader != null) {
bufferedReader.close();
}
}
if (exitValue == 0) {
return IJobHandler.SUCCESS;
} else {
return new ReturnT<String>(IJobHandler.FAIL.getCode(), "command exit value("+exitValue+") is failed");
}
}
/**
* 4、跨平臺Http任務
* 參數示例:
* "url: http://www.baidu.com\n" +
* "method: get\n" +
* "data: content\n";
*/
@XxlJob("httpJobHandler")
public ReturnT<String> httpJobHandler(String param) throws Exception {
// param parse
if (param==null || param.trim().length()==0) {
XxlJobLogger.log("param["+ param +"] invalid.");
return ReturnT.FAIL;
}
String[] httpParams = param.split("\n");
String url = null;
String method = null;
String data = null;
for (String httpParam: httpParams) {
if (httpParam.startsWith("url:")) {
url = httpParam.substring(httpParam.indexOf("url:") + 4).trim();
}
if (httpParam.startsWith("method:")) {
method = httpParam.substring(httpParam.indexOf("method:") + 7).trim().toUpperCase();
}
if (httpParam.startsWith("data:")) {
data = httpParam.substring(httpParam.indexOf("data:") + 5).trim();
}
}
// param valid
if (url==null || url.trim().length()==0) {
XxlJobLogger.log("url["+ url +"] invalid.");
return ReturnT.FAIL;
}
if (method==null || !Arrays.asList("GET", "POST").contains(method)) {
XxlJobLogger.log("method["+ method +"] invalid.");
return ReturnT.FAIL;
}
// request
HttpURLConnection connection = null;
BufferedReader bufferedReader = null;
try {
// connection
URL realUrl = new URL(url);
connection = (HttpURLConnection) realUrl.openConnection();
// connection setting
connection.setRequestMethod(method);
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setUseCaches(false);
connection.setReadTimeout(5 * 1000);
connection.setConnectTimeout(3 * 1000);
connection.setRequestProperty("connection", "Keep-Alive");
connection.setRequestProperty("Content-Type", "application/json;charset=UTF-8");
connection.setRequestProperty("Accept-Charset", "application/json;charset=UTF-8");
// do connection
connection.connect();
// data
if (data!=null && data.trim().length()>0) {
DataOutputStream dataOutputStream = new DataOutputStream(connection.getOutputStream());
dataOutputStream.write(data.getBytes("UTF-8"));
dataOutputStream.flush();
dataOutputStream.close();
}
// valid StatusCode
int statusCode = connection.getResponseCode();
if (statusCode != 200) {
throw new RuntimeException("Http Request StatusCode(" + statusCode + ") Invalid.");
}
// result
bufferedReader = new BufferedReader(new InputStreamReader(connection.getInputStream(), "UTF-8"));
StringBuilder result = new StringBuilder();
String line;
while ((line = bufferedReader.readLine()) != null) {
result.append(line);
}
String responseMsg = result.toString();
XxlJobLogger.log(responseMsg);
return ReturnT.SUCCESS;
} catch (Exception e) {
XxlJobLogger.log(e);
return ReturnT.FAIL;
} finally {
try {
if (bufferedReader != null) {
bufferedReader.close();
}
if (connection != null) {
connection.disconnect();
}
} catch (Exception e2) {
XxlJobLogger.log(e2);
}
}
}
/**
* 5、生命週期任務示例:任務初始化與銷燬時,支持自定義相關邏輯;
*/
@XxlJob(value = "demoJobHandler2", init = "init", destroy = "destroy")
public ReturnT<String> demoJobHandler2(String param) throws Exception {
XxlJobLogger.log("XXL-JOB, Hello World.");
return ReturnT.SUCCESS;
}
public void init(){
logger.info("init");
}
public void destroy(){
logger.info("destory");
}
}
- 啓動類:
package com.xxl.job.executor;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
/**
* @author xuxueli 2018-10-28 00:38:13
*/
@SpringBootApplication
public class XxlJobExecutorApplication {
public static void main(String[] args) {
SpringApplication.run(XxlJobExecutorApplication.class, args);
}
}
- logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false" scan="true" scanPeriod="1 seconds">
<contextName>logback</contextName>
<property name="log.path" value="/data/applogs/xxl-job/xxl-job-executor-sample-springboot.log"/>
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} %contextName [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<appender name="file" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}.%d{yyyy-MM-dd}.zip</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>%date %level [%thread] %logger{36} [%file : %line] %msg%n
</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</root>
</configuration>
- 測試:
- 啓動我們的應用: