這兩天用了一下Spring Batch來導數據,感覺是個很強大的工具。網上的資料很多,沒看到連續運行的示例。從mkong和spring官網上搞了幾段代碼,在本地跑了一下發現幾個問題:
1. initialize-database執行官方的drop table失敗
報權限問題,檢查了一下,無法就是清理spring batch任務表中的數據
DROP TABLE BATCH_STEP_EXECUTION_CONTEXT ;
DROP TABLE BATCH_JOB_EXECUTION_CONTEXT ;
DROP TABLE BATCH_STEP_EXECUTION ;
DROP TABLE BATCH_JOB_EXECUTION_PARAMS ;
DROP TABLE BATCH_JOB_EXECUTION ;
DROP TABLE BATCH_JOB_INSTANCE ;
DROP TABLE BATCH_STEP_EXECUTION_SEQ ;
DROP TABLE BATCH_JOB_EXECUTION_SEQ ;
DROP TABLE BATCH_JOB_SEQ ;
其實多次執行的時候不用drop這個表,所以將這塊去掉了,改成了由
<jdbc:script location="classpath:init.sql" />
調用一段本地的腳本
2. 連續運行job失敗
這個是傳參的問題,每次給JobParameters設置不同的參數才能順利的連續運行。
HashMap<String, JobParameter> parameters = new HashMap<String, JobParameter>();
parameters.put("currentTime", new JobParameter(System.currentTimeMillis()));
JobExecution execution = jobLauncher.run(job, new JobParameters(parameters));
加了個當前時間,搞定3.第二次執行導入的時候報主鍵錯誤
因爲每次都是完全拷貝,沒做檢查,所以在插入前有必要清理所有的內容。可是程序在多次運行(注意,是連續運行)的情況下,只會運行一次initialize-database的內容。
翻遍了例子所有的step都要有reader和writer。於是手動寫了一個Reader和Writer來跑清理的過程。注意還要寫一個Listener,不然會一直執行下去,這個是用來讓計數器歸位的。
job的配置
<batch:tasklet transaction-manager="transactionManagerJob">
<batch:chunk reader="deleteReader" writer="deleteWriter"
commit-interval="1">
</batch:chunk>
<batch:listeners>
<batch:listener ref="resettingListener"/>
</batch:listeners>
</batch:tasklet>
reader的配置
<beans:bean id="deleteReader" class="xxx.InitReader">
<property name="limit" value="1"/>
</beans:bean>
writer的配置
<beans:bean id="deleteWriter" class="xxxx.InitWriter">
<property name="dataSource" ref="targetDataSource" />
<property name="sql"
value="
DELETE FROM XXX;
DELETE FROM YYY;
" />
</beans:bean>
listener的配置
<beans:bean id="resettingListener" class="xxx.InitResettingListener">
<beans:property name="reader" ref="deleteReader" />
</beans:bean>
reader的代碼
public class InitReader extends ItemReaderAdapter<Object> {
private int limit = 1;
private int counter = 0;
public Object read() throws Exception, UnexpectedInputException, ParseException {
if(counter<limit){
counter++;
return new Object();
}
return null;
}
/**
* @param limit number of items that will be generated
* (null returned on consecutive calls).
*/
public void setLimit(int limit) {
this.limit = limit;
}
public int getCounter() {
return counter;
}
public int getLimit() {
return limit;
}
public void resetCounter()
{
this.counter = 0;
}
public void afterPropertiesSet() throws Exception {
Assert.notNull(limit, "limit must be set");
}
}
writer的代碼
public class InitWriter extends ItemWriterAdapter<Object>{
private DriverManagerDataSource dataSource;
private String sql;
public DriverManagerDataSource getDataSource() {
return dataSource;
}
public void setDataSource(DriverManagerDataSource dataSource) {
this.dataSource = dataSource;
}
public String getSql() {
return sql;
}
public void setSql(String sql) {
this.sql = sql;
}
public void write(List<? extends Object> items) throws Exception {
//System.out.println("init write delete");
Connection connection = dataSource.getConnection();
PreparedStatement preparedStatement=connection.prepareStatement(sql);
preparedStatement.executeUpdate();
System.out.println("init delete data complete.");
}
public void afterPropertiesSet() throws Exception {
Assert.notNull(dataSource, "dataSource limit must be set");
Assert.notNull(sql, "sql limit must be set");
}
}
listener的代碼
public class InitResettingListener extends StepExecutionListenerSupport implements InitializingBean {
private InitReader reader;
public ExitStatus afterStep(StepExecution stepExecution) {
this.reader.resetCounter();
return null;
}
public void setReader(InitReader reader) {
this.reader = reader;
}
public void afterPropertiesSet() throws Exception {
Assert.notNull(this.reader, "The 'reader' must be set.");
}
}
搞定!!!