最近項目遇到了同一方法內,主數據庫操作數據後,需往其他數據源同步數據的情景,在此記錄一下實現過程,也參照了下其他大牛的代碼
主要有兩種實現方式
- 通過主動方式切換數據源
- 直接獲取JdbcTemplate
參考文章:
SpringBoot多數據源切換詳解,以及開啓事務後數據源切換失敗處理
springboot+mybatis解決多數據源切換事務控制不生效的問題
實現流程
一、禁用數據庫自動配置
禁用數據庫自動配置需在Application類上增加配置,可在@SpringBootApplication
註解後,也可在@EnableAutoConfiguration
註解後配置。
@SpringBootApplication(exclude={DataSourceAutoConfiguration.class})
或
@EnableAutoConfiguration(exclude={DataSourceAutoConfiguration.class})
有時也需要屏蔽如下類:
DataSourceTransactionManagerAutoConfiguration.class
JdbcTemplateAutoConfiguration.class
HibernateJpaAutoConfiguration.class
二、主動切換數據源方式
2.1 配置數據源類型
通常採用常量或者枚舉類型
public enum DBType {
one("one"),
two("two");
private String value;
DBType(String value) {
this.value = value;
}
public String getValue() {
return value;
}
}
2.2 配置數據源切換上下文
public class DBContextHolder {
private static final ThreadLocal contextHolder = new ThreadLocal<>();
/**
* 設置數據源
* @param DBType
*/
public static void setDbType(DBType dbType) {
contextHolder.set(dbType.getValue());
}
/**
* 取得當前數據源
* @return
*/
public static String getDbType() {
return (String) contextHolder.get();
}
/**
* 清除上下文數據
*/
public static void clearDbType() {
contextHolder.remove();
}
}
2.3 配置動態切換數據源類
需要繼承AbstractRoutingDataSource
類,並重寫determineCurrentLookupKey()
方法,從數據源類型中獲取當前線程的數據源類型。
public class DynamicDataSource extends AbstractRoutingDataSource {
@Override
protected Object determineCurrentLookupKey() {
return DBContextHolder.getDbType();
}
}
2.4 重寫Transaction
當我們配置了事物管理器和攔截Service中的方法後,每次執行Service中方法前會開啓一個事務,並且同時會緩存一些東西:DataSource、SqlSessionFactory、Connection等,所以,我們在外面再怎麼設置要求切換數據源也沒用,因爲Conneciton都是從緩存中拿的,所以我們要想能夠順利的切換數據源,實際就是能夠動態的根據DatabaseType獲取不同的Connection,並且要求不能影響整個事物的特性。
主要包含兩個類:
import com.alibaba.druid.support.logging.Log;
import com.alibaba.druid.support.logging.LogFactory;
import org.apache.ibatis.transaction.Transaction;
import org.springframework.jdbc.CannotGetJdbcConnectionException;
import org.springframework.jdbc.datasource.DataSourceUtils;
import javax.sql.DataSource;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import static org.apache.commons.lang3.Validate.notNull;
/**
* <P>多數據源切換,支持事務</P>
*
* @author 高仕立
* @date 2018/2/6 9:09
* @since
*/
public class MultiDataSourceTransaction implements Transaction{
private static final Log LOGGER = LogFactory.getLog(MultiDataSourceTransaction.class);
private final DataSource dataSource;
private Connection mainConnection;
private String mainDatabaseIdentification;
private ConcurrentMap<String, Connection> otherConnectionMap;
private boolean isConnectionTransactional;
private boolean autoCommit;
public MultiDataSourceTransaction(DataSource dataSource) {
notNull(dataSource, "No DataSource specified");
this.dataSource = dataSource;
otherConnectionMap = new ConcurrentHashMap<>();
mainDatabaseIdentification=DBContextHolder.getDbType();
}
/**
* {@inheritDoc}
*/
@Override
public Connection getConnection() throws SQLException {
String databaseIdentification = DBContextHolder.getDbType();
if (databaseIdentification.equals(mainDatabaseIdentification)) {
if (mainConnection != null) return mainConnection;
else {
openMainConnection();
mainDatabaseIdentification =databaseIdentification;
return mainConnection;
}
} else {
if (!otherConnectionMap.containsKey(databaseIdentification)) {
try {
Connection conn = dataSource.getConnection();
otherConnectionMap.put(databaseIdentification, conn);
} catch (SQLException ex) {
throw new CannotGetJdbcConnectionException("Could not get JDBC Connection", ex);
}
}
return otherConnectionMap.get(databaseIdentification);
}
}
private void openMainConnection() throws SQLException {
this.mainConnection = DataSourceUtils.getConnection(this.dataSource);
this.autoCommit = this.mainConnection.getAutoCommit();
this.isConnectionTransactional = DataSourceUtils.isConnectionTransactional(this.mainConnection, this.dataSource);
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
"JDBC Connection ["
+ this.mainConnection
+ "] will"
+ (this.isConnectionTransactional ? " " : " not ")
+ "be managed by Spring");
}
}
/**
* {@inheritDoc}
*/
@Override
public void commit() throws SQLException {
if (this.mainConnection != null && !this.isConnectionTransactional && !this.autoCommit) {
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("Committing JDBC Connection [" + this.mainConnection + "]");
}
this.mainConnection.commit();
for (Connection connection : otherConnectionMap.values()) {
connection.commit();
}
}
}
/**
* {@inheritDoc}
*/
@Override
public void rollback() throws SQLException {
if (this.mainConnection != null && !this.isConnectionTransactional && !this.autoCommit) {
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("Rolling back JDBC Connection [" + this.mainConnection + "]");
}
this.mainConnection.rollback();
for (Connection connection : otherConnectionMap.values()) {
connection.rollback();
}
}
}
/**
* {@inheritDoc}
*/
@Override
public void close() throws SQLException {
DataSourceUtils.releaseConnection(this.mainConnection, this.dataSource);
for (Connection connection : otherConnectionMap.values()) {
DataSourceUtils.releaseConnection(connection, this.dataSource);
}
}
@Override
public Integer getTimeout() throws SQLException {
return null;
}
}
import org.apache.ibatis.session.TransactionIsolationLevel;
import org.apache.ibatis.transaction.Transaction;
import org.mybatis.spring.transaction.SpringManagedTransactionFactory;
import javax.sql.DataSource;
/**
* <P>支持Service內多數據源切換的Factory</P>
*
* @author 高仕立
* @date 2018/2/6 9:18
* @since
*/
public class MultiDataSourceTransactionFactory extends SpringManagedTransactionFactory {
@Override
public Transaction newTransaction(DataSource dataSource, TransactionIsolationLevel level, boolean autoCommit) {
DBContextHolder.setDbType(DBType.one);
return new MultiDataSourceTransaction(dataSource);
}
}
2.5 使用AOP或Interceptor配置默認數據源
採用攔截器舉例:
import org.springframework.stereotype.Component;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@Component
public class DBInterceptor implements HandlerInterceptor {
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
// 設置默認數據源
DBContextHolder.setDbType(DBType.one);
return true;
}
@Override
public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {
}
@Override
public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
// 清除數據源
DBContextHolder.clearDbType();
}
}
三、配置數據源
根據配置文件決定使用哪個數據源配置生效
@EnableTransactionManagement
@ConditionalOnProperty(value = "run.datasource.config", havingValue = "false")
@Configuration
public class DataSourceConfig {
@Resource
private Environment env;
//廠商平臺配置數據庫
@Bean(name = "one")
public DataSource one() {
AtomikosDataSourceBean ds = new AtomikosDataSourceBean();
ds.setXaDataSourceClassName("com.alibaba.druid.pool.xa.DruidXADataSource");
ds.setUniqueResourceName("one");
ds.setPoolSize(5);
ds.setXaProperties(build("spring.datasource.druid.one."));
return ds;
}
@Bean(name = "two")
public DataSource two() {
AtomikosDataSourceBean ds = new AtomikosDataSourceBean();
ds.setXaDataSourceClassName("com.alibaba.druid.pool.xa.DruidXADataSource");
ds.setUniqueResourceName("two");
ds.setPoolSize(5);
ds.setXaProperties(build("spring.datasource.druid.two."));
return ds;
}
/**
* 動態數據源配置
* @return
*/
@Bean
@Primary
public DataSource multipleDataSource(@Qualifier("one") DataSource one,
@Qualifier("two") DataSource two) {
DynamicDataSource dynamicDataSource = new DynamicDataSource();
Map<Object, Object> targetDataSources = new HashMap<>();
targetDataSources.put(DBType.one.getValue(), one);
targetDataSources.put(DBType.two.getValue(), two);
dynamicDataSource.setTargetDataSources(targetDataSources);
dynamicDataSource.setDefaultTargetDataSource(one);
return dynamicDataSource;
}
@Bean("sqlSessionFactory")
public SqlSessionFactory sqlSessionFactory() throws Exception {
SqlSessionFactoryBean sqlSessionFactory = new SqlSessionFactoryBean();
sqlSessionFactory.setDataSource(multipleDataSource(one(),two()));
// 使用自定義的多數據源事務工廠,如採用JdbcTemplate方式可不配置
sqlSessionFactory.setTransactionFactory(new MultiDataSourceTransactionFactory());
//添加XML目錄
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
sqlSessionFactory.setConfigLocation(resolver.getResource("classpath:mybatis-config.xml"));
sqlSessionFactory.setMapperLocations(resolver.getResources("classpath*:mapper/*.xml"));
return sqlSessionFactory.getObject();
}
// 此處是初始化JdbcTemplate,可直接獲取到數據源連接
@Bean(name = "jdbc_two")
public JdbcTemplate secondJdbcTemplate(
@Qualifier("two") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
private Properties build(String prefix) {
Properties prop = new Properties();
prop.put("url", env.getProperty(prefix + "url"));
prop.put("username", env.getProperty(prefix + "username"));
prop.put("password", env.getProperty(prefix + "password"));
prop.put("driverClassName", env.getProperty(prefix + "driverClassName", ""));
prop.put("initialSize", env.getProperty("spring.datasource.druid.initialSize", Integer.class));
prop.put("minIdle", env.getProperty("spring.datasource.druid.minIdle", Integer.class));
prop.put("maxActive", env.getProperty("spring.datasource.druid.maxActive", Integer.class));
prop.put("maxWait", env.getProperty("spring.datasource.druid.maxWait", Integer.class));
prop.put("timeBetweenEvictionRunsMillis", env.getProperty("spring.datasource.druid.timeBetweenEvictionRunsMillis", Integer.class));
prop.put("minEvictableIdleTimeMillis", env.getProperty("spring.datasource.druid.minEvictableIdleTimeMillis", Integer.class));
prop.put("validationQuery", env.getProperty("spring.datasource.druid.validationQuery"));
prop.put("testWhileIdle", env.getProperty("spring.datasource.druid.testWhileIdle", Boolean.class));
prop.put("testOnBorrow", env.getProperty("spring.datasource.druid.testOnBorrow", Boolean.class));
prop.put("testOnReturn", env.getProperty("spring.datasource.druid.testOnReturn", Boolean.class));
prop.put("filters", env.getProperty("spring.datasource.druid.filters"));
return prop;
}
}
四、service方法內切換數據源
方法上需要配置事務@Transactional(propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
4.1 主動切換方式
根據配置文件,獲取所有待操作的數據源,然後調用DBContextHolder
的切換數據源方法,由於其餘數據庫的表結構都是一致的,所以調用同一方法操作各個數據源。
List<DBType> dbs = DbUtils.getAllDBType();
for (DBType db : dbs){
DBContextHolder.setDbType(db);
mapper.saveAnother(entity);
}
4.2 JdbcTemplate方式
直接獲取初始化的JdbcTemplate
集合,逐個進行操作。
List<JdbcTemplate> jdbcTemplateList = JdbcTemplateUtils.getJdbcTemplates();
SqlContext sqlContext = SQLTemplate.createSql(entity);
for (JdbcTemplate jdbc : jdbcTemplateList){
jdbc.update(sqlContext.getSql(), sqlContext.getParams());
}
其中的JdbcTemplate
可以通過如下方式獲取:
SpringContextHolder.getBean("jdbc_two");
五、其他想法
一般情況下,不要直接操作跨項目的數據庫,最好讓其他項目(暫且叫做客戶端)暴露接口,通過遠程調用的方式通知其他客戶端有數據變動,然後各個項目根據數據變動進行相應的操作,並返回相應的操作結果給服務端。同時服務端提供相應的查詢結果,供客戶端定時進行數據的對比,防止有遺漏的數據變動。