springboot簡單的分表插件

業務上做了水平分表，公司基礎架構沒有提供分表中間件，開源的中間件用起來門檻高，有較大的學習成本。自己基於springboot+mybatis+jsqlparser實現了一個簡單的分表插件。
籠統來說就是攔截SQL，分析SQL，替換tableName，返回重寫後的SQL給Mybatis，支持簡單的CRUD分表,支持join分表。
項目裏引入maven依賴：

 		<dependency>
                <groupId>org.mybatis.spring.boot</groupId>
                <artifactId>mybatis-spring-boot-starter</artifactId>
                <version>x.x.x</version>
        </dependency>
		<dependency>
            <groupId>com.github.jsqlparser</groupId>
            <artifactId>jsqlparser</artifactId>
            <version>x.x.x</version>
        </dependency>

1.分表註解

首先需要一個註解，用來標識需要分表的類或方法。

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
public @interface TableSplit {

    //是否分表
    boolean split() default true;

    //表名 該字段暫未使用。使用註解會對SQL中所有的表名進行替換
    String table() default "";

    //分表字段 
    String field() default "";

    //分表策略
    String strategy() default "";

}

2.分表策略

用策略模式設計一個分表的策略，在後續的mybatis攔截器中使用

2.1策略接口

public interface Strategy {

    String TABLE_NAME = "table_name";
    String SPLIT_FIELD = "split_field";
    String EXECUTE_PARAM_DECLARE = "execute_param_declare";
    String EXECUTE_PARAM_VALUES = "execute_param_values";

    /**
     * convert sql
     * @param params
     * @return
     * @throws Exception
     */
    String convert(String sql, Map<String, Object> params) throws Exception;

    /**
     * 
     * @param params
     * @return
     * @throws Exception
     */
    Integer getTableIndex(Map<String, Object> params) throws Exception;
}

2.2 分表策略管理類

管理類維護一個map，可以通過key獲取不同的分表策略。

@Slf4j
public class StrategyManager {

    private Map<String, Strategy> strategies = new ConcurrentHashMap<>(10);

    public Strategy getStrategy(String key) {
        return strategies.get(key);
    }

    public Map<String, Strategy> getStrategies() {
        return strategies;
    }

    public void setStrategies(Map<String, String> strategies) {
        for (Map.Entry<String, String> entry : strategies.entrySet()) {
            try {
                this.strategies.put(entry.getKey(), (Strategy) Class.forName(entry.getValue()).newInstance());
            } catch (Exception e) {
                log.error("實例化策略出錯", e);
            }
        }
    }
}

2.3 分表策略的實現

分表策略的具體實現可以根據實際場景和業務自行實現，比如下面這實現類就是我在項目中使用的，用userId字段對8取模得到表的下標值。

public class RemainderStrategy implements Strategy {

    /**
     * 取模基數
     */
    private static final int DIVIDER = 8;


    @Override
    public String convert(String sql, Map<String, Object> params) throws Exception {
        MySqlParserFactory sqlParserFactory = new MySqlParserFactory();
        Integer reminder = this.getIndex(params);
        return sqlParserFactory.parser(sql, reminder);
    }

    @Override
    public Integer getTableIndex(Map<String, Object> params) throws Exception {
        Integer reminder = this.getIndex(params);
        return reminder;
    }

    private Integer getIndex(Map<String, Object> params) throws IllegalAccessException, InvocationTargetException, NoSuchMethodException {
        String field = (String) params.get(Strategy.SPLIT_FIELD);
        Object paramValue = params.get(Strategy.EXECUTE_PARAM_VALUES);
        String value;
        if (paramValue instanceof Map) {
            Map map = (Map) paramValue;
            value = String.valueOf(map.get(field));
        } else {
            value = BeanUtils.getProperty(paramValue, field);
        }
        Integer reminder = Integer.valueOf(value) % DIVIDER;
        return reminder;
    }

2.4 把分表策略註冊到策略管理類

我是用xml的方式，當然你也可以@Bean的方式。

@Configuration
@ImportResource(locations = {"classpath:application-bean.xml"})
public class BeanConfig {

}

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
    <!-- 配置分表策略 -->
    <bean id="strategyManager" class="xxx.xxx.xxx.StrategyManager">
        <property name="strategies">
            <map>
                <entry key="remainder" value="xxx.xxx.xxx.impl.RemainderStrategy"/>
            </map>
        </property>
    </bean>
</beans>

4.定義一個ContextHelper類

實現ApplicationContextAware接口，主要用來獲取策略管理類的bean，因爲我這邊沒有用IOC。

@Component
public class ContextHelper implements ApplicationContextAware {
    private static ApplicationContext context;

    @Override
    public void setApplicationContext(ApplicationContext context)
            throws BeansException {
        ContextHelper.context = context;
    }

    public static ApplicationContext getApplicationContext() {
        return ContextHelper.context;
    }

    public static <T> T getBean(Class<T> clazz) {
        return context.getBean(clazz);
    }

    public static Object getBean(String name) {
        return context.getBean(name);
    }

    public static <T> T getBean(String name, Class<T> clazz) {
        return context.getBean(name, clazz);
    }
}

4.基於jsqlparser重寫SQL

分表策略實現類的作用就是返回重寫後的SQL，這塊功能主要是基於jsqlparser實現的。

4.1 MySqlParserFactory 工廠類

編寫一個工廠類，用來封裝解析SQL的職責。

import net.sf.jsqlparser.JSQLParserException;
import net.sf.jsqlparser.parser.CCJSqlParserManager;
import net.sf.jsqlparser.parser.CCJSqlParserUtil;
import net.sf.jsqlparser.schema.Table;
import net.sf.jsqlparser.statement.Statement;
import net.sf.jsqlparser.statement.delete.Delete;
import net.sf.jsqlparser.statement.insert.Insert;
import net.sf.jsqlparser.statement.select.Select;
import net.sf.jsqlparser.statement.update.Update;
import net.sf.jsqlparser.util.deparser.ExpressionDeParser;
import net.sf.jsqlparser.util.deparser.SelectDeParser;

import java.io.StringReader;
public class MySqlParserFactory {

    public String parser(String sql, Integer reminder) throws Exception {
        CCJSqlParserManager parser = new CCJSqlParserManager();
        Statement stmt = parser.parse(new StringReader(sql));
        if (stmt instanceof Select) {
            return selectSqlParser(sql, reminder);
        }
        if (stmt instanceof Update) {
            return updateSqlParser(stmt, reminder);
        }
        if (stmt instanceof Insert) {
            return insertSqlParser(stmt, reminder);
        }
        if (stmt instanceof Delete) {
            return deleteSqlParser(stmt, reminder);
        }
        return StringUtils.EMPTY;
    }

    private String deleteSqlParser(Statement stmt, Integer reminder) {
        Delete statement = (Delete) stmt;
        Table t = statement.getTable();
        t.setName(t.getName() + reminder);
        statement.setTable(t);
        return statement.toString();
    }

    private String insertSqlParser(Statement stmt, Integer reminder) {
        Insert statement = (Insert) stmt;
        Table t = statement.getTable();
        t.setName(t.getName() + reminder);
        statement.setTable(t);
        return statement.toString();
    }


    private String updateSqlParser(Statement stmt, Integer reminder) {
        Update statement = (Update) stmt;
        Table t = statement.getTable();
        t.setName(t.getName() + reminder);
        statement.setTable(t);
        return statement.toString();
    }

    private String selectSqlParser(String sql, Integer reminder) throws JSQLParserException {
        Select select = (Select) CCJSqlParserUtil.parse(sql);
        //Start of value modification
        StringBuilder buffer = new StringBuilder();
        ExpressionDeParser expressionDeParser = new ExpressionDeParser();
        SelectDeParser deParser = new MySelectDeParser(reminder, expressionDeParser, buffer);
        expressionDeParser.setSelectVisitor(deParser);
        expressionDeParser.setBuffer(buffer);
        select.getSelectBody().accept(deParser);
        return buffer.toString();
    }

4.2 MySelectDeParser

select 操作比較複雜，使用下面的類做處理。

import net.sf.jsqlparser.expression.Alias;
import net.sf.jsqlparser.expression.ExpressionVisitor;
import net.sf.jsqlparser.schema.Table;
import net.sf.jsqlparser.statement.select.Pivot;
import net.sf.jsqlparser.util.deparser.SelectDeParser;

public class MySelectDeParser extends SelectDeParser {

    private Integer reminder;

    public MySelectDeParser(Integer reminder, ExpressionVisitor expressionVisitor, StringBuilder buffer) {
        super(expressionVisitor, buffer);
        this.reminder = reminder;
    }

    @Override
    public void visit(Table table) {
        String tableName = table.getName();
        table.setName(tableName + reminder.toString());
        StringBuilder buffer = getBuffer();
        buffer.append(table.getFullyQualifiedName());
        Pivot pivot = table.getPivot();
        if (pivot != null) {
            pivot.accept(this);
        }
        Alias alias = table.getAlias();
        if (alias != null) {
            buffer.append(alias);
        }
    }
}

該工廠類只是簡單的使用jsqlparser對CRUD的SQL做了替換。更多jsqlparser的功能需要自己探索了。

5.mybatis攔截器

編寫一個類實現mybatis的Interceptor接口，用來攔截mybatis發送給jdbc的sql。

import org.apache.ibatis.executor.statement.StatementHandler;
import org.apache.ibatis.mapping.BoundSql;
import org.apache.ibatis.mapping.MappedStatement;
import org.apache.ibatis.mapping.ParameterMap;
import org.apache.ibatis.plugin.*;
import org.apache.ibatis.reflection.DefaultReflectorFactory;
import org.apache.ibatis.reflection.MetaObject;
import org.apache.ibatis.reflection.ReflectorFactory;
import org.apache.ibatis.reflection.factory.DefaultObjectFactory;
import org.apache.ibatis.reflection.factory.ObjectFactory;
import org.apache.ibatis.reflection.wrapper.DefaultObjectWrapperFactory;
import org.apache.ibatis.reflection.wrapper.ObjectWrapperFactory;

import java.lang.reflect.Method;
import java.sql.Connection;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;

@Intercepts({@Signature(type = StatementHandler.class, method = "prepare", args = {Connection.class, Integer.class})})
public class TableSplitInterceptor implements Interceptor {

    private static final ObjectFactory DEFAULT_OBJECT_FACTORY = new DefaultObjectFactory();
    private static final ObjectWrapperFactory DEFAULT_OBJECT_WRAPPER_FACTORY = new DefaultObjectWrapperFactory();
    private static final ReflectorFactory DEFAULT_REFLECTOR_FACTORY = new DefaultReflectorFactory();

    @Override
    public Object intercept(Invocation invocation) throws Throwable {
        StatementHandler statementHandler = (StatementHandler) invocation.getTarget();
        MetaObject metaStatementHandler = MetaObject.forObject(statementHandler, DEFAULT_OBJECT_FACTORY, DEFAULT_OBJECT_WRAPPER_FACTORY, DEFAULT_REFLECTOR_FACTORY);

        BoundSql boundSql = (BoundSql) metaStatementHandler.getValue("delegate.boundSql");
        Object parameterObject = metaStatementHandler.getValue("delegate.boundSql.parameterObject");
        doSplitTable(metaStatementHandler, parameterObject);
        // 傳遞給下一個攔截器處理
        return invocation.proceed();
    }

    @Override
    public Object plugin(Object target) {
        // 當目標類是StatementHandler類型時，才包裝目標類，否者直接返回目標本身,減少目標被代理的次數
        if (target instanceof StatementHandler) {
            return Plugin.wrap(target, this);
        } else {
            return target;
        }
    }

    @Override
    public void setProperties(Properties properties) {

    }

    private void doSplitTable(MetaObject metaStatementHandler, Object param) throws Exception {
        String originalSql = (String) metaStatementHandler.getValue("delegate.boundSql.sql");
        if (originalSql != null && !originalSql.equals("")) {
            MappedStatement mappedStatement = (MappedStatement) metaStatementHandler.getValue("delegate.mappedStatement");
            String id = mappedStatement.getId();
            String className = id.substring(0, id.lastIndexOf("."));
            String methodName = id.substring(id.lastIndexOf(".") + 1);
            Class<?> clazz = Class.forName(className);
            ParameterMap paramMap = mappedStatement.getParameterMap();
            Method method = findMethod(clazz.getDeclaredMethods(), methodName);
            // 根據配置自動生成分表SQL
            TableSplit tableSplit = null;
            if (method != null) {
                tableSplit = method.getAnnotation(TableSplit.class);
            }
            if (tableSplit == null) {
                tableSplit = clazz.getAnnotation(TableSplit.class);
            }
            if (tableSplit != null && tableSplit.split() && StringUtils.isNotBlank(tableSplit.strategy())) {
                StrategyManager strategyManager = ContextHelper.getBean(StrategyManager.class);
                String convertedSql = "";
                Strategy strategy = strategyManager.getStrategy(tableSplit.strategy());
                Map<String, Object> params = getStringObjectMap(param, paramMap, tableSplit, tableSplit.table());
                convertedSql = strategy.convert(originalSql, params);
                metaStatementHandler.setValue("delegate.boundSql.sql", convertedSql);
            }
        }
    }

    private Map<String, Object> getStringObjectMap(Object param, ParameterMap paramMap, TableSplit tableSplit, String table) {
        Map<String, Object> params = new HashMap<>();
        params.put(Strategy.TABLE_NAME, table);
        params.put(Strategy.SPLIT_FIELD, tableSplit.field());
        params.put(Strategy.EXECUTE_PARAM_DECLARE, paramMap);
        params.put(Strategy.EXECUTE_PARAM_VALUES, param);
        return params;
    }

    private Method findMethod(Method[] methods, String methodName) {
        for (Method method : methods) {
            if (method.getName().equals(methodName)) {
                return method;
            }
        }
        return null;
    }
}

6.使用

代碼就是上面的這些。最後別忘了把自定義的攔截器註冊到mybatis的SqlSessionFactory中。

/**
     * sql會話工廠類
     *
     * @return 會話工廠
     * @throws Exception 異常
     */
    @Bean(name = "sqlSessionFactory")
    public SqlSessionFactory sqlSessionFactory() throws Exception {
        SqlSessionFactoryBean factoryBean = new SqlSessionFactoryBean();
        factoryBean.setDataSource(dataSource);
        factoryBean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath:mapper/*Mapper.xml"));
        org.apache.ibatis.session.Configuration configuration = new org.apache.ibatis.session.Configuration();
        configuration.setMapUnderscoreToCamelCase(true);
        factoryBean.setConfiguration(configuration);
        factoryBean.setPlugins(new Interceptor[]{new TableSplitInterceptor()});
        return factoryBean.getObject();
    }

6.1使用示例

在mapper中添加註解，指定分表策略的key和分表的分表字段即可。
如果你項目中使用統一分表策略和分表字段，那麼直接在TableSplit註解類中default指定默認值就好了。
TableSplit可以標識類和方法。方法上粒度更細。
攔截器優先獲取method上的註解。然後獲取Class上的註解，

@TableSplit(strategy = "key的名稱",field = "userId")
public interface TestMapper {
    int countByExample(@Param("userId") Integer userId, @Param("example") countByExample);
}

<select id="countByExample" parameterType="xxx.xxx.xxx"
            resultType="java.lang.Integer">
        select count(*) from test
    </select>

業務中直接使用Mapper即可，SQL會被TableSplitInterceptor 攔截器攔截，完成表名的替換。比如上面的SQL中 test 表名會被替換成int index = (userId%8);
test_index

以上就是自己搭建的一套簡單分表插件。業務中耦合還是比較小的，唯一耦合的地方就是需要指定分表字段，一定要在MyBatis的Mapper接口的方法上添加正確的分表參數。比如目前使用userId取模，則需要在接口方法額外添加
@Param(“userId”) Integer userId，或者 @Param(“example”) countByExample中一定需要有uesrId的字段。不然分表策略無法得到分表的下標值。

springboot簡單的分表插件

1.分表註解

2.分表策略

2.1策略接口

2.2 分表策略管理類

2.3 分表策略的實現

2.4 把分表策略註冊到策略管理類

4.定義一個ContextHelper類

4.基於jsqlparser重寫SQL

4.1 MySqlParserFactory 工廠類

4.2 MySelectDeParser

5.mybatis攔截器

6.使用

6.1使用示例

PDManer [元數建模]-v4.9.0 發佈：一款簡單好用的數據庫建模平臺

使用neovim打造go ide(支持代碼跳轉, 代碼補全, 實時語法檢查)

cs01 CSS Syntax

挑戰程序設計競賽 2.3章習題 poj 3046 Ant Counting

[MASM拾遺]Offset僞指令

h30 HTML Layout Elements

瞭解顯卡

一款基於C#開發的通訊調試工具（支持Modbus RTU、MQTT調試）

Linux/Golang/glibC系統調用

cs04 CSS Measurement Units

springboot優雅的使用線程池

ES 2.x查詢

springboot簡單的分表插件

基於spring-beans的深複製工具類

MySQL的視圖

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結