Mybatis+Oracle插入萬條數據

最近工作中使用到批量刪除，試了試網上的幾種方法，下面三種方法都是插入2萬條數據

使用oracle的insert all

特別注意：mysql默認接受sql的大小是1048576(1M)，若數據量超過1M會報如異常錯誤者可以，進行分開處理，每次提交一定的數據到數據庫，還可以通過調整MySQL安裝目錄下的my.ini文件中[mysqld]段的＂max_allowed_packet = 1M＂），增加max_allowed_packet 的容量。

mybatis的xml編寫

使用到oracle批量插入的方法insert all

    <insert id="insertBatch3" useGeneratedKeys="false" parameterType="list">
        INSERT ALL
        <foreach collection ="list" item="applicationConsumables" index="index" separator =" ">
            INTO APPLICATION_CONSUMABLES_TB(
            BUSINESS_ID,
            GENERATE_DATE,
            CREATE_DATE,
            UPDATE_DATE,
            ARCHIVE_DATE,
            PATIENT_ID,
            AREA_ID,
            ORG_ID,
            ORG_NAME,
            WS98_00_909_35,
            WS98_00_909_36,
            ZJ99_JH_SBS,
            ZJ99_TY_CG_CS,
            ZJ99_TY_DB_CS,
            ZJ99_CG_JH_ZXS,
            ZJ99_CG_ZS,
            ZJ99_GYS_PS_WCZS,
            ZJ99_XPS_ZS
            )
            VALUES
            (
            #{applicationConsumables.businessId,jdbcType=VARCHAR},
            decode(#{applicationConsumables.generateDate,jdbcType=DATE},null,null,to_date(to_char(#{applicationConsumables.generateDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            decode(#{applicationConsumables.createDate,jdbcType=DATE},null,sysdate,to_date(#{applicationConsumables.createDate,jdbcType=DATE},'yyyy-MM-dd')),
            decode(#{applicationConsumables.updateDate,jdbcType=DATE},null,sysdate,to_date(to_char(#{applicationConsumables.updateDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            decode(#{applicationConsumables.archiveDate,jdbcType=DATE},null,null,to_date(to_char(#{applicationConsumables.archiveDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            #{applicationConsumables.patientId,jdbcType=VARCHAR},
            #{applicationConsumables.areaId,jdbcType=VARCHAR},
            #{applicationConsumables.orgId,jdbcType=VARCHAR},
            #{applicationConsumables.orgName,jdbcType=VARCHAR},
            #{applicationConsumables.ws980090935,jdbcType=VARCHAR},
            #{applicationConsumables.ws980090936,jdbcType=VARCHAR},
            #{applicationConsumables.zj99JhSbs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99TyCgCs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99TyDbCs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99CgJhZxs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99CgZs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99GysPsWczs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99XpsZs,jdbcType=NUMERIC}
            )
        </foreach>
        SELECT 1 FROM DUAL
    </insert>

代碼

把json數據轉對象，一次提交插入500條

    @Override
    public boolean insertBatch3(Map param) throws Exception {
        List list = JsonArrayToPojoList.jsonToList(param, ApplicationConsumablesPojo.class);
        int nums = 500; // 一次插入200條
        int times = (int)Math.ceil((float)list.size() / nums);// 插入次數
        try{
            ApplicationConsumablesPojo pojo= (ApplicationConsumablesPojo)list.get(0);
            applicationConsumablesMapper.deleteBatch(pojo.getGenerateDate(),pojo.getOrgId());
            for(int i=0; i < times; i++){
                if(i == times - 1) {
                    applicationConsumablesMapper.insertBatch3(list.subList(i * nums, list.size()));
                }else {
                    applicationConsumablesMapper.insertBatch3(list.subList(i * nums, (i+1) * nums));
                }
            }
        } catch(Exception e){
            return false;
        }
        return true;
    }

利用存儲過程實現批量插入

mybatis的xml編寫

useGeneratedKeys=“false”,表示不需要JDBC自動生成主鍵

     <insert id="insertBatch" useGeneratedKeys="false" parameterType="list">
        BEGIN
        <foreach collection ="list" item="applicationConsumables" index="index" separator =";">
        INSERT INTO APPLICATION_CONSUMABLES_TB(
            BUSINESS_ID,
            GENERATE_DATE,
            CREATE_DATE,
            UPDATE_DATE,
            ARCHIVE_DATE,
            PATIENT_ID,
            AREA_ID,
            ORG_ID,
            ORG_NAME,
            WS98_00_909_35,
            WS98_00_909_36,
            ZJ99_JH_SBS,
            ZJ99_TY_CG_CS,
            ZJ99_TY_DB_CS,
            ZJ99_CG_JH_ZXS,
            ZJ99_CG_ZS,
            ZJ99_GYS_PS_WCZS,
            ZJ99_XPS_ZS
        )
        VALUES
        (
            #{applicationConsumables.businessId,jdbcType=VARCHAR},
            decode(#{applicationConsumables.generateDate,jdbcType=DATE},null,null,to_date(to_char(#{applicationConsumables.generateDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            decode(#{applicationConsumables.createDate,jdbcType=DATE},null,sysdate,to_date(#{applicationConsumables.createDate,jdbcType=DATE},'yyyy-MM-dd')),
            decode(#{applicationConsumables.updateDate,jdbcType=DATE},null,sysdate,to_date(to_char(#{applicationConsumables.updateDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            decode(#{applicationConsumables.archiveDate,jdbcType=DATE},null,null,to_date(to_char(#{applicationConsumables.archiveDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            #{applicationConsumables.patientId,jdbcType=VARCHAR},
            #{applicationConsumables.areaId,jdbcType=VARCHAR},
            #{applicationConsumables.orgId,jdbcType=VARCHAR},
            #{applicationConsumables.orgName,jdbcType=VARCHAR},
            #{applicationConsumables.ws980090935,jdbcType=VARCHAR},
            #{applicationConsumables.ws980090936,jdbcType=VARCHAR},
            #{applicationConsumables.zj99JhSbs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99TyCgCs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99TyDbCs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99CgJhZxs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99CgZs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99GysPsWczs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99XpsZs,jdbcType=NUMERIC}
        )
        </foreach>
        ;END ;
    </insert>

代碼

    @Override
    public boolean insertBatch(Map param) throws Exception{
        List list = JsonArrayToPojoList.jsonToList(param, ApplicationConsumablesPojo.class);

        int result = 1;
        SqlSession batchSqlSession = null;
        try {
            ApplicationConsumablesPojo pojo= (ApplicationConsumablesPojo)list.get(0);
            applicationConsumablesMapper.deleteBatch(pojo.getGenerateDate(),pojo.getOrgId());
            batchSqlSession = this.sqlSessionTemplate
                    .getSqlSessionFactory()
                    .openSession(ExecutorType.BATCH, false);// 獲取批量方式的sqlsession
            int batchCount = 500;// 每批commit的個數
            int batchLastIndex = batchCount;// 每批最後一個的下標
            long startDate=System.currentTimeMillis();

            for (int index = 0; index < list.size(); ) {
                if (batchLastIndex >= list.size()) {
                    batchLastIndex = list.size();

                    result = result + applicationConsumablesMapper.insertBatch(list.subList(index, batchLastIndex));
                    batchSqlSession.commit();
                    //清理緩存，防止溢出
                    batchSqlSession.clearCache();
                    System.out.println("index:" + index + " batchLastIndex:" + batchLastIndex);
                    break;// 數據插入完畢，退出循環
                } else {

                    result = result + applicationConsumablesMapper.insertBatch(list.subList(index, batchLastIndex));
                    batchSqlSession.commit();
                    //清理緩存，防止溢出
                    batchSqlSession.clearCache();
                    System.out.println("index:" + index + " batchLastIndex:" + batchLastIndex);
                    index = batchLastIndex;// 設置下一批下標
                    batchLastIndex = index + (batchCount - 1);
                }
            }
            long end=System.currentTimeMillis();

            long costTime=end-startDate;
            log.info("----------------------------"+costTime+"--------------------");
            batchSqlSession.commit();
        } catch (Exception e) {
            e.printStackTrace();
            return false;
        } finally {
            batchSqlSession.close();
        }
        return true;
    }

利用標籤

mybatis的xml編寫

    <insert id="insertBatch2" useGeneratedKeys="false" parameterType="list">
            INSERT INTO APPLICATION_CONSUMABLES_TB(
            BUSINESS_ID,
            GENERATE_DATE,
            CREATE_DATE,
            UPDATE_DATE,
            ARCHIVE_DATE,
            PATIENT_ID,
            AREA_ID,
            ORG_ID,
            ORG_NAME,
            WS98_00_909_35,
            WS98_00_909_36,
            ZJ99_JH_SBS,
            ZJ99_TY_CG_CS,
            ZJ99_TY_DB_CS,
            ZJ99_CG_JH_ZXS,
            ZJ99_CG_ZS,
            ZJ99_GYS_PS_WCZS,
            ZJ99_XPS_ZS
            )
            (
        <foreach collection ="list" item="applicationConsumables" index="index" separator="union all">
            select
            #{applicationConsumables.businessId,jdbcType=VARCHAR},
            decode(#{applicationConsumables.generateDate,jdbcType=DATE},null,null,to_date(to_char(#{applicationConsumables.generateDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            decode(#{applicationConsumables.createDate,jdbcType=DATE},null,sysdate,to_date(#{applicationConsumables.createDate,jdbcType=DATE},'yyyy-MM-dd')),
            decode(#{applicationConsumables.updateDate,jdbcType=DATE},null,sysdate,to_date(to_char(#{applicationConsumables.updateDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            decode(#{applicationConsumables.archiveDate,jdbcType=DATE},null,null,to_date(to_char(#{applicationConsumables.archiveDate,jdbcType=DATE},'yyyy-MM-dd'),'yyyy-MM-dd')),
            #{applicationConsumables.patientId,jdbcType=VARCHAR},
            #{applicationConsumables.areaId,jdbcType=VARCHAR},
            #{applicationConsumables.orgId,jdbcType=VARCHAR},
            #{applicationConsumables.orgName,jdbcType=VARCHAR},
            #{applicationConsumables.ws980090935,jdbcType=VARCHAR},
            #{applicationConsumables.ws980090936,jdbcType=VARCHAR},
            #{applicationConsumables.zj99JhSbs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99TyCgCs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99TyDbCs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99CgJhZxs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99CgZs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99GysPsWczs,jdbcType=NUMERIC},
            #{applicationConsumables.zj99XpsZs,jdbcType=NUMERIC}
            from dual
           </foreach>
          )
    </insert>

代碼

    @Override
    public boolean insertBatch2(Map param) throws Exception {
        List list = JsonArrayToPojoList.jsonToList(param, ApplicationConsumablesPojo.class);
        int nums = 500; // 一次插入200條
        int times = (int)Math.ceil((float)list.size() / nums);// 插入次數
        try{
            ApplicationConsumablesPojo pojo= (ApplicationConsumablesPojo)list.get(0);
            applicationConsumablesMapper.deleteBatch(pojo.getGenerateDate(),pojo.getOrgId());
            for(int i=0; i < times; i++){
                if(i == times - 1) {
                    applicationConsumablesMapper.insertBatch2(list.subList(i * nums, list.size()));
                }else {
                    applicationConsumablesMapper.insertBatch2(list.subList(i * nums, (i+1) * nums));
                }
            }
        } catch(Exception e){
            return false;
        }
        return true;
    }

三種方式耗時對比

批量插入條數200條時

方式	數據量	耗時（秒）
利用存儲過程實現批量插入	2萬	22.233
利用foreach標籤	2萬	13.023
使用oracle的insert all	2萬	17.489

批量插入條數500條時

方式	數據量	耗時（秒）
利用存儲過程實現批量插入	2萬	16.906
利用foreach標籤	2萬	13.058
使用oracle的insert all	2萬	16.139

批量插入條數1000條時

方式	數據量	耗時（秒）
利用存儲過程實現批量插入	2萬	60.583
利用foreach標籤	2萬	13.124
使用oracle的insert all	2萬	16.478

每次寫入的數量,以及網絡環境對花費的時間也是有很大的影響的。