PostGrepSql 線程池多線程數據插入

需求

數據庫表複製，數據量大時，用單純的客戶端工具只能單線程導入，時間耗時長。本文主要採用 JDBC 和 線程池 解決這個問題。

JDBC

我用的 PostGrepSql 數據庫，根據自己的數據庫 添加pom.xml

 <dependency>
        <groupId>postgresql</groupId>
        <artifactId>postgresql</artifactId>
        <version>9.1-901-1.jdbc4</version>
 </dependency>

工具類

static String url = "*";
static String user = "*";
static String password = "*";

public static Connection getConnect() {
        Connection con = null;
        try {
            Class.forName("org.postgresql.Driver");
            con = DriverManager.getConnection(url, user, password);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return con;
    }

需要執行的SQL 添加到一個集合，我用的是

INSERT INTO TableA (select * from TableB)

定義線程池

public static void main(String[] args) {

        List<String> recordList = DbUtils.getSqlList();

        int threadCount = 20;

        if (recordList.size() > 0) {

            System.out.println(new Date() + "準備執行的 SQL Total --- > " + recordList.size());

            final BlockingQueue<String> queue = new ArrayBlockingQueue<String>(recordList.size());

            for (String record : recordList) {
                queue.offer(record);
            }

            ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
            final CountDownLatch countDownLatch = new CountDownLatch(recordList.size());

            for (int i = 0; i < threadCount; i++) {
                executorService.submit(new Runnable() {
                    @Override
                    public void run() {
                        while (true) {
                            if (queue.isEmpty()) {
                                break;
                            }
                            String string = null;
                            try {
                                string = queue.take();

                            } catch (InterruptedException e1) {
                                System.out.println(new Date() + " " + e1);
                            }
                            try {
                            	//單線程執行SQL
                                singleInsert(string);
                                System.out.println(new Date()+"  "+string);
                            } catch (Exception e) {
                                System.out.println(e.getMessage());
                            } finally {
                                countDownLatch.countDown();
                            }
                        }
                    }
                });
            }
            while (countDownLatch.getCount() != 0) {
                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            executorService.shutdownNow();
            System.out.println(new Date() + "多線程導入結束");
        }
    }