【日常總結一】批量處理List中的數據

需求

將List集合中的數據分批次插入到數據庫

實現方式

• Java8方式

List<Integer> oldList = Arrays.asList(1, 2, 3, 4, 5, 6, 7);
int limit = (oldList.size() + MAX_NUMBER - 1) / MAX_NUMBER;
//方法一：使用流遍歷操作
List<List<Integer>> newList = new ArrayList<>();
Stream.iterate(0, n -> n + 1)
        .limit(limit)
        .forEach(i -> {
            newList.add(oldList.stream()
                    .skip(i * MAX_NUMBER)
                    .limit(MAX_NUMBER)
                    .collect(Collectors.toList()));
        });

//方法二：獲取分割後的集合
List<List<Integer>> splitList =
       Stream.iterate(0, n -> n + 1)
               .limit(limit)
               .parallel()
               .map(a -> oldList.stream()
                       .skip(a * MAX_NUMBER)
                       .limit(MAX_NUMBER)
                       .parallel()
                       .collect(Collectors.toList()))
               .collect(Collectors.toList());

• google-guava方式

List<Integer> oldList = Arrays.asList(1, 2, 3, 4, 5, 6, 7);
// 每3個爲一組
List<List<Integer>> newList = Lists.partition(oldList, 3);

• apache-common-collection包

List<Integer> oldList = Lists.newArrayList(1, 2, 3, 4, 5, 6, 7, 8);
// 每3個爲一組
List<List<Integer>> subs = ListUtils.partition(oldList, 3);

• 手動寫一個函數將list等分成多個list

public static <T> List<List<T>> averageAssign(List<T> source, int n) {
    List<List<T>> result = new ArrayList<>();
    //(先計算出餘數)
    int remainder = source.size() % n;
    //然後是商
    int number = source.size() / n;
    //偏移量
    int offset = 0;
    for (int i = 0; i < n; i++) {
        List<T> value;
        if (remainder > 0) {
            value = source.subList(i * number + offset, (i + 1) * number + offset + 1);
            remainder--;
            offset++;
        } else {
            value = source.subList(i * number + offset, (i + 1) * number + offset);
        }
        result.add(value);
    }
    return result;
}

參考鏈接

• java8 Stream 大數據量List分批處理切割