List列表去重方法

---

一. 對列表元素爲String類的去重

1. 法1：使用Jav8 Stream類的distinct()方法（推薦）

1. 概述
   • distinct()方法使用hashCode()和eqauls()方法獲取不同元素。因此，需要去重的類必須實現這兩個方法。
   • 因爲String類已經重寫了這兩個方法，因此去重可以直接使用。
2. 實現示例

/**
 * 測試
 */
public void streamDeduplication1() {
    List<String> lists = Lists.newArrayList("a", "b", "f", "a", "z");
    System.out.println("去重前："+JSON.toJSON(lists));
    List<String> deDuplicationList = lists.stream().distinct().collect(Collectors.toList());
    System.out.println("去重後："+JSON.toJSON(deDuplicationList));
}

• 輸出結果

去重前：["a","b","f","a","z"]
去重後：["a","b","f","z"]

2. 法2：使用臨時List在添加元素時進行重複判斷

public void streamDeduplication1_1(){
    List<String> lists = Lists.newArrayList("a", "b", "f", "a", "z");
    System.out.println("去重前："+JSON.toJSON(lists));
    List<String> tempList = Lists.newArrayList();
    for(String str:lists){
        if(!tempList.contains(str)){
            tempList.add(str);
        }
    }
    System.out.println("去重後："+JSON.toJSON(tempList));
}

• 輸出結果

去重前：["a","b","f","a","z"]
去重後：["a","b","f","z"]

3. 法3：使用set去重

public void streamDeduplication1_2(){
    List<String> lists = Lists.newArrayList("a", "b", "f", "a", "z");
    System.out.println("去重前："+JSON.toJSON(lists));
    lists = Lists.newArrayList(Sets.newHashSet(lists));
    System.out.println("去重後："+JSON.toJSON(lists));
}

• 輸出結果

去重前：["a","b","f","a","z"]
去重後：["a","b","f","z"]

4. 法4：使用Collections.frequency()方法

1. Collections.frequency()：統計每個元素的在集合中的個數。當集合中數據量特別大的時候，性能不好。

public void streamDeduplication1_3(){
   List<String> lists = Lists.newArrayList("a", "b", "f", "a", "z");
   System.out.println("去重前："+JSON.toJSON(lists));
   for(int i=0;i<lists.size();i++){
       String currentValue = lists.get(i);
       if(Collections.frequency(lists,currentValue)>1){
           lists.remove(currentValue);
       }
   }
   System.out.println("去重後："+JSON.toJSON(lists));
}

• 輸出結果

去重前：["a","b","f","a","z"]
去重後：["a","b","f","z"]

2. 注意⚠️：在循環過程中刪除集合中元素時，只能通過iterator.remove()方法和使用for循環方式，不能使用增強for循環和forEach()方法。

---

二. 對列表元素爲實體類的去重

1. 推薦方法：使用Jav8 Stream類的distinct()方法。
   • 對於實體類，可以通過重寫其hashCode()和equals()方法來達到去重。
   • 更快速的方法，可以使用Lombok插件的@Data註解，它自動重寫了equals()和hashCode()方法。

1. 法1：使用Lombok插件的@Data註解

1. 示例代碼

/**
 * 定義一個實體類
 */ 
@Data
public class StudentDTO {
    private Integer studentId;
    private String studentName;

    public StudentDTO(Integer studentId, String studentName) {
        this.studentId = studentId;
        this.studentName = studentName;
    }
}

/**
 * 測試
 */
public void streamDeduplication2() {
        List<StudentDTO> studentDTOS = Lists.newArrayList();
        studentDTOS.add(new StudentDTO(1,"xixi"));
        studentDTOS.add(new StudentDTO(2,"houhou"));
        studentDTOS.add(new StudentDTO(2,"houhou"));
        System.out.println("去重前："+JSON.toJSON(studentDTOS));

        List<StudentDTO> deDupDTOS = studentDTOS.stream().distinct().collect(Collectors.toList());
        System.out.println("去重後："+JSON.toJSON(deDupDTOS));
    }

• 輸出結果

去重前：[{"studentId":1,"studentName":"xixi"},{"studentId":2,"studentName":"houhou"},{"studentId":2,"studentName":"houhou"}]
去重後：[{"studentId":1,"studentName":"xixi"},{"studentId":2,"studentName":"houhou"}]

2. 法2：重寫equals和hashCode方法

/**
  * 定義實體類
  */
public class StuDTO {
    private Integer studentId;
    private String studentName;

    public StuDTO(Integer studentId, String studentName) {
        this.studentId = studentId;
        this.studentName = studentName;
    }

   // getter and setter

    @Override
    public boolean equals(Object obj) {
        if(this==obj){
            return true;
        }
        if(obj==null || getClass()!=obj.getClass()) {
            return false;
        }
        StuDTO stuDTO = (StuDTO)obj;
        return Objects.equals(getStudentId(),stuDTO.getStudentId()) &&
                Objects.equals(getStudentName(),stuDTO.getStudentName());
    }

    @Override
    public int hashCode() {
        return Objects.hash(getStudentId(),getStudentName());
    }
}

/**
  * 測試
  */
public void streamDeduplication2_1() {
    List<StuDTO> stuDTOS = Lists.newArrayList();
    stuDTOS.add(new StuDTO(1,"xixi"));
    stuDTOS.add(new StuDTO(2,"houhou"));
    stuDTOS.add(new StuDTO(2,"houhou"));
    System.out.println("去重前："+JSON.toJSON(stuDTOS));

    List<StuDTO> deDupDTOS = stuDTOS.stream().distinct().collect(Collectors.toList());
    System.out.println("去重後："+JSON.toJSON(deDupDTOS));
}

• 輸出結果

去重前：[{"studentId":1,"studentName":"xixi"},{"studentId":2,"studentName":"houhou"},{"studentId":2,"studentName":"houhou"}]
去重後：[{"studentId":1,"studentName":"xixi"},{"studentId":2,"studentName":"houhou"}]

---

三. 對列表元素爲實體類某屬性的去重

1. 方法：首先創建一個方法作爲 Stream.filter() 的參數，其返回類型爲 Predicate，原理就是判斷一個元素能否加入到 Set 中，代碼如下：

private static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
    Set<Object> seen = ConcurrentHashMap.newKeySet();
    return t -> seen.add(keyExtractor.apply(t));
}

2. 示例代碼

public void streamDeduplication3() {
    List<StudentDTO> studentDTOS = Lists.newArrayList();
    studentDTOS.add(new StudentDTO(1,"xixi"));
    studentDTOS.add(new StudentDTO(2,"houhou"));
    studentDTOS.add(new StudentDTO(3,"houhou"));
    System.out.println("去重前："+JSON.toJSON(studentDTOS));

    studentDTOS = studentDTOS.stream().filter(distinctByKey(StudentDTO::getStudentName)).collect(Collectors.toList());
    System.out.println("去重後："+JSON.toJSON(studentDTOS));
}

• 輸出結果

去重前：[{"studentId":1,"studentName":"xixi"},{"studentId":2,"studentName":"houhou"},{"studentId":3,"studentName":"houhou"}]
去重後：[{"studentId":1,"studentName":"xixi"},{"studentId":2,"studentName":"houhou"}]

---

參考資料

• 分享幾種 Java8 中通過 Stream 對列表進行去重的方法