使用elasticdump遷移數據到新es集羣

參考文檔

https://blog.csdn.net/u012731379/article/details/79856113 
https://blog.csdn.net/u010466329/article/details/79209236 
https://blog.csdn.net/laoyang360/article/details/65449407 

遷移方法

  • 通過logstash的input和output配置遷移(配置靈活適用於長期數據同步等)
  • 通過遷移工具如elasticdump等(適用於備份一次性小量數據操作)
  • 通過elasticsarch自帶快照功能(適用於一次性遷移大量數據)

操作步驟

1、安裝elasticdump

[root@VM_8_24_centos ~]# yum install nodejs npm  
[root@VM_8_24_centos ~]# npm install elasticdump 
/root
└─┬ [email protected] 
  ├── [email protected] 
  ├─┬ [email protected] 
  │ ├─┬ [email protected] 
  │ │ ├── [email protected] 
  │ │ └── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├─┬ [email protected] 
  │ │ └── [email protected] 
  │ ├── [email protected] 
  │ └─┬ [email protected] 
  │   └── [email protected] 
  ├── [email protected] 
  ├── [email protected] 
  ├── [email protected] 
  ├── [email protected] 
  ├─┬ [email protected] 
  │ ├── [email protected] 
  │ └── [email protected] 
  ├── [email protected] 
  ├── [email protected] 
  ├─┬ [email protected] 
  │ ├── [email protected] 
  │ └── [email protected] 
  ├─┬ [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├─┬ [email protected] 
  │ │ └── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├─┬ [email protected] 
  │ │ └── [email protected] 
  │ ├─┬ [email protected] 
  │ │ ├─┬ [email protected] 
  │ │ │ ├── [email protected] 
  │ │ │ ├── [email protected] 
  │ │ │ ├── [email protected] 
  │ │ │ └─┬ [email protected] 
  │ │ │   └── [email protected] 
  │ │ └── [email protected] 
  │ ├─┬ [email protected] 
  │ │ ├── [email protected] 
  │ │ ├─┬ [email protected] 
  │ │ │ ├── [email protected] 
  │ │ │ ├── [email protected] 
  │ │ │ └── [email protected] 
  │ │ └─┬ [email protected] 
  │ │   ├── [email protected] 
  │ │   ├── [email protected] 
  │ │   ├── [email protected] 
  │ │   ├── [email protected] 
  │ │   ├── [email protected] 
  │ │   ├── [email protected] 
  │ │   ├── [email protected] 
  │ │   └── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├─┬ [email protected] 
  │ │ └── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├── [email protected] 
  │ ├─┬ [email protected] 
  │ │ ├── [email protected] 
  │ │ └── [email protected] 
  │ └── [email protected] 
  ├─┬ [email protected] 
  │ └── [email protected] 
  └─┬ [email protected] 
    ├── [email protected] 
    └─┬ [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      └── [email protected] 

npm WARN enoent ENOENT: no such file or directory, open '/root/package.json'
npm WARN root No description
npm WARN root No repository field.
npm WARN root No README data
npm WARN root No license field.
[root@VM_8_24_centos ~]# cd node_modules/elasticdump/bin 

2、導出mapping

[root@VM_8_24_centos bin]# ./elasticdump --input=http://10.2.3.159:9200/reconciliation  --output=http://10.2.100.24:9200/reconciliationonline --type=mapping
Mon, 11 Feb 2019 09:28:10 GMT | starting dump
Mon, 11 Feb 2019 09:28:10 GMT | got 1 objects from source elasticsearch (offset: 0)
Mon, 11 Feb 2019 09:28:16 GMT | sent 1 objects to destination elasticsearch, wrote 1
Mon, 11 Feb 2019 09:28:16 GMT | got 0 objects from source elasticsearch (offset: 1)
Mon, 11 Feb 2019 09:28:16 GMT | Total Writes: 1
Mon, 11 Feb 2019 09:28:16 GMT | dump complete
[root@VM_8_24_centos bin]# curl  10.2.100.24:9200/_cat/indices|grep reconciliation
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 43 32578   43 14037    0     0   5694      0  0:00:05  0:00:02  0:00:03  5694yellow open reconciliationonline   KnZrsU-7TmWWBs-1-dBRrw 5 1       0  0    1.1kb    1.1kb

3、導出data

[root@VM_8_24_centos bin]# ./elasticdump --input=http://10.2.3.159:9200/reconciliation  --output=http://10.2.100.24:9200/reconciliationonline --type=data
Mon, 11 Feb 2019 09:28:58 GMT | starting dump
Mon, 11 Feb 2019 09:28:58 GMT | got 100 objects from source elasticsearch (offset: 0)
Mon, 11 Feb 2019 09:29:19 GMT | sent 100 objects to destination elasticsearch, wrote 100
Mon, 11 Feb 2019 09:29:19 GMT | got 100 objects from source elasticsearch (offset: 100)
Mon, 11 Feb 2019 09:29:51 GMT | sent 100 objects to destination elasticsearch, wrote 100
Mon, 11 Feb 2019 09:29:51 GMT | got 100 objects from source elasticsearch (offset: 200)
Mon, 11 Feb 2019 09:30:13 GMT | sent 100 objects to destination elasticsearch, wrote 100
Mon, 11 Feb 2019 09:30:13 GMT | got 100 objects from source elasticsearch (offset: 300)
Mon, 11 Feb 2019 09:30:39 GMT | sent 100 objects to destination elasticsearch, wrote 100
Mon, 11 Feb 2019 09:30:39 GMT | got 100 objects from source elasticsearch (offset: 400)
Mon, 11 Feb 2019 09:31:53 GMT | sent 100 objects to destination elasticsearch, wrote 100
Mon, 11 Feb 2019 09:31:53 GMT | got 100 objects from source elasticsearch (offset: 500)
Mon, 11 Feb 2019 09:32:28 GMT | sent 100 objects to destination elasticsearch, wrote 100
Mon, 11 Feb 2019 09:32:28 GMT | got 100 objects from source elasticsearch (offset: 600)
Mon, 11 Feb 2019 09:33:24 GMT | sent 100 objects to destination elasticsearch, wrote 100
Mon, 11 Feb 2019 09:33:24 GMT | got 100 objects from source elasticsearch (offset: 700)
Mon, 11 Feb 2019 09:34:19 GMT | sent 100 objects to destination elasticsearch, wrote 100
Mon, 11 Feb 2019 09:34:19 GMT | got 7 objects from source elasticsearch (offset: 800)
Mon, 11 Feb 2019 09:34:47 GMT | sent 7 objects to destination elasticsearch, wrote 7
Mon, 11 Feb 2019 09:34:47 GMT | got 0 objects from source elasticsearch (offset: 807)
Mon, 11 Feb 2019 09:34:47 GMT | Total Writes: 807
Mon, 11 Feb 2019 09:34:47 GMT | dump complete
[root@VM_8_24_centos bin]# curl  10.2.100.24:9200/_cat/indices|grep reconciliationonline
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:16 --:--:--     0yellow open reconciliationonline   KnZrsU-7TmWWBs-1-dBRrw 5 1     807  0    4.3mb    4.3mb
100 32578  100 32578    0     0   2010      0  0:00:16  0:00:16 --:--:--  7769
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章