操作命令:
ceph osd getcrushmap -o map_old 導出map
crushtool -d map_old -o map_old.txt 轉化成可編輯格式
crushtool -c map_new.txt -o map_new 還原爲map
ceph osd setcrushmap -i map_new 將map導入ceph
更改Crush map流程:
1、修改配置文件,防止ceph自動更新crushmap
echo 'osd_crush_update_on_start = false' >> /etc/ceph/ceph.conf /etc/init.d/ceph restart
2、導出map,並轉化成可編輯格式
ceph osd getcrushmap -o map_old crushtool -d map_old -o map_old.txt cp map_old.txt map_new.txt
3、更改map_new.txt
如:
##################sas host node1-sas { id -2 # do not change unnecessarily # weight 0.120 alg straw hash 0 # rjenkins1 item osd.0 weight 0.040 item osd.1 weight 0.040 } host node2-sas { id -3 # do not change unnecessarily # weight 0.120 alg straw hash 0 # rjenkins1 item osd.3 weight 0.040 item osd.4 weight 0.040 } ##################ssd host node1-ssd { id -5 # do not change unnecessarily # weight 0.120 alg straw hash 0 # rjenkins1 item osd.2 weight 0.040 } host node2-ssd { id -6 # do not change unnecessarily # weight 0.120 alg straw hash 0 # rjenkins1 item osd.5 weight 0.040 } #################pool root sas-pool { id -1 # do not change unnecessarily # weight 0.360 alg straw hash 0 # rjenkins1 item node1-sas weight 0.080 item node2-sas weight 0.080 } root ssd-pool { id -8 # do not change unnecessarily # weight 0.360 alg straw hash 0 # rjenkins1 item node1-ssd weight 0.040 item node2-ssd weight 0.040 } ##################rule rule sas { ruleset 0 type replicated min_size 1 max_size 10 step take sas-pool step choose firstn 0 type osd step emit } rule ssd { ruleset 1 type replicated min_size 1 max_size 10 step take ssd-pool step choose firstn 0 type osd step emit }
4、將修改後的crushmap編譯並且注入集羣中
crushtool -c map_new.txt -o map_new ceph osd setcrushmap -i map_new ceph osd tree
5、創建資源池,使用新創建的規則
ceph osd pool create sas 128 128 ceph osd pool create ssd 128 128 ceph osd pool set sas crush_ruleset 0 ceph osd pool set ssd crush_ruleset 1
6、重新對pool授權(如果對接過openstack)
ceph auth del client.cinder ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children,allow rwx pool=volumes, \ allow rwx pool=vms, allow rx pool=p_w_picpaths, allow rwx pool=ssd'
更改Monitor IP流程:
1、獲取monmap,並查看
ceph mon getmap -o map monmaptool --print map
2、刪除舊的map配置,新增配置到map
monmaptool --rm node1 --rm node2 --rm node3 map monmaptool --add node1 10.0.2.21:6789 --add node2 10.0.2.22:6789 --add node3 10.0.2.23:6789 map monmaptool --print map
3、複製新map到所有mon節點
scp mon node2:~ scp mon node3:~
4、更改/etc/ceph/ceph.conf中的mon_host(所有mon節點執行)
vim /etc/ceph/ceph.conf mon_host= new1_ip:6789,new2_ip:6789,new3_ip:6789:
5、停止mon進程(所有mon節點執行)
/etc/init.d/ceph stop mon
6、載入新的monmap(所有mon節點執行)
ceph-mon -i node1 --inject-monmap map
7、啓動mon(所有mon節點執行)
/etc/init.d/ceph start mon