1、硬件優化
2、操作系統優化
3、網絡優化
4、Ceph自身優化
一、硬件優化
1、OSD節點的CPU應當依照一個osd一個cpu核心匹配。
2、Mon節點對cpu要求不高,內存>2G
3、mds節點cpu要求較高,可以適當配置。
4、BIOS設置啓用HT(超線程)、VT(虛擬化)
5、BIOS關閉節能選項
6、BIOS關閉NUMA
二、操作系統優化
1、修改Linux默認的IO調度:
SSD:echo noop > /sys/block/sdX/queue/scheduler //SSD優化
HDD:echo deadline > /sys/block/sdX/queue/scheduler //機械硬盤優化
2、Linux默認的預讀不適合RADOS,建議設置更大值。
修改如下:
echo “1892” > /sys/block/sdX/queue/read_ahead_kb
3、進程數量
OSD進程需要消耗大量進程,關於內核PID,如果單服務器OSD數量過多,建議設置更大值。
Echo 4194303 > /proc/sys/kernel/pid_max
4、調整CPU頻率,使其運行在高性能之下。
echo performance | tee /sys/devices/system/cpu/cpu*/cpufreg/scaling_governor >/dev/null
5、SWAP交換分區
修改 vi /etc/sysctl.conf ,將vm.swappiness=0加入進去,降低SWAP使用率,減少磁盤IO,提升性能。
三、網絡優化
建議雙網卡,萬兆網卡,提升帶寬。
四、Ceph自身優化
參考如下/etc/ceph.conf的配置:
[global]
fsid = 059f27e8-a23f-4587-9033-3e3679d03b31
mon_host = 10.10.20.102, 10.10.20.101, 10.10.20.100
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd pool default size = 3
osd pool default min size = 1
public network = 10.10.20.0/24
cluster network = 10.10.20.0/24
max open files = 131072
[mon]
mon data = /var/lib/ceph/mon/ceph-$id
[osd]
osd data = /var/lib/ceph/osd/ceph-$id
osd journal size = 20000
osd mkfs type = xfs
osd mkfs options xfs = -f
filestore xattr use omap = true
filestore min sync interval = 10
filestore max sync interval = 15
filestore queue max ops = 25000
filestore queue max bytes = 10485760
filestore queue committing max ops = 5000
filestore queue committing max bytes = 10485760000
journal max write bytes = 1073714824
journal max write entries = 10000
journal queue max ops = 50000
journal queue max bytes = 10485760000
osd max write size = 512
osd client message size cap = 2147483648
osd deep scrub stride = 131072
osd op threads = 8
osd disk threads = 4
osd map cache size = 1024
osd map cache bl size = 128
osd mount options xfs = "rw,noexec,nodev,noatime,nodiratime,nobarrier"
osd recovery op priority = 4
osd recovery max active = 10
osd max backfills = 4
[client]
rbd cache = true
rbd cache size = 268435456
rbd cache max dirty = 134217728
rbd cache max dirty age = 5
參考:https://blog.csdn.net/fuzhongfaya/article/details/80932766