Ceph測試系列（一）：Ceph 應用方案的服務器硬件選型 (2 of 2)

一、前言

redhat官網給出不同應用場景時，其推薦的Ceph集羣服務器硬件配置（包括CPU/Memory/Disk/Network）。僅作爲服務器配置選擇時參考，不做推薦使用。

場景包含如下：

場景一側重IOPS的，例如實時性要求高的，但數據量不大的。比如訂單的生成。

場景二側重Throughput的，吞吐量大，但IOPS延時性要求適當。例如，實時流。

場景三側重容量和價格Cost/Capacity的，例如大文件的存儲。

SOLUTIONS	CPU	RAM	Networking	OSD Media	OSDs	Journal Media	Controller/HBA	Vendor(i.e.)	Example Uses
IOPS-OPTIMIZED	10 cores per NVMe SSD, assuming a 2 GHz CPU.	16GB baseline, plus 2GB per OSD	10 Gigabit Ethernet (GbE) per 12 OSDs (each for client- and cluster-facing networks)	High-performance, high-endurance enterprise NVMe SSDs	Four per NVMe SSD	High-performance, high-endurance enterprise NVMe SSD, co-located with OSDs	Native PCIe bus	SuperMICRO(SYS-5038MR-OSD006P)	Database
THROUGHPUT-OPTIMIZED	0.5 cores per HDD, assuming a 2 GHz CPU	16GB baseline, plus 2GB per OSD	10 GbE per 12 OSDs (each for client- and cluster-facing networks)	7,200 RPM enterprise HDDs	One per HDD	High-endurance, high-performance enterprise serial-attached SCSI (SAS) or NVMe SSDs; OSD-to-journal ratio: 4-5:1 for an SSD journal, or 12-18:1 for an NVMe journal	Host bus adapter (HBA): just a bunch of disks (JBOD)	SuperMICRO(SSG-6028R-OSD072P)	Streaming media
COST/CAPACITY-OPTIMIZED	0.5 cores per HDD, assuming a 2 GHz CPU	16GB baseline, plus 2GB per OSD	10 GbE per 12 OSDs (each for client- and cluster-facing networks)	7,200 RPM enterprise HDDs	One per HDD	Co-located on the HDD	HBA:JBOD	SuperMICRO(SSG-6048R-OSD216P)	Archive
Reference: https://www.redhat.com/en/resources/resources-red-hat-ceph-storage-hardware-selection-guide-html

塊和文件存儲類服務器：OSD 服務器 (CPU負載低)

Category	Block and File OSD Node

Server SKU	SSG-6019P-ACR12L+			SSG-5029P-E1CTR12L			SSG-6029P-E1CR12L
Profile	Density Optimized			Cost Optimized			Compute Optimized
Form Factor	1U			2U			2U
CPU	Dual Intel® Xeon® Scalable Processors (8C/16T each)			Dual Intel® Xeon® Scalable Processors (12C/24T each)			Dual Intel® Xeon® Scalable Processors (8C/16T each)
Memory	96GB DDR4 RDIMM
HBA	Broadcom 3216 (IT mode)			Broadcom 3008 (IT mode)			Broadcom 3008 (IT mode)
Power	600W Redundant Power Supplies			800W Redundant Power Supplies			1200W Redundant Power Supplies
NIC	Dual-Port 25G Mellanox (SFP+ compatibility) via AOC
Storage	Type	Qty	Form Factor	Type	Qty	Form Factor	Type	Qty	Form Factor
Caching	Optional NVMe	Up to 2	2.5" 7mm	Optional NVMe	1	AIC	Optional NVMe	1	AIC
Capacity	6-12 TB 7K RPM	12	3.5" SAS3 HDD	6-12 TB 7K RPM	12	3.5" SAS3 HDD	6-12 TB 7K RPM	12	3.5" SAS3 HDD
Drive (OS)	2x 2.5" 7mm SATA SSD			2x 2.5" SATA SSD			2x 2.5" SATA SSD

媒體類媒體流和媒體倉庫服務器：OSD服務器 (CPU負載適中)

Category	Media OSD Node (File and Object)

Server SKU	SSG-6029P-E1CR24L			SSG-6049P-E1CR45L SSG-6049P-E1CR45L+
Profile	Media Streaming			Media Repository
Form Factor	1U			2U
CPU	Dual Intel® Xeon® Scalable Processors (12C/24T each)			Dual Intel® Xeon® Scalable Processors (12C/24T each)
Memory	192GB DDR4 RDIMM			384GB DDR4 RDIMM
HBA	Broadcom 3008 (IT mode)			Broadcom 3008 (IT mode)(L model) Broadcom 3216 (IT mode)(L+ model)
Power	1600W Redundant Power Supplies			1600W Redundant Power Supplies
NIC	SIOM-based Dual-Port 25G Mellanox (SFP+ compatibility) + Dual-Port 10G RJ45
Storage	Type	Qty	Form Factor	Type	Qty	Form Factor
Caching	Optional NVMe	Up to 2	AIC	Optional NVMe	Up to 6	2.5" U.2
Capacity	6-12 TB 7K RPM	24	3.5" SAS3 HDD	6-12 TB 7K RPM	45	3.5" SAS3 HDD
Drive (OS)	2x 2.5" SATA SSD			2x 2.5" SATA SSD

媒體類存檔服務器：OSD服務器(CPU負載稍高)

Category	Media OSD Node (File and Object)

Server SKU	SSG-6049P-E1CR60L SSG-6049P-E1CR60L+
Profile	Archive (minimum recommended deployment size of 10+ OSD Nodes)
Form Factor	4U
CPU	Dual Intel® Xeon® Scalable Processors (14C/28T each)
Memory	384GB DDR4 RDIMM
HBA	Broadcom 3008 (IT mode)(L model) Broadcom 3616 (IT mode)(L+ model)
Power	2000W Redundant Power Supplies
NIC	SIOM-based Dual-Port 25G Mellanox (SFP+ compatibility) + Dual-Port 10G RJ45
Storage	Type	Qty	Form Factor
Caching	Optional NVMe	Up to 6	2.5" U.2
Capacity	6-12 TB 7K RPM	60	3.5" SAS3 HDD
Drive (OS)	2x 2.5" SATA SSD

Monitor/網關服務器 (CPU負載適中)

Category	Monitor Node / Gateway

Server SKU	SYS-1019P-WTR
Profile	Flash Performance (minimum recommended deployment size of 3 Monitor Nodes)
Form Factor	1U
CPU	Single Intel® Xeon® Scalable Processor (12C/24T each)
Memory	96GB DDR4 RDIMM
HBA	Broadcom 3008 (IT mode)(L model)
Power	500W Redundant Power Supplies
NIC	Dual-Port 25G Mellanox (SFP+ compatibility) via AOC
Storage	Type	Qty	Form Factor
Caching	N/A	-	-
Capacity	N/A	-	-
Drive (OS)	2x 2.5" SATA SSD

美光固態高速服務器：(CPU負載高)

Category	Micron Solid Ready

Reference BOM Check with PM for details	SYS-1028U-MON50M			SYS-1029U-OSD100M
Server SKU	SYS-1028U-TNRT+			SYS-1029U-TN10RT
Profile	Monitor Node			OSD Node
Form Factor	1U			1U
CPU	Dual Intel® Xeon® Scalable Processors (10C/20T each)			Dual Intel® Xeon® Scalable Processors (24C/48T each)
Memory	128GB DDR4 RDIMM			384GB DDR4 RDIMM
HBA	Broadcom 3008 (IT mode)			N/A
Power	750W Redundant Power Supplies			1000W Redundant Power Supplies
NIC	2x Single-Port 50GbE QSFP28 (Mellanox)			2x Dual-Port 100GbE QSFP (Mellanox)
Storage	Type	Qty	Form Factor	Type	Qty	Form Factor
Caching	N/A	-	-	Optional NVMe	-	-
Capacity	N/A	-	-	6.4 TB NVMe	10	U.2 NVMe
Drive (OS)	1x SATADOM			1x M.2 SATA SSD

可以看到高負載的CPU服務器需求集中在(美光)固態高速服務器和媒體類存檔服務器(OSD服務器)。由於沒有高負載計算過，存儲服務器總體來說對CPU要求不高。所以如果你的CPU配置高，CPU負載不高，可以用如下服務器配置優化方案：