實驗環境
node1:192.168.0.1
node2:192.168.0.2
node3:192.168.0.3
fence&qdisk:192.168.0.4
實驗前準備
1.同步格服務器的時間（誤差1分鐘之內）
date -s
2.確認網關是否正確
route -n
3.開啓ssh圖形化
vim /etc/ssh/ssh_config
ForwardX11 yes
一、fence配置ssh key、hosts文件，發送到3臺node上
ssh-keygen
ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.0.1
ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.0.2
ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.0.3
vim /etc/hosts
192.168.0.1 node1.test.com node1
192.168.0.2 node2.test.com node2
192.168.0.3 node3.test.com node3
192.168.0.4 fence.test.com fence
for i in 1 2 3 ;do scp /etc/hosts node$i:/etc/ ;done
二、安裝fence所需的包，遠程安裝node上所需的包
for i in 1 2 3 ;do ssh node$i "yum groupinstall -y "Clustering"" ;done
for i in 1 2 3 ;do ssh node$i "yum install -y scsi-target-utils gfs2-utils lvm2-cluster mysql-server" ;done
yum groupinstall -y "Clustering"
yum install -y gnbd_serv iscsi-initiator-utils lvm2-cluster mysql-server
三、fence上導出兩個iscsi共享
vim /etc/tgt/targets.conf
<target iqn.2011-12.com.test:fence.qdisk>
backing-store /dev/sda7
</target>
<target iqn.2011-12.com.test:fence.mysql>
backing-store /dev/vg0/lv0
</target>
/etc/init.d/tgtd start
chkconfig tgtd on
四、3臺node導入iscsi共享，創建qdisk。
vim /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.2011-12.com.test:fence.qdisk
InitiatorName=iqn.2011-12.com.test:fence.mysql
/etc/init.d/iscsi start
iscsiadm -m discovery -t st -p 192.168.0.4
/etc/init.d/iscsi restart
mkqdisk -c /dev/sda -l myqdisk （在一臺node上操作即可）
五、使用system-config-cluster配置cluster
system-config-cluster
1.New Configuration頁
Choose a name for the cluster
clu_test 集羣名稱
Custom Configure Multicast
Address:225.225.225.225 廣播地址
Use a Quorum Disk
Interval: 1投票週期1秒
TKO:10 10次投票沒有成功
Votes:3 權重（大於n／2+1）
Minimum Score：1 最小分數1（只要有一分就可以工作）
Device： 設備文件名
Lable：myqdisk 設備標籤名
Program:ping 192.168.0.1 -c2 -W1 判斷是否應該投票的程序
Score:1 如果允許投票，每次投票的分數是1
Interval：2 間隔2秒投票一次
OK
2.Cluster Configuration頁
Cluster Nodes 集羣節點
Add a Cluster Node
Cluster Node Name:node1
OK
類似操作增加node2、node3、fence
Fence Devices 仲裁設備
Add a Fence Device
Global Network Block Device
Name:fd1
Servers:fence
OK
Cluster Nodes 節點關聯仲裁
node1
Manage Fencing For This Node
Add a New Fence Level
Fence-Level-1
Add a New Fence to this Level
OK
類似操作爲node2、node3，關聯fence
Managed Resources 設置資源
Fallover Domains
Create a Failover Domain
Name for new Failover Domain:fd1
OK
Resources
Create a Resource
IP Address
192.168.0.5 浮動IP
OK
Create a Resource
Script
Name:mysql
File:/etc/init.d/mysqld
OK
Services
Create a Service
Name sv1
OK
Failover Domain:fd1
Add a Shared Resource to this service
加入IP資源、mysql資源
Close
保存退出
3.配置文件發送到node上
for i in 1 2 3 ;do scp /etc/cluster/cluster.conf node$i:/etc/cluster/cluster.conf ;done
六、配置mysql，啓動集羣
1.fence上啓動相關服務
/etc/init.d/qdiskd start
/etc/init.d/cman start
gnbd_serv
2.在3臺node上啓動相關服務
/etc/init.d/qdiskd start
/etc/init.d/cman start
/etc/init.d/clvmd start
node1上用導入的mysql target創建一個lv0
3.檢查node2、node3上是否同步了lv0，否則重啓clvmd服務
4.格式化lv0（在一臺node上操作即可）
mkfs.gfs2 -j3 -p lock_dlm -t clu_test:mysql /dev/vg0/lv0
5.修改mysqld腳本文件
vim /etc/init.d/mysqld
將
status mysqld >/dev/null >&1
if [ $? -eq 3 ] ; then
return 0
fi
加在 stop()函數的起始位置。
6.三臺node上掛載lv0，啓動rgmanager服務
mount /dev/vg0/lv0 /var/lib/mysql
/etc/init.d/rgmanager start