MHA的維護操作
在線切換功能
只切換角色
masterha_master_switch --conf=/etc/mha/app1.cnf --master_state=alive --new_master_host=10.0.0.51 --orig_master_is_new_slave --running_updates_limit=10000
注意
master_ip_online_change_script is not defined. If you do not disable writes on the current master manually, applications keep writing on the current master. Is it ok to proceed? (yes/NO): yes
1、此種辦法切換,要注意將原主庫,ftwrl(flush table with read lock),否則會造成主從不一致。
2、手工切換vip
3、重新拉取新主庫的binlog
master_ip_online_change_script功能實現
功能: 在線切換時,自動鎖原主庫,vip自動切換
1、準備切換腳本
vim /usr/local/bin/master_ip_online_change
my $vip = "10.0.0.55/24";
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig ens33:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig ens33:$key $vip down";
my $ssh_Bcast_arp= "/sbin/arping -I ens33 -c 3 -A 10.0.0.55";
2、修改MHA配置文件
將腳本加載到配置文件中 使得可以自動執行腳本 實現vip自動切換
vim /etc/mha/app1.cnf
[server default]
manager_log=/var/log/mha/app1/manager
master_ip_online_change_script=/usr/local/bin/master_ip_online_change
注意放置的位置,防止的位置是在server default的標籤下。
3、停止MHA
[root@db03 bin]# masterha_stop --conf=/etc/mha/app1.cnf
很多MHA的命令(也就是腳本)是需要加–conf=/etc/mha/app1.cnf的。
4、檢查repl
檢查配置文件是否寫錯,是否配置成功
[root@db03 bin]# masterha_check_repl --conf=/etc/mha/app1.cnf
5、在線切換
masterha_master_switch --conf=/etc/mha/app1.cnf --master_state=alive --new_master_host=10.0.0.51 --orig_master_is_new_slave --running_updates_limit=10000
參數解釋
masterha_master_switch 腳本名稱
–conf=/etc/mha/app1.cnf 指定MHA的配置文件
–master_state=alive 選存活的
–new_master_host=10.0.0.51 指定主庫ip地址
–orig_master_is_new_slave 作爲新主
–running_updates_limit=10000 如果網絡延遲超過10000則不選爲主庫 可以理解爲判斷指定的主庫是否可以勝任。
6、重構binlogserver
檢查是否還在拉取日誌
[root@db03 bin]# ps -ef |grep mysqlbinlog
root 28144 16272 0 17:50 pts/1 00:00:00 mysqlbinlog -R --host=10.0.0.52 --user=mha --password=x x --raw --stop-never mysql-bin.000005
root 28529 16272 0 18:03 pts/1 00:00:00 grep --color=auto mysqlbinlog
如果有之前的拉取日誌的進程,殺掉
[root@db03 bin]# kill -9 28144
重新拉取日誌前需要將原binlogserver目錄下的日誌給清除乾淨 ,再重新拉取
[root@db03 bin]# cd /data/binlog_server/
[root@db03 binlog_server]# ll
total 4
-rw-r----- 1 root root 194 Apr 1 17:50 mysql-bin.000005
[root@db03 binlog_server]# rm -rf *
重新拉取新主庫的binlog日誌 注意在主庫show
master status 查看一下正在使用的日誌。
[root@db03 binlog_server]# mysqlbinlog -R --host=10.0.0.51 --user=mha --password=mha --raw --stop-never mysql-bin.000009 &
[1] 28534
7、重新啓動MHA
[root@db03 bin]# nohup masterha_manager --conf=/etc/mha/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null> /var/log/mha/app1/manager.log 2>&1 &
重啓後檢查MHA狀態
[root@db03 binlog_server]# masterha_check_status --conf=/etc/mha/app1.cnf
app1 (pid:28535) is running(0:PING_OK), master:10.0.0.51
Altas 讀寫分離
架構圖
介紹:
Atlas是由 Qihoo 360, Web平臺部基礎架構團隊開發維護的一個基於MySQL協議的數據中間層項目。
它是在mysql-proxy 0.8.2版本的基礎上,對其進行了優化,增加了一些新的功能特性。
360內部使用Atlas運行的mysql業務,每天承載的讀寫請求數達幾十億條。
下載地址
https://github.com/Qihoo360/Atlas/releases
注意:
1、Atlas 只能安裝運行在64位的系統上
2、centos 5.x安裝Atlas-xx.e15.x86_64.rpm centos6.x 安裝atlas-xx.e16.x86_64.rpm
3、後端mysql版本應該大於5.1 建議使用mysql5.6以上。
安裝配置
yum localinstall -y Atlas*(先上傳rpm包,再本地方式yum安裝)
cd /usr/local/mysql-proxy/conf
mv test.cnf test.cnf.bak
vi test.cnf
[mysql-proxy]
admin-username = user 管理用戶
admin-password = pwd 管理用戶的密碼
proxy-backend-addresses = 10.0.0.55:3306 vip登錄mysql地址以及端口號
proxy-read-only-backend-addresses = 10.0.0.51:3306,10.0.0.53:3306 從節點的mysql信息
pwds = repl:3yb5jEku5h4=,mha:O2jBXONX098= 登錄用戶的信息
daemon = true
keepalive = true
event-threads = 8
log-level = message
log-path = /usr/local/mysql-proxy/log 日誌信息
sql-log=ON
proxy-address = 0.0.0.0:33060 登錄操作的進程端口號
admin-address = 0.0.0.0:2345 管理用戶登錄端口號
charset=utf8 指定字符集
啓動atlas以及查看進程信息
/usr/local/mysql-proxy/bin/mysql-proxyd test start
ps -ef |grep proxy
配置文件參數說明
#帶#號的爲非必需的配置項目
#管理接口的用戶名
admin-username=usrname
#管理接口的密碼
admin-password=password
#Atlas後端連接的MySQL主庫的IP和端口,可設置多項,用逗號分隔
proxy-backend-addresses=192.168.1.200:3306
#Atlas後端連接的MySQL從庫的IP和端口,@後面的數字代表權重,用來作負載均衡,若省略則默認爲1,可設置多項,用逗號分隔
proxy-read-only-backend-addresses = 192.168.1.201:3306@2,192.168.1.202:3306@1
#用戶名與其對應的加密過的MySQL密碼,密碼使用PREFIX/bin目錄下的加密程序encrypt加密,下行的user1和user2爲示例,將其替換爲你的MySQL的用戶名和加密密碼!
pwds=username:encrypt(password)
#設置Atlas的運行方式,設爲true時爲守護進程方式,設爲false時爲前臺方式,一般開發調試時設爲false,線上運行時設爲true,true後面不能有空格。
daemon=true
#設置Atlas的運行方式,設爲true時Atlas會啓動兩個進程,一個爲monitor,一個爲worker,monitor在worker意外退出後會自動將其重啓,設爲false時只有worker,沒有monitor,一般開發調試時設爲false,線上運行>時設爲true,true後面不能有空格。
keepalive=true
#工作線程數,對Atlas的性能有很大影響,可根據情況適當設置
event-threads=4
#日誌級別,分爲message、warning、critical、error、debug五個級別
log-level=message
#日誌存放的路徑
log-path=/var/logs/mysql-proxy
#SQL日誌的開關,可設置爲OFF、ON、REALTIME,OFF代表不記錄SQL日誌,ON代表記錄SQL日誌,REALTIME代表記錄SQL日誌且實時寫入磁盤,默認爲OFF
#sql-log = OFF
#慢日誌輸出設置。當設置了該參數時,則日誌只輸出執行時間超過sql-log-slow(單位:ms)的日誌記錄。不設置該參數則輸出全部日誌。
#sql-log-slow = 10
#實例名稱,用於同一臺機器上多個Atlas實例間的區分
#instance = database
#Atlas監聽的工作接口IP和端口
proxy-address=127.0.0.1:3301
#Atlas監聽的管理接口IP和端口
admin-address=127.0.0.1:3302
#分表設置,此例中person爲庫名,mt爲表名,id爲分表字段,3爲子表數量,可設置多項,以逗號分隔,若不分表則不需要設置該項
#tables = person.mt.id.3
#默認字符集,設置該項後客戶端不再需要執行SET NAMES語句
#charset = utf8
#允許連接Atlas的客戶端的IP,可以是精確IP,也可以是IP段,以逗號分隔,若不設置該項則允許所有IP連接,否則只允許列表中的IP連接
client-ips=127.0.0.1,192.168.1.201,192.168.1.202
#Atlas前面掛接的LVS的物理網卡的IP(注意不是虛IP),若有LVS且設置了client-ips則此項必須設置,否則可以不設置
#lvs-ips = 192.168.1.1
Atlas功能測試
測試讀操作:
mysql -umha -pmha -h 10.0.0.53 -P 33060 使用普通用戶方式登錄 進行操作測試
db03 [(none)]>select @@server_id;
測試寫操作:
mysql> begin;select @@server_id;commit;
注意:
DDL建議不要再Atlas觸發,最好是到主庫觸發(onlineDDL或者 PT-OSC)
DML建議begin;DML;commit;
Atlas的管理操作
使用管理用戶登錄
[root@db03 conf]# mysql -uuser -ppwd -h 10.0.0.53 -P2345
查看幫助命令
db03 [(none)]>select * from help;
查看所有節點
db03 [(none)]>SELECT * FROM backends;
+-------------+----------------+-------+------+
| backend_ndx | address | state | type |
+-------------+----------------+-------+------+
| 1 | 10.0.0.55:3306 | up | rw |
| 2 | 10.0.0.52:3306 | up | ro |
| 3 | 10.0.0.53:3306 | up | ro |
+-------------+----------------+-------+------+
3 rows in set (0.00 sec)
節點的上線和下線
db03 [(none)]>SET OFFLINE 1; 指定id爲1的節點下線
+-------------+----------------+---------+------+
| backend_ndx | address | state | type |
+-------------+----------------+---------+------+
| 1 | 10.0.0.55:3306 | offline | rw |
+-------------+----------------+---------+------+
1 row in set (0.01 sec)
查看所有節點狀態
db03 [(none)]>SELECT * FROM backends;
+-------------+----------------+---------+------+
| backend_ndx | address | state | type |
+-------------+----------------+---------+------+
| 1 | 10.0.0.55:3306 | offline | rw |
| 2 | 10.0.0.52:3306 | up | ro |
| 3 | 10.0.0.53:3306 | up | ro |
+-------------+----------------+---------+------+
db03 [(none)]>SET ONLINE 1; 設置id爲1的節點上線
+-------------+----------------+---------+------+
| backend_ndx | address | state | type |
+-------------+----------------+---------+------+
| 1 | 10.0.0.55:3306 | unknown | rw |
+-------------+----------------+---------+------+
刪除和添加節點
db03 [(none)]>REMOVE BACKEND 3; 刪除節點
db03 [(none)]>ADD SLAVE 10.0.0.53:3306; 添加節點
用戶管理
注意:如果是添加管理用戶 需要現在節點上創建該用戶 然後在Atlas上進行創建
注意順序問題:先在主節點創建(主從複製也會使得從庫執行),然後在創建Atlas用戶
db01 [(none)]>grant all on *.* to oldguo@'10.0.0.%' identified by '123';
db03 [(none)]>SELECT * FROM pwds;
db03 [(none)]>add pwd oldguo:123;
如果順序不對 或者在節點中沒有創建用戶,只是在Atlas上創建了用戶,那麼使用這個用戶登錄進行操作時會出現問題:
使配置永久生效,寫入配置文件中
db03 [(none)]>save config;
分佈式結構mycat
基礎架構圖
1、準備環境
兩臺虛擬機 db01 db02
每臺創建四個mysql實例:3307 3308 3309 3310
2、刪除歷史環境
pkill mysqld
rm -rf /data/330*
mv /etc/my.cnf /etc/my.cnf.bak
3、創建相關目錄並初始化數據
mkdir /data/33{07..10}/data -p
mysqld --initialize-insecure --user=mysql --datadir=/data/3307/data --basedir=/data/app/mysql
mysqld --initialize-insecure --user=mysql --datadir=/data/3308/data --basedir=/data/app/mysql
mysqld --initialize-insecure --user=mysql --datadir=/data/3309/data --basedir=/data/app/mysql
mysqld --initialize-insecure --user=mysql --datadir=/data/3310/data --basedir=/data/app/mysql
4、準備配置文件和啓動腳本
db01節點
cat >/data/3307/my.cnf<<EOF
[mysqld]
basedir=/data/app/mysql
datadir=/data/3307/data
socket=/data/3307/mysql.sock
port=3307
log-error=/data/3307/mysql.log
log_bin=/data/3307/mysql-bin
binlog_format=row
skip-name-resolve
server-id=7
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
EOF
cat >/data/3308/my.cnf<<EOF
[mysqld]
basedir=/data/app/mysql
datadir=/data/3308/data
port=3308
socket=/data/3308/mysql.sock
log-error=/data/3308/mysql.log
log_bin=/data/3308/mysql-bin
binlog_format=row
skip-name-resolve
server-id=8
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
EOF
cat >/data/3309/my.cnf<<EOF
[mysqld]
basedir=/data/app/mysql
datadir=/data/3309/data
socket=/data/3309/mysql.sock
port=3309
log-error=/data/3309/mysql.log
log_bin=/data/3309/mysql-bin
binlog_format=row
skip-name-resolve
server-id=9
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
EOF
cat >/data/3310/my.cnf<<EOF
[mysqld]
basedir=/data/app/mysql
datadir=/data/3310/data
socket=/data/3310/mysql.sock
port=3310
log-error=/data/3310/mysql.log
log_bin=/data/3310/mysql-bin
binlog_format=row
skip-name-resolve
server-id=10
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
EOF
cat >/etc/systemd/system/mysqld3307.service<<EOF
[Unit]
Description=MySQL Server
Documentation=man:mysqld(8)
Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html
After=network.target
After=syslog.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
ExecStart=/data/app/mysql/bin/mysqld --defaults-file=/data/3307/my.cnf
LimitNOFILE = 5000
EOF
cat >/etc/systemd/system/mysqld3308.service<<EOF
[Unit]
Description=MySQL Server
Documentation=man:mysqld(8)
Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html
After=network.target
After=syslog.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
ExecStart=/data/app/mysql/bin/mysqld --defaults-file=/data/3308/my.cnf
LimitNOFILE = 5000
EOF
cat >/etc/systemd/system/mysqld3309.service<<EOF
[Unit]
Description=MySQL Server
Documentation=man:mysqld(8)
Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html
After=network.target
After=syslog.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
ExecStart=/data/app/mysql/bin/mysqld --defaults-file=/data/3309/my.cnf
LimitNOFILE = 5000
EOF
cat >/etc/systemd/system/mysqld3310.service<<EOF
[Unit]
Description=MySQL Server
Documentation=man:mysqld(8)
Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html
After=network.target
After=syslog.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
ExecStart=/data/app/mysql/bin/mysqld --defaults-file=/data/3310/my.cnf
LimitNOFILE = 5000
EOF
db02節點
cat >/data/3307/my.cnf<<EOF
[mysqld]
basedir=/data/app/mysql
datadir=/data/3307/data
socket=/data/3307/mysql.sock
port=3307
log-error=/data/3307/mysql.log
log_bin=/data/3307/mysql-bin
binlog_format=row
skip-name-resolve
server-id=17
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
EOF
cat >/data/3308/my.cnf<<EOF
[mysqld]
basedir=/data/app/mysql
datadir=/data/3308/data
port=3308
socket=/data/3308/mysql.sock
log-error=/data/3308/mysql.log
log_bin=/data/3308/mysql-bin
binlog_format=row
skip-name-resolve
server-id=18
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
EOF
cat >/data/3309/my.cnf<<EOF
[mysqld]
basedir=/data/app/mysql
datadir=/data/3309/data
socket=/data/3309/mysql.sock
port=3309
log-error=/data/3309/mysql.log
log_bin=/data/3309/mysql-bin
binlog_format=row
skip-name-resolve
server-id=19
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
EOF
cat >/data/3310/my.cnf<<EOF
[mysqld]
basedir=/data/app/mysql
datadir=/data/3310/data
socket=/data/3310/mysql.sock
port=3310
log-error=/data/3310/mysql.log
log_bin=/data/3310/mysql-bin
binlog_format=row
skip-name-resolve
server-id=20
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
EOF
cat >/etc/systemd/system/mysqld3307.service<<EOF
[Unit]
Description=MySQL Server
Documentation=man:mysqld(8)
Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html
After=network.target
After=syslog.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
ExecStart=/data/app/mysql/bin/mysqld --defaults-file=/data/3307/my.cnf
LimitNOFILE = 5000
EOF
cat >/etc/systemd/system/mysqld3308.service<<EOF
[Unit]
Description=MySQL Server
Documentation=man:mysqld(8)
Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html
After=network.target
After=syslog.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
ExecStart=/data/app/mysql/bin/mysqld --defaults-file=/data/3308/my.cnf
LimitNOFILE = 5000
EOF
cat >/etc/systemd/system/mysqld3309.service<<EOF
[Unit]
Description=MySQL Server
Documentation=man:mysqld(8)
Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html
After=network.target
After=syslog.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
ExecStart=/data/app/mysql/bin/mysqld --defaults-file=/data/3309/my.cnf
LimitNOFILE = 5000
EOF
cat >/etc/systemd/system/mysqld3310.service<<EOF
[Unit]
Description=MySQL Server
Documentation=man:mysqld(8)
Documentation=http://dev.mysql.com/doc/refman/en/using-systemd.html
After=network.target
After=syslog.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
ExecStart=/data/app/mysql/bin/mysqld --defaults-file=/data/3310/my.cnf
LimitNOFILE = 5000
EOF
5、修改權限,啓動多實例
chown -R mysql.mysql /data/*
systemctl start mysqld3307
systemctl start mysqld3308
systemctl start mysqld3309
systemctl start mysqld3310
mysql -S /data/3307/mysql.sock -e "show variables like 'server_id'"
mysql -S /data/3308/mysql.sock -e "show variables like 'server_id'"
mysql -S /data/3309/mysql.sock -e "show variables like 'server_id'"
mysql -S /data/3310/mysql.sock -e "show variables like 'server_id'"
6、節點主從規劃
箭頭指向誰是主庫
10.0.0.51:3307 <-----> 10.0.0.52:3307
10.0.0.51:3309 ------> 10.0.0.51:3307
10.0.0.52:3309 ------> 10.0.0.52:3307
10.0.0.52:3308 <-----> 10.0.0.51:3308
10.0.0.52:3310 -----> 10.0.0.52:3308
10.0.0.51:3310 -----> 10.0.0.51:3308
分片規劃
shard1:
Master:10.0.0.51:3307
slave1:10.0.0.51:3309
Standby Master:10.0.0.52:3307
slave2:10.0.0.52:3309
shard2:
Master:10.0.0.52:3308
slave1:10.0.0.52:3310
Standby Master:10.0.0.51:3308
slave2:10.0.0.51:3310
2.8 開始配置主從環境
配置db01節點和db02節點互爲主從庫
#shard1
##10.0.0.51:3307 <-----> 10.0.0.52:3307
db02
在db02節點登錄3307創建兩個用戶
mysql -S /data/3307/mysql.sock -e "grant replication slave on *.* to repl@'10.0.0.%' identified by '123';"
mysql -S /data/3307/mysql.sock -e "grant all on *.* to root@'10.0.0.%' identified by '123' with grant option;"
db01
在db01節點登錄3307 輸入db02爲主庫信息
mysql -S /data/3307/mysql.sock -e "CHANGE MASTER TO MASTER_HOST='10.0.0.52', MASTER_PORT=3307, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';"
開啓從庫並查看主從信息
mysql -S /data/3307/mysql.sock -e "start slave;"
mysql -S /data/3307/mysql.sock -e "show slave status\G"
db02節點
在db02節點輸入主庫信息
mysql -S /data/3307/mysql.sock -e "CHANGE MASTER TO MASTER_HOST='10.0.0.51', MASTER_PORT=3307, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';"
並查看主從信息
mysql -S /data/3307/mysql.sock -e "start slave;"
mysql -S /data/3307/mysql.sock -e "show slave status\G"
配置51節點的3307爲51節點3309的主庫
10.0.0.51:3309 ------> 10.0.0.51:3307
db01
輸入配置信息並檢測
mysql -S /data/3309/mysql.sock -e "CHANGE MASTER TO MASTER_HOST='10.0.0.51', MASTER_PORT=3307, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';"
mysql -S /data/3309/mysql.sock -e "start slave;"
mysql -S /data/3309/mysql.sock -e "show slave status\G"
配置52節點的3309 爲52節點的3307從庫
10.0.0.52:3309 ------> 10.0.0.52:3307
mysql -S /data/3309/mysql.sock -e "CHANGE MASTER TO MASTER_HOST='10.0.0.52', MASTER_PORT=3307, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';"
mysql -S /data/3309/mysql.sock -e "start slave;"
mysql -S /data/3309/mysql.sock -e "show slave status\G"
10.0.0.52:3308 <-----> 10.0.0.51:3308
db01
mysql -S /data/3308/mysql.sock -e "grant replication slave on *.* to repl@'10.0.0.%' identified by '123';"
mysql -S /data/3308/mysql.sock -e "grant all on *.* to root@'10.0.0.%' identified by '123' with grant option;"
db02
mysql -S /data/3308/mysql.sock -e "CHANGE MASTER TO MASTER_HOST='10.0.0.51', MASTER_PORT=3308, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';"
mysql -S /data/3308/mysql.sock -e "start slave;"
mysql -S /data/3308/mysql.sock -e "show slave status\G"
db01
mysql -S /data/3308/mysql.sock -e "CHANGE MASTER TO MASTER_HOST='10.0.0.52', MASTER_PORT=3308, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';"
mysql -S /data/3308/mysql.sock -e "start slave;"
mysql -S /data/3308/mysql.sock -e "show slave status\G"
10.0.0.52:3310 -----> 10.0.0.52:3308
#db02
mysql -S /data/3310/mysql.sock -e "CHANGE MASTER TO MASTER_HOST='10.0.0.52', MASTER_PORT=3308, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';"
mysql -S /data/3310/mysql.sock -e "start slave;"
mysql -S /data/3310/mysql.sock -e "show slave status\G"
10.0.0.51:3310 -----> 10.0.0.51:3308
#db01
mysql -S /data/3310/mysql.sock -e "CHANGE MASTER TO MASTER_HOST='10.0.0.51', MASTER_PORT=3308, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';"
mysql -S /data/3310/mysql.sock -e "start slave;"
mysql -S /data/3310/mysql.sock -e "show slave status\G"
檢測主從狀態
mysql -S /data/3307/mysql.sock -e "show slave status\G"|grep Yes
mysql -S /data/3308/mysql.sock -e "show slave status\G"|grep Yes
mysql -S /data/3309/mysql.sock -e "show slave status\G"|grep Yes
mysql -S /data/3310/mysql.sock -e "show slave status\G"|grep Yes
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
注:如果中間出現錯誤,在每個節點進行執行以下命令,從2.8從頭執行
mysql -S /data/3307/mysql.sock -e “stop slave; reset slave all;”
mysql -S /data/3308/mysql.sock -e “stop slave; reset slave all;”
mysql -S /data/3309/mysql.sock -e “stop slave; reset slave all;”
mysql -S /data/3310/mysql.sock -e “stop slave; reset slave all;”
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
mycat的安裝
1、預先安裝java運行環境
yum install -y java
2、下載
Mycat-server-xxxxx.linux.tar.gz
http://dl.mycat.io/
3、解壓文件
tar xf Mycat-server-*
4、軟件目錄
ls
bin catlet conf lib logs version.txt
5、啓動和連接
配置環境變量
vim /etc/profile
export PATH=/data/app/mycat/bin:$PATH
source /etc/profile
啓動
mycat start
連接mycat
mysql -uroot -p123456 -h 127.0.0.1 -P8066
配置文件介紹
bin目錄 程序目錄
conf 配置文件目錄
schema.xml 主配置文件:節點信息、讀寫分離、高可用設置、調用分片策略...
rule.xml 分片策略的定義、功能、使用方法
server.xml mycat服務有關配置:用戶、網絡、權限、策略、資源等
xx.txt文件 分片參數定義文件
log4j2.xml Mycat 相關日誌記錄配置
logs
wrapper.log 啓動日誌
mycat.log 工作日誌
mycat 核心應用
schema.xml配置文件結構
[root@db01 conf]# cd /data/mycat/conf/
mv schema.xml schema.xml.bak
vim schema.xml
#邏輯庫:
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
</schema>
#DN數據節點(邏輯分片):數據節點(邏輯分片):
<dataNode name="dn1" dataHost="localhost1" database= "world" />
作用:
垂直和水平查分。
# DH 數據主機
作用: 高可用和讀寫分離
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="1">
<heartbeat>select user()</heartbeat>
<writeHost host="db1" url="10.0.0.51:3307" user="root" password="123">
<readHost host="db2" url="10.0.0.51:3309" user="root" password="123" />
</writeHost>
</dataHost>
</mycat:schema>
<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
</schema>
<dataNode name="dn1" dataHost="localhost1" database= "world" />
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="1">
<heartbeat>select user()</heartbeat>
<writeHost host="db1" url="10.0.0.51:3307" user="root" password="123">
<readHost host="db2" url="10.0.0.51:3309" user="root" password="123" />
</writeHost>
</dataHost>
</mycat:schema>
測試環境準備
db01:
mysql -S /data/3307/mysql.sock
grant all on *.* to root@'10.0.0.%' identified by '123';
source /root/world.sql
mysql -S /data/3308/mysql.sock
grant all on *.* to root@'10.0.0.%' identified by '123';
source /root/world.sql
重啓mycat
mycat restart
讀寫分離測試
連接mycat服務
mysql -uroot -p123456 -h 10.0.0.51 -P8066
測試讀
select @@server_id;
測試寫
begin;select @@server_id; commit;
配置讀寫分離以及高可用
[root@db01 conf]# mv schema.xml schema.xml.rw
[root@db01 conf]# vim schema.xml
<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="sh1">
</schema>
<dataNode name="sh1" dataHost="oldguo1" database= "world" />
<dataHost name="oldguo1" maxCon="1000" minCon="10" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="1">
<heartbeat>select user()</heartbeat>
<writeHost host="db1" url="10.0.0.51:3307" user="root" password="123">
<readHost host="db2" url="10.0.0.51:3309" user="root" password="123" />
</writeHost>
<writeHost host="db3" url="10.0.0.52:3307" user="root" password="123">
<readHost host="db4" url="10.0.0.52:3309" user="root" password="123" />
</writeHost>
</dataHost>
</mycat:schema>
正真的writehost :負責寫操作的writehost
standby writehost :和readhost一樣 只提供讀服務
當寫節點宕機後,後面跟的readhost也不提供服務,這個時候standby的writehost就提供寫服務。
後面跟的readhost提供讀服務
測試讀寫分離
mycat restart
mysql -uroot -p123456 -h 10.0.0.51 -P 8066
mysql> select @@server_id;
mysql> begin;select @@server_id;commit;
測試高可用
[root@db01 conf]# systemctl stop mysqld3307
mysql -uroot -p123456 -h 10.0.0.51 -P 8066
mysql> select @@server_id;
mysql> begin;select @@server_id;commit;
[root@db01 conf]# systemctl start mysqld3307
mysql -uroot -p123456 -h 10.0.0.51 -P 8066
mysql> select @@server_id;
mysql> begin;select @@server_id;commit;
參數介紹
balance屬性
讀操作負載均衡類型,目前的取值有三種:
- balance=“0”, 不開啓讀寫分離機制,所有讀操作都發送到當前可用的writeHost上。
- balance=“1”,全部的readHost與standby writeHost參與select語句的負載均衡,簡單的說,
當雙主雙從模式(M1->S1,M2->S2,並且M1與 M2互爲主備),正常情況下,M2,S1,S2都參與select語句的負載均衡。 - balance=“2”,所有讀操作都隨機的在writeHost、readhost上分發。
writeType屬性
寫操作,負載均衡類型,目前的取值有2種:
- writeType=“0”, 所有寫操作發送到配置的第一個writeHost,
第一個掛了切到還生存的第二個writeHost,重新啓動後已切換後的爲主,切換記錄在配置文件中:dnindex.properties . - writeType=“1”,所有寫操作都隨機的發送到配置的writeHost,但不推薦使用
switchType屬性
-1 表示不自動切換
1 默認值,自動切換
2 基於MySQL主從同步的狀態決定是否切換 ,心跳語句爲 show slave status
datahost其他配置
連接有關
maxCon=“1000”:最大的併發連接數
minCon=“10” :mycat在啓動之後,會在後端節點上自動開啓的連接線程
tempReadHostAvailable="1"
這個一主一從時(1個writehost,1個readhost時),可以開啓這個參數,如果2個writehost,2個readhost時
select user() 監測心跳
Mycat 分佈式架構–垂直分表
cd /data/app/mycat/conf
mv schema.xml schema.xml.ha
vim schema.xml
<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="sh1">
<table name="user" dataNode="sh1"/>
<table name="order_t" dataNode="sh2"/>
</schema>
<dataNode name="sh1" dataHost="oldguo1" database= "taobao" />
<dataNode name="sh2" dataHost="oldguo2" database= "taobao" />
<dataHost name="oldguo1" maxCon="1000" minCon="10" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="1">
<heartbeat>select user()</heartbeat>
<writeHost host="db1" url="10.0.0.51:3307" user="root" password="123">
<readHost host="db2" url="10.0.0.51:3309" user="root" password="123" />
</writeHost>
<writeHost host="db3" url="10.0.0.52:3307" user="root" password="123">
<readHost host="db4" url="10.0.0.52:3309" user="root" password="123" />
</writeHost>
</dataHost>
<dataHost name="oldguo2" maxCon="1000" minCon="10" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="1">
<heartbeat>select user()</heartbeat>
<writeHost host="db1" url="10.0.0.51:3308" user="root" password="123">
<readHost host="db2" url="10.0.0.51:3310" user="root" password="123" />
</writeHost>
<writeHost host="db3" url="10.0.0.52:3308" user="root" password="123">
<readHost host="db4" url="10.0.0.52:3310" user="root" password="123" />
</writeHost>
</dataHost>
</mycat:schema>
創建測試庫和表:
mysql -S /data/3307/mysql.sock -e "create database taobao charset utf8;"
mysql -S /data/3308/mysql.sock -e "create database taobao charset utf8;"
mysql -S /data/3307/mysql.sock -e "use taobao;create table user(id int,name varchar(20))";
mysql -S /data/3308/mysql.sock -e "use taobao;create table order_t(id int,name varchar(20))"
重啓mycat
mycat restart
mycat中對user 和 order 數據插入
[root@db01 conf]# mysql -uroot -p123456 -h 10.0.0.51 -P 8066
mysql> insert into user values(1,'a');
mysql> insert into user values(2,'b');
mysql> insert into user values(3,'c');
mysql> commit;
mysql> insert into order_t values(1,'x'),(2,'y');
mysql> commit;
[root@db01 conf]# mysql -S /data/3307/mysql.sock -e "show tables from taobao"
+------------------+
| Tables_in_taobao |
+------------------+
| user |
+------------------+
[root@db01 conf]# mysql -S /data/3308/mysql.sock -e "show tables from taobao"
+------------------+
| Tables_in_taobao |
+------------------+
| order_t |
+------------------+
[root@db01 conf]# mysql -S /data/3307/mysql.sock -e "select * from taobao.user"
+------+------+
| id | name |
+------+------+
| 1 | a |
| 2 | b |
| 3 | c |
+------+------+
[root@db01 conf]# mysql -S /data/3308/mysql.sock -e "select * from taobao.order_t"
+------+------+
| id | name |
+------+------+
| 1 | x |
| 2 | y |
+------+------+
[root@db01 conf]#
mycat分佈式架構–水平拆分
重要概念
分片策略:幾乎融合經典業務中大部分的分片策略。mycat已經開發了相應的算法,非常方便調用。
範圍分片
取模
枚舉
日期
HASH
等
分片鍵:作爲分片條件的列
範圍分片
比如說t3表
(1)行數非常多,2000w(1-1000w:sh1 1000w01-2000w:sh2)
(2)訪問非常頻繁,用戶訪問較離散
1、修改schema.xml文件,定製分片策略
cp schema.xml schema.xml.1
vim schema.xml
添加:
<table name="t3" dataNode="sh1,sh2" rule="auto-sharding-long" />
2、定義和使用分片策略
vim rule.xml
<tableRule name="auto-sharding-long">
<rule>
<columns>id</columns>
<algorithm>rang-long</algorithm>
</rule>
<function name="rang-long"
class="io.mycat.route.function.AutoPartitionByLong">
<property name="mapFile">autopartition-long.txt</property>
</function>
3、定義範圍
(爲了測試方便)
vim autopartition-long.txt
0-10=0
10-20=1
4、創建測試表
mysql -S /data/3307/mysql.sock -e "use taobao;create table t3 (id int not null primary key auto_increment,name varchar(20) not null);"
mysql -S /data/3308/mysql.sock -e "use taobao;create table t3 (id int not null primary key auto_increment,name varchar(20) not null);"
5、測試
重啓mycat
mycat restart
mysql -uroot -p123456 -h 10.0.0.51 -P 8066
insert into t3(id,name) values(1,'a');
insert into t3(id,name) values(2,'b');
insert into t3(id,name) values(3,'c');
insert into t3(id,name) values(4,'d');
insert into t3(id,name) values(11,'aa');
insert into t3(id,name) values(12,'bb');
insert into t3(id,name) values(13,'cc');
insert into t3(id,name) values(14,'dd');
[root@db01 conf]# mysql -S /data/3308/mysql.sock -e "select * from taobao.t3"
[root@db01 conf]# mysql -S /data/3307/mysql.sock -e "select * from taobao.t3"
取模分片
取餘分片方式:分片鍵(一個列)與節點數量進行取餘,得到餘數,將數據寫入對應節點
1、修改配置文件
vim schema.xml
<table name="t4" dataNode="sh1,sh2" rule="mod-long" />
2、查看和定義分片使用
vim rule.xml
<property name="count">2</property>
3、準備測試環境
創建測試表:
mysql -S /data/3307/mysql.sock -e "use taobao;create table t4 (id int not null primary key auto_increment,name varchar(20) not null);"
mysql -S /data/3308/mysql.sock -e "use taobao;create table t4 (id int not null primary key auto_increment,name varchar(20) not null);"
重啓mycat
mycat restart
4、測試
mysql -uroot -p123456 -h10.0.0.51 -P8066
use TESTDB
insert into t4(id,name) values(1,'a');
insert into t4(id,name) values(2,'b');
insert into t4(id,name) values(3,'c');
insert into t4(id,name) values(4,'d');
insert into t4(id,name) values(6,'x'),(8,'y'),(10,'z');
分別登錄後端節點查詢數據
mysql -S /data/3308/mysql.sock -e "select * from taobao.t4"
mysql -S /data/3307/mysql.sock -e "select * from taobao.t4"
枚舉分片
t5 表
id name telnum
1 bj 1212
2 sh 22222
3 bj 3333
4 sh 44444
5 bj 5555
sharding-by-intfile
1、 設計分片策略
vim schema.xml
2、應用分片策略
vim rule.xml
<tableRule name="sharding-by-intfile">
<rule>
<columns>name</columns>
<algorithm>hash-int</algorithm>
</rule>
</tableRule>
<function name="hash-int" class="org.opencloudb.route.function.PartitionByFileMap">
<property name="mapFile">partition-hash-int.txt</property>
<property name="type">1</property>
</function>
vim partition-hash-int.txt 配置:
bj=0
sh=1
DEFAULT_NODE=1
3、準備測試環境
mysql -S /data/3307/mysql.sock -e "use taobao;create table t5 (id int not null primary key auto_increment,name varchar(20) not null);"
mysql -S /data/3308/mysql.sock -e "use taobao;create table t5 (id int not null primary key auto_increment,name varchar(20) not null);"
重啓mycat
mycat restart
4、插入測試數據:
mysql -uroot -p123456 -h10.0.0.51 -P8066
use TESTDB
insert into t5(id,name) values(1,'bj');
insert into t5(id,name) values(2,'sh');
insert into t5(id,name) values(3,'bj');
insert into t5(id,name) values(4,'sh');
insert into t5(id,name) values(5,'tj');
mysql -S /data/3308/mysql.sock -e "select * from taobao.t5"
mysql -S /data/3307/mysql.sock -e "select * from taobao.t5"
mycat全局表
a b c d …
join
t
a
id name age
1 zs 18
2 ls 19
b
id addr aid
1001 bj 1
1002 sh 2
使用場景:
如果你的業務中有些數據類似於數據字典,比如配置文件的配置,
常用業務的配置或者數據量不大很少變動的表,這些表往往不是特別大,
而且大部分的業務場景都會用到,那麼這種表適合於Mycat全局表,無須對數據進行切分,
要在所有的分片上保存一份數據即可,Mycat 在Join操作中,業務表與全局表進行Join聚合會優先選擇相同分片內的全局表join,
避免跨庫Join,在進行數據插入操作時,mycat將把數據分發到全局表對應的所有分片執行,在進行數據讀取時候將會隨機獲取一個節點讀取數據。
1、設置全局表策略
vim schema.xml
<table name="t_area" primaryKey="id" type="global" dataNode="sh1,sh2" />
#2. 後端數據準備
mysql -S /data/3307/mysql.sock -e "use taobao;create table t_area (id int not null primary key auto_increment,name varchar(20) not null);"
mysql -S /data/3308/mysql.sock -e "use taobao;create table t_area (id int not null primary key auto_increment,name varchar(20) not null);"
重啓mycat
mycat restart
#3. 測試:
mysql -uroot -p123456 -h10.0.0.51 -P8066
use TESTDB
insert into t_area(id,name) values(1,'a');
insert into t_area(id,name) values(2,'b');
insert into t_area(id,name) values(3,'c');
insert into t_area(id,name) values(4,'d');
mysql -S /data/3308/mysql.sock -e "select * from taobao.t_area"
mysql -S /data/3307/mysql.sock -e "select * from taobao.t_area"
7.5 E-R分片
a
join
b
on a.xx =b.yy
爲了防止跨分片join,可以使用E-R模式
<table name="a" dataNode="sh1,sh2" rule="mod-long">
<childTable name="b" joinKey="aid" parentKey="id" />
</table>
a
id name
1 a
2 b
3 c
4 d
b
id addr aid
1001 bj 1
1002 sh 2
1003 tj 3
1004 wh 4
select * from a join b on a.id = b.aid where a.name=d
例子:
- 修改配置文件
vim schema.xml
<table name="a" dataNode="sh1,sh2" rule="mod-long_oldguo">
<childTable name="b" joinKey="aid" parentKey="id" />
</table>
- 修改rule.xml mod-log分片策略:
vim rule.xml
<tableRule name="mod-long_oldguo">
<rule>
<columns>id</columns>
<algorithm>mod-long_oldguo</algorithm>
</rule>
</tableRule>
<function name="mod-long_oldguo" class="io.mycat.route.function.PartitionByMod">
<!-- how many data nodes -->
<property name="count">2</property>
</function>
- 創建測試表
mysql -S /data/3307/mysql.sock -e "use taobao;create table a (id int not null primary key auto_increment,name varchar(20) not null);"
mysql -S /data/3307/mysql.sock -e "use taobao;create table b (id int not null primary key auto_increment,addr varchar(20) not null ,aid int );"
mysql -S /data/3308/mysql.sock -e "use taobao;create table a (id int not null primary key auto_increment,name varchar(20) not null);"
mysql -S /data/3308/mysql.sock -e "use taobao;create table b (id int not null primary key auto_increment,addr varchar(20) not null ,aid int );"
- 重啓mycat 測試
mycat restart
mysql -uroot -p123456 -h10.0.0.51 -P8066
use TESTDB
insert into a(id,name) values(1,'a');
insert into a(id,name) values(2,'b');
insert into a(id,name) values(3,'c');
insert into a(id,name) values(4,'d');
insert into a(id,name) values(5,'e');
insert into b(id,addr,aid) values(1001,'bj',1);
insert into b(id,addr,aid) values(1002,'sj',3);
insert into b(id,addr,aid) values(1003,'sd',4);
insert into b(id,addr,aid) values(1004,'we',2);
insert into b(id,addr,aid) values(1005,'er',5);
========
後端數據節點數據分佈:
mysql -S /data/3307/mysql.sock -e "select * from taobao.a"
mysql -S /data/3307/mysql.sock -e "select * from taobao.b"
mysql -S /data/3308/mysql.sock -e "select * from taobao.a"
mysql -S /data/3308/mysql.sock -e "select * from taobao.b"
- 管理類操作
[root@db01 conf]# mysql -uroot -p123456 -h10.0.0.51 -P9066
查看幫助
show @@help;
查看Mycat 服務情況
show @@server ;
查看分片信息
mysql> show @@datanode;
查看數據源
show @@datasource
重新加載配置信息
reload @@config : schema.xml
reload @@config_all : 所有配置重新加載
- 修改邏輯庫:
邏輯庫名
#總配置文件
schema.xml
<schema name="oldboy" checkSQLschema="false" sqlMaxLimit="100" dataNode="sh1"
#mycat 服務配置
server.xml
<user name="root" defaultAccount="true">
<property name="password">123456</property>
<property name="schemas">oldboy</property>
<property name="defaultSchema">oldboy</property>
<!--No MyCAT Database selected 錯誤前會嘗試使用該schema作爲schema,不設置則爲null,報錯 -->
<!-- 表級 DML 權限設置 -->
<!--
<privileges check="false">
<schema name="TESTDB" dml="0110" >
<table name="tb01" dml="0000"></table>
<table name="tb02" dml="1111"></table>
</schema>
</privileges>
-->
</user>
<user name="user">
<property name="password">user</property>
<property name="schemas">oldboy</property>
<property name="readOnly">true</property>
<property name="defaultSchema">oldboy</property>
</user>
reload @@config_all : 所有配置重新加載
- 添加一個邏輯庫
schema.xml
<schema name="oldguo" checkSQLschema="false" sqlMaxLimit="100" dataNode="sh1">
</schema>
server.xml
<property name="schemas">oldboy,oldguo</property>
mycat restart
課後練習:
<tableRule name="sharding-by-month">