1. 背景
* MySQL Replication默認都是異步（asynchronous），當主庫在執行完一些事務後，是不會管備庫的進度的。如果備庫不幸落後，而更不幸的是主庫此時又出現Crash（例如宕機），這時備庫中的數據就是不完整的。簡而言之，在主庫發生故障的時候，我們無法使用備庫來繼續提供數據一致的服務了。
* Semi sync Replication(半同步複製)是在master上提交完成後，再傳送到slave等待ack應答，僅僅在一定情況下事務的已經傳遞到一個slave上，但是並不確保已經在備庫上執行完成，會造成最後一次events的主備不一致。
* lossless replication(無損複製)是在master提前過程中，傳送到slave中等待應答。當至少一個slave request bilog後寫入到relay-log並flush disk，就返回ack
2. lossless replication傳輸過程
3. 環境
* master 實例環境
mysql> system cat /etc/redhat-release CentOS release 6.8 (Final) mysql> system ifconfig eth0 | sed -rn '2s#^.*addr:(.*) Bca.*$#\1#gp' 172.18.0.1 mysql> show variables like 'version'; +---------------+------------+ | Variable_name | Value | +---------------+------------+ | version | 5.7.18-log | +---------------+------------+ 1 row in set (0.00 sec)
* slave 實例環境
mysql> system cat /etc/redhat-release CentOS release 6.8 (Final) mysql> system ifconfig eth0 | sed -rn '2s#^.*addr:(.*) Bca.*$#\1#gp' 172.18.4.1 mysql> show variables like 'version'; +---------------+------------+ | Variable_name | Value | +---------------+------------+ | version | 5.7.18-log | +---------------+------------+ 1 row in set (0.00 sec)
* master 實例my.cnf文件
[mysqld] ########basic settings######## # 主從server-id一定要設置不同 server-id = 110 port = 3306 user = mysql bind_address = 0.0.0.0 character_set_server=utf8mb4 skip_name_resolve = 1 datadir = /data/mysql_data log_error = error.log #######replication settings######## master_info_repository = TABLE relay_log_info_repository = TABLE # MySQL複製是基於binlog日誌的 log_bin = bin.log sync_binlog = 1 log_slave_updates # MySQL binlog格式搭建主從時必須設置爲row binlog_format = row relay_log = relay.log relay_log_recovery = 1 slave_skip_errors = ddl_exist_errors ######semi sync replication settings######## # 設置插件目錄路徑 plugin_dir=/usr/local/mysql/lib/plugin # 加載插件 plugin_load = "rpl_semi_sync_master=semisync_master.so;rpl_semi_sync_slave=semisync_slave.so" # 開啓master semi sync replication rpl_semi_sync_master_enabled = 1 # 開啓slave semi sync replication rpl_semi_sync_slave_enabled = 1 # 等待5秒無ack應答自動切換爲異步模式 rpl_semi_sync_master_timeout = 5000 # 開啓lossless replication rpl_semi_sync_master_wait_point= AFTER_SYNC # 至少有1個slave接收到日誌 rpl_semi_sync_master_wait_for_slave_count = 1
* slave 實例my.cnf文件
[mysqld] ########basic settings######## server-id = 210 port = 3306 user = mysql bind_address = 0.0.0.0 character_set_server=utf8mb4 skip_name_resolve = 1 datadir = /data/mysql_data log_error = error.log # slave上開啓只讀，避免應用誤寫導致主從數據不一致 read_only = on super_read_only = on #######replication settings######## master_info_repository = TABLE relay_log_info_repository = TABLE log_bin = bin.log sync_binlog = 1 log_slave_updates binlog_format = row relay_log = relay.log relay_log_recovery = 1 binlog_gtid_simple_recovery = 1 slave_skip_errors = ddl_exist_errors ######semi sync replication settings######## plugin_dir=/usr/local/mysql/lib/plugin plugin_load = "rpl_semi_sync_master=semisync_master.so;rpl_semi_sync_slave=semisync_slave.so" loose_rpl_semi_sync_master_enabled = 1 loose_rpl_semi_sync_slave_enabled = 1 loose_rpl_semi_sync_master_timeout = 5000 rpl_semi_sync_master_wait_point = AFTER_SYNC rpl_semi_sync_master_wait_for_slave_count = 1
4. 搭建無數據基於無損全複製主從 [ master原來無數據 ]
* Master 創建複製所使用的用戶 [ 此處ip設置爲slave服務IP或者% ]
mysql> grant replication slave on *.* to 'rpl'@'172.18.4.1' identified by '123'; Query OK, 0 rows affected, 1 warning (0.00 sec)
* master服務器上查看binlog文件名和日誌位置
mysql> show master status; +------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------+----------+--------------+------------------+-------------------+ | bin.000002 | 689 | | | | +------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
* slave服務器上設置master信息
未開啓slave服務時，Slave_IO_Running與Slave_SQL_Running狀態成No
master_log_file 設置開始複製文件, master_log_pos 開始文件複製點
mysql> show slave status; # 未開啓複製功能時，slave狀態是空的 Empty set (0.00 sec) mysql> change master to master_host='172.18.0.1',master_user='rpl',master_password='123',master_log_file='bin.000002',master_log_pos=689; Query OK, 0 rows affected, 2 warnings (0.03 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.18.0.1 Master_User: rpl Master_Port: 3306 Connect_Retry: 60 Master_Log_File: bin.000002 Read_Master_Log_Pos: 689 Relay_Log_File: relay.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: bin.000002 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 689 Relay_Log_Space: 154 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 Master_UUID: Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
* 開啓slave服務，並查看狀態
正常開啓slave服務後，Slave_IO_Running與Slave_SQL_Running狀態成Yes
mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.18.0.1 Master_User: rpl Master_Port: 3306 Connect_Retry: 60 Master_Log_File: bin.000002 Read_Master_Log_Pos: 689 Relay_Log_File: relay.000002 Relay_Log_Pos: 314 Relay_Master_Log_File: bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 689 Relay_Log_Space: 511 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 110 Master_UUID: d7d5a01b-6ea0-11e7-9773-00163e0432c5 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
* Master上查看Slave連接信息
mysql> show slave hosts; +-----------+------+------+-----------+--------------------------------------+ | Server_id | Host | Port | Master_id | Slave_UUID | +-----------+------+------+-----------+--------------------------------------+ | 210 | | 3306 | 110 | 499ecfb3-6ea2-11e7-aec1-00163e028c02 | +-----------+------+------+-----------+--------------------------------------+ 1 row in set (0.00 sec)
* Master上操作創建數據庫與表，並插入數據
mysql> create database mytest character set utf8mb4;
Query OK, 1 row affected (0.01 sec)
mysql> use mytest;
Database changed
mysql> create table users(
-> id BIGINT NOT NULL AUTO_INCREMENT,
-> name VARCHAR(255) NOT NULL,
-> sex ENUM('M', 'F') NOT NULL DEFAULT 'M',
-> age INT SIGNED NOT NULL DEFAULT '0',
-> PRIMARY KEY (id)
-> )ENGINE=INNODB DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected (0.02 sec)
mysql> insert into users values(null, 'tom', 'M', 24), (null, 'jak', 'F', 32), (null, 'sea', 'M', 35), (null, 'lisea', 'M', 29);
Query OK, 4 rows affected (0.01 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql> select * from users;
+----+-------+-----+-----+
| id | name | sex | age |
+----+-------+-----+-----+
| 1 | tom | M | 24 |
| 2 | jak | F | 32 |
| 3 | sea | M | 35 |
| 4 | lisea | M | 29 |
+----+-------+-----+-----+
4 rows in set (0.00 sec)* Slave上查看
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | mytest | | performance_schema | | sys | +--------------------+ 5 rows in set (0.00 sec) mysql> use mytest; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +------------------+ | Tables_in_mytest | +------------------+ | users | +------------------+ 1 row in set (0.00 sec) mysql> select * from users; +----+-------+-----+-----+ | id | name | sex | age | +----+-------+-----+-----+ | 1 | tom | M | 24 | | 2 | jak | F | 32 | | 3 | sea | M | 35 | | 4 | lisea | M | 29 | +----+-------+-----+-----+ 4 rows in set (0.00 sec)
5. 搭建有數據基於無損全複製主從 [ master原來有數據 ]
* 查看mytest庫內容
Database changed mysql> show tables; +------------------+ | Tables_in_mytest | +------------------+ | users | +------------------+ 1 row in set (0.00 sec) mysql> select * from users; +----+-------+-----+-----+ | id | name | sex | age | +----+-------+-----+-----+ | 1 | tom | M | 24 | | 2 | jak | F | 32 | | 3 | sea | M | 35 | | 4 | lisea | M | 29 | +----+-------+-----+-----+ 4 rows in set (0.00 sec)
* 使用mysqldump原子導出master庫數據，並記錄binlog [ 測試只有mytest庫 ]
如果有多個庫，-B參數後逗號分隔。
[root@master ~]# mysqldump --single-transaction --master-data -B mytest -uroot -p > mytest.sql Enter password:
* 將導出的備份文件mytest.sql傳輸到slave
[root@master ~]# scp ./mytest.sql [email protected]:/root
* slave創建相同的數據庫，並將備份導入
mysql> create database mytest character set utf8mb4; Query OK, 1 row affected (0.01 sec) [root@slave ~]# mysql -uroot -p mytest < mytest.sql Enter password:
* Master 創建複製所使用的用戶 [ 此處ip設置爲slave服務IP或者% ]
mysql> grant replication slave on *.* to 'rpl'@'172.18.4.1' identified by '123'; Query OK, 0 rows affected, 1 warning (5.01 sec)
* 查看備份文件mytest.sql查看binlog文件名和日誌位置
[root@slave ~]# grep 'CHANGE MASTER TO' mytest.sql CHANGE MASTER TO MASTER_LOG_FILE='bin.000002', MASTER_LOG_POS=1575;
* slave服務器上設置master信息
未開啓slave服務時，Slave_IO_Running與Slave_SQL_Running狀態成No
mysql> show slave status; # 未開啓複製功能時，slave狀態是空的 Empty set (0.00 sec) mysql> change master to master_host='172.18.0.1',master_user='rpl',master_password='123',master_log_file='bin.000002',master_log_pos=1575; Query OK, 0 rows affected, 2 warnings (0.02 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.18.0.1 Master_User: rpl Master_Port: 3306 Connect_Retry: 60 Master_Log_File: bin.000002 Read_Master_Log_Pos: 1575 Relay_Log_File: relay.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: bin.000002 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 1575 Relay_Log_Space: 154 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 Master_UUID: Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
* 開啓slave服務，並查看狀態
mysql> start slave; Query OK, 0 rows affected (0.01 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.18.0.1 Master_User: rpl Master_Port: 3306 Connect_Retry: 60 Master_Log_File: bin.000002 Read_Master_Log_Pos: 1872 Relay_Log_File: relay.000002 Relay_Log_Pos: 611 Relay_Master_Log_File: bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 1872 Relay_Log_Space: 808 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 110 Master_UUID: d7d5a01b-6ea0-11e7-9773-00163e0432c5 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
* master上mytest庫數據操作
mysql> select * from mytest.users; +----+-------+-----+-----+ | id | name | sex | age | +----+-------+-----+-----+ | 1 | tom | M | 24 | | 2 | jak | F | 32 | | 3 | sea | M | 35 | | 4 | lisea | M | 29 | +----+-------+-----+-----+ 4 rows in set (0.00 sec) mysql> insert into mytest.users select null, 'test', 'M', 42; Query OK, 1 row affected (0.01 sec) Records: 1 Duplicates: 0 Warnings: 0 mysql> update mytest.users set name='seasea' where id = 3; Query OK, 1 row affected (0.01 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> select * from mytest.users; +----+--------+-----+-----+ | id | name | sex | age | +----+--------+-----+-----+ | 1 | tom | M | 24 | | 2 | jak | F | 32 | | 3 | seasea | M | 35 | | 4 | lisea | M | 29 | | 5 | test | M | 42 | +----+--------+-----+-----+ 5 rows in set (0.00 sec)
* slave上查看
mysql> select * from mytest.users; +----+--------+-----+-----+ | id | name | sex | age | +----+--------+-----+-----+ | 1 | tom | M | 24 | | 2 | jak | F | 32 | | 3 | seasea | M | 35 | | 4 | lisea | M | 29 | | 5 | test | M | 42 | +----+--------+-----+-----+ 5 rows in set (0.00 sec)
6. 總結
以需求驅動技術，技術本身沒有優略之分，只有業務之分。