Mysql主從複製及讀寫分離的實現

原創 ALLEN_YNAG 2019-02-22 20:18

Mysql 5.6 基於GTID的主從複製及使用Amoeba配置讀寫分離

一、簡介

二、Mysql主從配置

三、讀寫分離配置

一、Amoeba簡介

Amoeba(變形蟲)項目,該開源框架於2008年開始發佈一款 Amoeba for Mysql軟件。這個軟件致力於MySQL的分佈式數據庫前端代理層，它主要在應用層訪問MySQL的 時候充當SQL路由功能，專注於分佈式數據庫代理層（Database Proxy）開發。座落與 Client、DB Server(s)之間,對客戶端透明。具有負載均衡、高可用性、SQL 過濾、讀寫分離、可路由相關的到目標數據庫、可併發請求多臺數據庫合併結果。 通過Amoeba你能夠完成多數據源的高可用、負載均衡、數據切片的功能，目前Amoeba已在很多企業的生產線上面使用

Amoeba優缺點

優點：

1、降低費用，簡單易用

2、提高系統整體可用性

3、易於擴展處理能力與系統規模

4、可以直接實現讀寫分離及負載均衡效果，而不用修改代碼

缺點：

1、不支持事務與存儲過程

2、暫不支持分庫分表，amoeba目前只做到分數據庫實例

3、不適合從amoeba導數據的場景或者對大數據量查詢的query並不合適（比如一次請求返回10w以上甚至更多數據的場合）

Mysql GTID

Mysql 5.6的新特性之一，加入了全局事務性ID(GTID:Global Transactions Identifier)來強化數據庫的主備一致性，故障恢復，以及容錯能力;也使得複製功能的配置、監控及管理變得更加易於實現，且更加健壯

二、Mysql主從配置

1、環境介紹：

兩臺Mysql數據庫實現主從配置，172.16.14.2主機爲Master；172.16.14.3爲Slave

2、在Master服務器上安裝並配置Mysql

######安裝Mysql並加入到系統服務
[root@master ~]# tar xf mysql-5.6.13-linux-glibc2.5-x86_64.tar.gz -C /usr/local/
[root@master ~]# cd /usr/local/
[root@master local]# ln -s mysql-5.6.13-linux-glibc2.5-x86_64 mysql
[root@master local]# cd mysql
[root@master mysql]# cp support-files/mysql.server /etc/init.d/mysqld
[root@master mysql]# chmod +x /etc/init.d/mysqld
[root@master mysql]# chkconfig --add mysqld
[root@master mysql]# echo "PATH=/usr/local/mysql/bin:$PATH" >> /etc/profile
[root@master mysql]# . /etc/profile
----------------------------------------------------------------------
######提供主配置文件
[root@master mysql]# vim /etc/my.cnf
[client]
#password       = your_password
port            = 3306
socket          = /tmp/mysql.sock
[mysqld]
port            = 3306
socket          = /tmp/mysql.sock
skip-external-locking
key_buffer_size = 256M
max_allowed_packet = 1M
table_open_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 8
binlog-format=ROW
log-slave-updates=true
gtid-mode=on
enforce-gtid-consistency=true
master-info-repository=TABLE
relay-log-info-repository=TABLE
sync-master-info=1
slave-parallel-workers=2
binlog-checksum=CRC32
master-verify-checksum=1
slave-sql-verify-checksum=1
binlog-rows-query-log_events=1
report-port=3306
datadir=/data
report-host=master.allen.com
log-bin=mysql-bin
server-id       = 10
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[myisamchk]
key_buffer_size = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
----------------------------------------------------------------------
######初始化Mysql
[root@master mysql]# useradd -r mysql
[root@master mysql]# mkdir /data
[root@master mysql]# chown -R mysql.mysql /data
[root@master mysql]# chown -R root.mysql /usr/local/mysql/*
[root@master mysql]# ./scripts/mysql_install_db --user=mysql --datadir=/data/
[root@master ~]# service mysqld start

3、在Slave服務器上安裝Mysql與在Master服務器上安裝方法相同，這裏不在介紹，而在Slave服務器上安裝Mysql有兩個參數與Master服務器不同；如下

server-id=11
report-host=slave.allen.com
[root@slave ~]# service mysqld start

4、在Master服務器上爲Slave創建複製用戶並測試連接

[root@master ~]# mysql
mysql> grant replication slave,replication client on *.* to 'slave'@'172.16.%.%' identified by 'passwd';
mysql> flush privileges;
------------------------------------------------------------------------
######測試連接
[root@slave ~]# mysql -uslave -ppasswd -h 172.16.14.2

5、啓動從節點的複製線程

[root@slave ~]# mysql
mysql> change master to master_host='172.16.14.2',master_user='slave',master_password='passwd',master_auto_position=1;
mysql> start slave;
mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 172.16.14.2
                  Master_User: slave
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000005
          Read_Master_Log_Pos: 191
               Relay_Log_File: slave-relay-bin.000003
                Relay_Log_Pos: 401
        Relay_Master_Log_File: mysql-bin.000005
             Slave_IO_Running: Yes      #主要看這兩項爲“YES”說明成功
            Slave_SQL_Running: Yes

6、在Master服務器創建數據庫查看Slave服務器是否更新

[root@master ~]# mysql -e 'create database allen;'
------------------------------------------------------------------------
[root@slave ~]# mysql -e 'show databases;'
+--------------------+
| Database           |
+--------------------+
| information_schema |
| allen              |
| mysql              |
| performance_schema |
| test               |
+--------------------+
######由上可見，新創建的"allen"數據庫已成功同步

至此Mysql 5.6 基於GTID的複製已經完成，下面將介紹如何基於Mysql的主從複製架構做讀寫分離

三、讀寫分離配置

1、基於前面做的Mysql主從架構，然後在前端加一臺服務器，用於實現Mysql的讀寫分離，IP地址爲：172.16.14.1；由於Amoeba是java程序所研發，所以需要先安裝JDK程序

2、安裝JDK

[root@amoeba ~]# chmod +x jdk-6u31-linux-x64-rpm.bin
[root@amoeba ~]# ./jdk-6u31-linux-x64-rpm.bin
[root@amoeba ~]# vim /etc/profile.d/java.sh
export JAVA_HOME=/usr/java/latest
export PATH=$JAVA_HOME/bin:$PATH
[root@amoeba ~]# . /etc/profile.d/java.sh
[root@amoeba ~]# java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)

3、安裝Amoeba

[root@amoeba ~]# mkdir /usr/local/amoeba
[root@amoeba ~]# tar xf amoeba-mysql-binary-2.2.0.tar.gz -C /usr/local/amoeba/
[root@amoeba ~]# vim /etc/profile.d/amoeba.sh
export AMOEBA_HOME=/usr/local/amoeba
export PATH=$AMOEBA_HOME/bin:$PATH
[root@amoeba ~]# . /etc/profile.d/amoeba.sh
[root@amoeba ~]# amoeba
amoeba start|stop

4、授權Mysql用戶，用於實現前端Amoeba連接，由於上面授權的主從複製帳號不能同步"mysql"數據庫，所以用戶名也無法同步，要在兩臺數據庫上同時授權，用戶名密碼保持一致

[root@master ~]# mysql
mysql> grant all on *.* to 'amoeba'@'172.16.%.%' identified by 'amoebapass';
mysql> flush privileges;
------------------------------------------------------------------------
[root@slave ~]# mysql
mysql> grant all on *.* to 'amoeba'@'172.16.%.%' identified by 'amoebapass';
mysql> flush privileges;

5、配置Amoeba

[root@amoeba ~]# cd /usr/local/amoeba/conf/  #主要配置文件爲以下兩個
amoeba.xml        #定義數據庫讀寫分離及節點管理信息等
dbServers.xml     #定義連接後端Mysql服務器信息
------------------------------------------------------------------------
[root@amoeba conf]# vim dbServers.xml
<?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd">
<amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/">
        <!--
            Each dbServer needs to be configured into a Pool,
            If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration:
             add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig
             such as 'multiPool' dbServer
        -->
    <dbServer name="abstractServer" abstractive="true">
        <factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">
            <property name="manager">${defaultManager}</property>
            <property name="sendBufferSize">64</property>
            <property name="receiveBufferSize">128</property>
            <!-- mysql port -->
            <property name="port">3306</property>
            #連接後端Mysql服務器端口
            <!-- mysql schema -->
            <property name="schema">test</property>
            #連接到後端Mysql使用的默認數據庫
            <!-- mysql user -->
            <property name="user">amoeba</property>
            #連接後端Mysql數據庫的用戶名
            <property name="password">amoebapass</property>
                #連接後端Mysql數據庫的用戶名
        </factoryConfig>
        <poolConfig class="com.meidusa.amoeba.net.poolable.PoolableObjectPool">
            <property name="maxActive">500</property>
            <property name="maxIdle">500</property>
            <property name="minIdle">10</property>
            <property name="minEvictableIdleTimeMillis">600000</property>
            <property name="timeBetweenEvictionRunsMillis">600000</property>
            <property name="testOnBorrow">true</property>
            <property name="testOnReturn">true</property>
            <property name="testWhileIdle">true</property>
        </poolConfig>
    </dbServer>
                        #定義"master"數據庫節點,"name"名稱可以自定義
    <dbServer name="master"  parent="abstractServer">
        <factoryConfig>
            <!-- mysql ip -->
            <property name="ipAddress">172.16.14.2</property>
        </factoryConfig>
    </dbServer>
                       #定義"slave"數據庫節點
    <dbServer name="slave"  parent="abstractServer">
        <factoryConfig>
            <!-- mysql ip -->
            <property name="ipAddress">172.16.14.3</property>
        </factoryConfig>
    </dbServer>
                                                                                                                                                                                                                             
    <dbServer name="multiPool" virtual="true">
        <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
            <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA--> #這裏註釋的爲算法
            <property name="loadbalance">1</property>
            #定義選擇哪一種算法進行負載均衡調度
            <!-- Separated by commas,such as: server1,server2,server1 -->
#定義數據庫池,用於實現負載均衡."slave"爲上面定義的從數據庫節點，可以寫多個用","分隔;
如:"slave,slave,master"可以實現基於權重負載的效果;當然這裏也可以不用定義
            <property name="poolNames">slave</property>
        </poolConfig>
    </dbServer>
</amoeba:dbServers>

========================================================================
[root@amoeba conf]# vim amoeba.xml
<?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">
<amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">
    <proxy>
        <!-- service class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba for Mysql" class="com.meidusa.amoeba.net.ServerableConnectionManager">
            <!-- port --> #定義amoeba代理服務器的對外連接監聽端口
            <property name="port">3306</property>    
            <!-- bind ipAddress --> #定義amoeba代理服務器對外連接的監聽IP
            <property name="ipAddress">172.16.14.1</property>    
            <property name="manager">${clientConnectioneManager}</property>  
            <property name="connectionFactory">
                <bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
                    <property name="sendBufferSize">128</property>
                    <property name="receiveBufferSize">64</property>
                </bean>
            </property>    
            <property name="authenticator">
                <bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">               
    #定義amoeba連接用戶名和密碼，客戶端或程序只需要使用此用戶名和密碼連接即可
                    <property name="user">admin</property>           
                    <property name="password">password</property>            
                    <property name="filter">
                        <bean class="com.meidusa.amoeba.server.IPAccessController">
                    <property name="ipFile">${amoeba.home}/conf/access_list.conf</property>
                        </bean>
                    </property>
                </bean>
            </property>    
        </service> 
        <!-- server class must implements com.meidusa.amoeba.service.Service -->
        <service name="Amoeba Monitor Server" class="com.meidusa.amoeba.monitor.MonitorServer">
            <!-- port -->
            <!--  default value: random number
            <property name="port">9066</property>
            -->
            <!-- bind ipAddress -->
            <property name="ipAddress">127.0.0.1</property>
            <property name="daemon">true</property>
            <property name="manager">${clientConnectioneManager}</property>
            <property name="connectionFactory">
            <bean class="com.meidusa.amoeba.monitor.net.MonitorClientConnectionFactory"></bean>
            </property>    
        </service> 
        <runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">
            <!-- proxy server net IO Read thread size -->
            <property name="readThreadPoolSize">20</property>    
            <!-- proxy server client process thread size -->
            <property name="clientSideThreadPoolSize">30</property>  
            <!-- mysql server data packet process thread size -->
            <property name="serverSideThreadPoolSize">30</property>  
            <!-- per connection cache prepared statement size  -->
            <property name="statementCacheSize">500</property>   
            <!-- query timeout( default: 60 second , TimeUnit:second) -->
            <property name="queryTimeout">60</property>
        </runtime> 
    </proxy>
    <!--
        Each ConnectionManager will start as thread
        manager responsible for the Connection IO read , Death Detection
    -->
    <connectionManagerList>
    <connectionManager name="clientConnectioneManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
    <property name="subManagerClassName">com.meidusa.amoeba.net.ConnectionManager</property>
            <!--
              default value is avaliable Processors
            <property name="processors">5</property>
             -->
        </connectionManager>
<connectionManager name="defaultManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
<property name="subManagerClassName">com.meidusa.amoeba.net.AuthingableConnectionManager</property>  
            <!--
              default value is avaliable Processors
            <property name="processors">5</property>
             -->
        </connectionManager>
    </connectionManagerList>
                                                                                                                                                                  
        <!-- default using file loader -->
    <dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">
        <property name="configFile">${amoeba.home}/conf/dbServers.xml</property>
    </dbServerLoader>
    <queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">
        <property name="ruleLoader">
            <bean class="com.meidusa.amoeba.route.TableRuleFileLoader">
                <property name="ruleFile">${amoeba.home}/conf/rule.xml</property>
                <property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>
            </bean>
        </property>
        <property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>
        <property name="LRUMapSize">1500</property>
        <property name="defaultPool">master</property>
        #定義默認池,一些sql語句默認會在此定義的服務器上執行
        <property name="writePool">master</property>
                #定義只寫數據庫
        <property name="readPool">slave</property>
#定義只讀數據庫,此處定義的是在"dbServer.xml"文件中定義的後端服務器名稱，也可以定義數據庫池的名稱,實現負載均衡
        <property name="needParse">true</property>
    </queryRouter>
</amoeba:configuration>

6、啓動amoeba服務並連接測試

[root@amoeba ~]# amoeba start &
[root@amoeba ~]# ss -tanlp | grep 3306
LISTEN     0      128      ::ffff:172.16.14.1:3306     :::*      users:(("java",29448,54))
######由上可見已啓動成功並監聽在3306端口
------------------------------------------------------------------------

100603942.gif

7、連接到amoeba代理服務器,執行插入與查詢操作,分別在後端兩臺服務器上抓包,查看是否實現讀寫分離

[root@amoeba ~]# mysql -uadmin -ppassword -h 172.16.14.1
mysql> create table allen.table (id int(4));  #執行寫入操作
Query OK, 0 rows affected (0.08 sec)
mysql> select User,Host from mysql.user;      #執行查詢操作

102539195.gif

102542226.gif

由上圖可見，抓包的結果實現了讀寫分離的效果

到此，Mysql基於GTID的主從複製及Mysql的讀寫分離已完成，後續會更新其他內容，敬請關注...


發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

MySQL数据库集群-PXC方案

wy53780 2020-04-23 13:55:07

sql語法、關係建模

死瘦宅 2019-02-24 22:35:22

MySQL 事務、函數、觸發器、數據庫備份、權限管理

死瘦宅 2019-02-24 22:35:12

《從0到1學習Flink》—— Flink 讀取 Kafka 數據批量寫入到 MySQL

zhisheng 2019-02-24 21:04:11

SpringBoot 填坑 (一) | CentOS7.4 環境下，表時間字段默認值設置失效

一個優秀的廢人 2019-02-24 15:33:55

mysql-存儲過程

雙子城 2019-02-24 13:43:57

jmeter學習指南之操作 mysql 數據庫

小強測試 2019-02-24 13:34:06

20190222全天的實驗

清歡難尋 2019-02-24 13:17:21

數據庫監控：zabbix（fpmmm）

AMGYM 2019-02-24 13:05:14

數據庫監控：nagios（check_mysql_health）

AMGYM 2019-02-24 13:05:14

如何釋放已刪除的InnoDB磁盤空間

lvrenX 2019-02-24 13:04:33

MYSQL中視圖的使用

雙子城 2019-02-24 08:44:17

MySQL從刪庫到跑路

司木 2019-02-24 00:24:07

JavaBean操作

yi_sixian 2019-02-23 14:05:20

Amoeba for MySQL學習研究之配置

lvrenX 2019-02-24 13:04:33