一 、背景介紹
近年來，TokuDB作爲MySQL的大數據（Big Data）存儲引擎受到人們的普遍關注。其架構的核心基於一種新的叫做分形樹（Fractal Trees）的索引數據結構，該結構是緩存無關的，即使索引數據大小超過內存性能也不會下降，也即沒有內存生命週期和碎片的問題。特別引人注意的是，TokuDB擁有很高的壓縮比（官方稱最大可達25倍），可以在很大的數據上創建大量的索引，並保持性能不下降。同時，TokuDB支持ACID和MVCC，還有在線修改表結構（Live Schema Modification）以及增加的複製性能等特性，使其在某些特定的應用領域（如日誌存儲與分析）有着獨特的優勢。
在TokuDB的應用場景中，通常是數據庫插入操作的量遠遠大於讀取的量，因而本此本測試是在公司生產庫上增加一個從庫，來體驗下TokuDB的高壓縮比和低存儲佔用率，而且在相同數據的情況下io使用率更優秀。
本測試是在生產環境中增加一個從庫，來驗證ToKuDB在大頻率寫入，小頻率讀的場景驗證。
鏈接：https://pan.baidu.com/s/1hs8IOao 密碼：nrxh Centos7
鏈接：https://pan.baidu.com/s/1c2dIPZM 密碼：opmx Centos6
Github地址：https://github.com/XeLabs/tokudb.git
安利下這是我參加的全宇宙最牛逼的MySQL培訓班知數堂MySQL tokudb分支版本，學最專業的MySQL，請來知數堂找我們吳老師，刪庫跑路跑的快的葉老師
http://www.3wedu.net/
一、首先我們導出生產環境主庫的表結構跟數據。
1、導出主庫表結構(爲脫敏處理把庫名更改爲test)
shell>
mysqldump -p --no-data --databases test >test_table.sql
2、導出數據
shell>
mysqldump -p -t --master-data=2 test >test_data.sql
3、處理表結構語句
把所有innodb存儲引擎爲表改爲tokudb
shell>
sed -i 's@InnoDB@TokuDB@g' test_table.sql
4、拷貝數據到需要建立的TokuDB從庫服務器上
二、創建percona TokuDB版本從庫
下載相應的二進制包
下載地址
1、優化系統參數，是系統可以滿足TokuBD的安裝要求
關閉系統透明大頁
shell>
echo never > /sys/kernel/mm/transparent_hugepage/enabled
shell>
echo never > /sys/kernel/mm/transparent_hugepage/defrag
2、創建mysql用戶增加數據目錄
shell>
groupadd mysql
shell>
useradd -r -g mysql -s /sbin/nologin mysql
shell>
cd /opt
shell>
tar -xf XeLabs-TokuDB-5.7.19-Centos6-r1.X86_64.tar.gz
shell>
ln -s /opt/XeLabs-TokuDB-5.7.19-Centos6-r1.X86_64 /usr/local/mysql
shell>
mkdir /data/mysql/mysql3306/{data,logs,tmp} -pshell>
chown mysql.mysql /usr/local/mysql/ /usr/local/mysql /data/mysql -R
[root@smsdb1 ~]#cat /etc/my.cnf
[client] port = 3306 socket = /tmp/mysql3306.sock default-character-set=utf8mb4 [mysql] prompt="\\u@\\h [\\d]>" #pager="less -i -n -S" #tee=/opt/mysql/query.log no-auto-rehash [mysqld] #misc user = mysql basedir = /usr/local/mysql datadir = /data/mysql/mysql3306/data port = 3306 socket = /tmp/mysql3306.sock event_scheduler = 0 default-storage-engine=tokudb skip-name-resolve tmpdir = /data/mysql/mysql3306/tmp #timeout interactive_timeout = 2880000 wait_timeout = 2880000 #character set character-set-server = utf8mb4 open_files_limit = 65535 max_connections = 100 max_connect_errors = 100000 lower_case_table_names =1 # explicit_defaults_for_timestamp=1 #symi replication #rpl_semi_sync_master_enabled=1 #rpl_semi_sync_master_timeout=1000 # 1 second #rpl_semi_sync_slave_enabled=1 #logs log-output=file slow_query_log = 1 slow_query_log_file = slow.log log-error = error.log pid-file = mysql.pid long_query_time = 1 log_error_verbosity=1 #log-slow-admin-statements = 1 #log-queries-not-using-indexes = 1 log-slow-slave-statements = 1 #binlog #binlog_format = STATEMENT binlog_format = row server-id = 43306 log-bin = /data/mysql/mysql3306/logs/mysql-bin binlog_cache_size = 1M max_binlog_size = 256M max_binlog_cache_size = 4M sync_binlog = 0 expire_logs_days = 10 #procedure log_bin_trust_function_creators=1 # gtid-mode = on enforce-gtid-consistency=1 #relay log skip_slave_start = 1 max_relay_log_size = 128M relay_log_purge = 1 relay_log_recovery = 1 relay-log=relay-bin relay-log-index=/data/mysql/mysql3306/logs/relay-bin.index log_slave_updates #slave-skip-errors=1032,1053,1062 #skip-grant-tables #buffers & cache table_open_cache = 2048 table_definition_cache = 2048 table_open_cache = 2048 max_heap_table_size = 96M sort_buffer_size = 128K join_buffer_size = 128K thread_cache_size = 200 query_cache_size = 0 query_cache_type = 0 query_cache_limit = 256K query_cache_min_res_unit = 512 thread_stack = 192K tmp_table_size = 96M key_buffer_size = 8M read_buffer_size = 2M read_rnd_buffer_size = 16M bulk_insert_buffer_size = 32M #myisam myisam_sort_buffer_size = 128M myisam_max_sort_file_size = 10G myisam_repair_threads = 1 #innodb innodb_buffer_pool_size = 100M innodb_buffer_pool_instances = 1 innodb_data_file_path = ibdata1:100M:autoextend innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 8M innodb_log_file_size = 100M innodb_log_files_in_group = 3 innodb_max_dirty_pages_pct = 50 innodb_file_per_table = 1 innodb_rollback_on_timeout innodb_status_file = 1 innodb_io_capacity = 2000 transaction_isolation = READ-COMMITTED innodb_flush_method = O_DIRECT loose_tokudb_cache_size=4G loose_tokudb_directio=ON loose_tokudb_fsync_log_period=1000 tokudb_commit_sync=0
初始化無密碼的數據庫
shell>
cd /usr/local/mysql/bin/ && ./mysqld --initialize-insecure
啓動MySQL數據庫
shell>
cp support-files/mysql.server /etc/init.d/mysqld
shell>
/etc/init.d/mysqld start
增加MySQL環境變量
shell>
echo 'export PATH=/usr/local/mysql/bin:$PATH' >>/etc/profile
shell>
source /etc/profile
導入表結構
shell>
mysql -p test<test_table.sql
清理GTID信息
shell>
mysql -e 'reset master'
導入數據
shell>
mysql -p test <test_data.sql
執行change master 操作
mysql>
change master to master_host='192.168.1.10',master_port='3306',master_user='repl',master_password='repl',master_auto_position=1;
mysql>
start slave;
mysql>
show slave status\G
root@localhost [(none)]>show slave status\G
* 1. row *
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.1.10
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000029
Read_Master_Log_Pos: 900797194
Relay_Log_File: relay-bin.000028
Relay_Log_Pos: 68956928
Relay_Master_Log_File: mysql-bin.000029
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 900797194
Relay_Log_Space: 203175161
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1303306
Master_UUID: d54f3590-7760-11e7-9f22-fa163e3dde13
Master_Info_File: /data/mysql/mysql3306/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: d54f3590-7760-11e7-9f22-fa163e3dde13:31849593-32456621
Executed_Gtid_Set: 05c9edae-dee5-11e7-91fb-fa163ecf7778:1-2,
d54f3590-7760-11e7-9f22-fa163e3dde13:1-32456621
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
root@localhost [(none)]>
至此主從環境搭建完畢。
對比下主庫InnoDB存貯引擎數據與從庫TokuDB存儲引擎的數據大小對比
InnoDB存儲引擎在實例數據6G,
TokuDB 存儲引擎存儲的數據不到1G
在3.6G 大小sql語句下插入 innodb存儲引擎與tokudb存儲引擎磁盤iO對比
由此可見tokudb在磁盤節約和io使用上在寫場景比較多的業務中相當給力的。