percona-tools工具在線DDL的使用

好久沒有寫博客了,今天發表一篇吧:)

通常的在線進行表字段的增減都會造成表所,如果表較小還能接受,如果過大則這個鎖持續的時間會讓人比較煩惱,對業務持續性影響較大。


Percona 提供了一款關於MySQL管理的工具集很強大,包括了在線DDL工具 pt-online-schema-change

安裝就不介紹了,自行百度吧


下面的原理介紹摘自網絡http://www.it165.net/pro/html/201312/8928.html,寫的比較清晰,就不重複造輪子了。

+++++++++++++++++++++

模仿MySQL的alter,但不同的是在alter操作更改表結構的時候不用鎖定表,也就是說執行alter的時候不會阻塞寫和讀取操作,客戶端可以繼續都和修改數據。注意執行這個工具的時候必須做好備份,操作之前最好詳細讀一下官方文檔。


1、如果存在外鍵,根據alter-foreign-keys-method參數的值,檢測外鍵相關的表,做相應設置的處理。沒有使用 --alter-foreign-keys-method 指定特定的值,該工具不予執行

2、創建一個新的表,表結構爲修改後的數據表,用於從源數據表向新表中導入數據。

3、創建觸發器,用於記錄從拷貝數據開始之後,對源數據表繼續進行數據修改的操作記錄下來,用於數據拷貝結束後,執行這些操作,保證數據不會丟失。如果表中已經定義了觸發器這個工具就不能工作了。

4、拷貝數據,從源數據表中拷貝數據到新表中。

5、修改外鍵相關的子表,根據修改後的數據,修改外鍵關聯的子表。

6、rename源數據表爲old表,把新表rename爲源表名,並將old表刪除。

7、刪除觸發器。

+++++++++++++++++++

相關的參數請 --help

1. 進行--dry-run測試

[root@master ~]#  pt-online-schema-change  --user=root --password=root --host=localhost  --alter='ENGINE=Innodb' D=baby,t=baby_account --dry-run

Operation, tries, wait:

  analyze_table, 10, 1

  copy_rows, 10, 0.25

  create_triggers, 10, 1

  drop_triggers, 10, 1

  swap_tables, 10, 1

  update_foreign_keys, 10, 1

Child tables:

  `baby`.`baby_account_billing_contract` (approx. 21 rows)

  `baby`.`baby_account_tactics` (approx. 13308 rows)

  `baby`.`baby_account_tactics_bak` (approx. 1672 rows)

  `baby`.`baby_billing` (approx. 11856 rows)

  `baby`.`baby_billing_apply` (approx. 128 rows)

  `baby`.`baby_order_bak_haochongpei` (approx. 1 rows)

You did not specify --alter-foreign-keys-method, but there are foreign keys that reference the table. Please read the tool's documentation carefully.

##表有主鍵,對主鍵的處理方式需要明確指出

--alter-foreign-keys-method 說明:

如何把外鍵引用到新表?需要特殊處理帶有外鍵約束的表,以保證它們可以應用到新表.當重命名錶的時候,外鍵關係會帶到重命名後的表上。

該工具有兩種方法,可以自動找到子表,並修改約束關係。

auto 在rebuild_constraints和drop_swap兩種處理方式中選擇一個

rebuild_constraints使用 ALTER TABLE語句先刪除外鍵約束,然後再添加.如果子表很大的話,會導致長時間的阻塞。

drop_swap 執行FOREIGN_KEY_CHECKS=0,禁止外鍵約束,刪除原表,再重命名新表。這種方式很快,也不會產生阻塞,但是有風險:

1, 在刪除原表和重命名新表的短時間內,表是不存在的,程序會返回錯誤。

2, 如果重命名錶出現錯誤,也不能回滾了.因爲原表已經被刪除。

none 類似"drop_swap"的處理方式,但是它不刪除原表,並且外鍵關係會隨着重命名轉到老表上面。

我這裏指定的是auto自動選擇,可以看到下文的提示


[root@master ~]#  pt-online-schema-change  --user=root --password=root --host=localhost  --alter='ENGINE=Innodb' D=baby,t=baby_account --alter-foreign-keys-method=auto --dry-run

Operation, tries, wait:

  analyze_table, 10, 1

  copy_rows, 10, 0.25

  create_triggers, 10, 1

  drop_triggers, 10, 1

  swap_tables, 10, 1

  update_foreign_keys, 10, 1

Child tables:

  `baby`.`baby_account_billing_contract` (approx. 21 rows)

  `baby`.`baby_account_tactics` (approx. 13308 rows)

  `baby`.`baby_account_tactics_bak` (approx. 1672 rows)

  `baby`.`baby_billing` (approx. 11856 rows)

  `baby`.`baby_billing_apply` (approx. 128 rows)

  `baby`.`baby_order_bak_haochongpei` (approx. 1 rows)

Will automatically choose the method to update foreign keys.

Starting a dry run.  `baby`.`baby_account` will not be altered.  Specify --execute instead of --dry-run to alter the table.

Creating new table...

Created new table baby._baby_account_new OK.

Altering new table...

Altered `baby`.`_baby_account_new` OK.

Not creating triggers because this is a dry run.

Not copying rows because this is a dry run.

Not determining the method to update foreign keys because this is a dry run.

Not swapping tables because this is a dry run.

Not updating foreign key constraints because this is a dry run.

Not dropping old table because this is a dry run.

Not dropping triggers because this is a dry run.

2016-08-29T14:31:59 Dropping new table...

2016-08-29T14:31:59 Dropped new table OK.

Dry run complete.  `baby`.`baby_account` was not altered.


執行操作,看到報錯,因爲我這環境是主從複製的環境,所以--check-replication-filters參數需要明確指出


[root@master ~]#  pt-online-schema-change  --user=root --password=root --host=localhost  --alter='ENGINE=Innodb' D=baby,t=baby_account --alter-foreign-keys-method=auto --execute

Found 2 slaves:

  client.wboy.com

  client1.wboy.com

Will check slave lag on:

  client.wboy.com

  client1.wboy.com

Replication filters are set on these hosts:

  client.wboy.com

    binlog_ignore_db = mysql,test,information_schema,performance_schema

  client1.wboy.com

    binlog_ignore_db = mysql,test,information_schema,performance_schema

Please read the --check-replication-filters documentation to learn how to solve this problem. at /usr/local/bin/pt-online-schema-change line 8493.



[root@master ~]#  pt-online-schema-change  --user=root --password=root --host=localhost  --alter='ENGINE=Innodb' D=baby,t=baby_account --alter-foreign-keys-method=auto --no-check-replication-filters --execute

Found 2 slaves:

  client.wboy.com

  client1.wboy.com

Will check slave lag on:

  client.wboy.com

  client1.wboy.com

Operation, tries, wait:

  analyze_table, 10, 1

  copy_rows, 10, 0.25

  create_triggers, 10, 1

  drop_triggers, 10, 1

  swap_tables, 10, 1

  update_foreign_keys, 10, 1

Child tables:

  `baby`.`baby_account_billing_contract` (approx. 21 rows)

  `baby`.`baby_account_tactics` (approx. 13308 rows)

  `baby`.`baby_account_tactics_bak` (approx. 1672 rows)

  `baby`.`baby_billing` (approx. 11856 rows)

  `baby`.`baby_billing_apply` (approx. 128 rows)

  `baby`.`baby_order_bak_haochongpei` (approx. 1 rows)

Will automatically choose the method to update foreign keys.

Altering `baby`.`baby_account`...

Creating new table...

Created new table baby._baby_account_new OK.

Waiting forever for new table `baby`.`_baby_account_new` to replicate to client1.wboy.com...

Waiting for client1.wboy.com:   0% 00:00 remain

Waiting for client1.wboy.com:   0% 00:00 remain

Altering new table...

Altered `baby`.`_baby_account_new` OK.

2016-08-29T14:54:50 Creating triggers...

2016-08-29T14:54:50 Created triggers OK.

2016-08-29T14:54:50 Copying approximately 20189 rows...

Replica lag is 32 seconds on client1.wboy.com.  Waiting.

Replica lag is 60 seconds on client1.wboy.com.  Waiting.

Copying `baby`.`baby_account`:  13% 06:43 remain

Replica lag is 32 seconds on client1.wboy.com.  Waiting.

Replica lag is 60 seconds on client1.wboy.com.  Waiting.

Copying `baby`.`baby_account`:  58% 01:33 remain

Replica lag is 31 seconds on client1.wboy.com.  Waiting.

Replica lag is 59 seconds on client1.wboy.com.  Waiting.

Copying `baby`.`baby_account`:  99% 00:01 remain

2016-08-29T14:58:04 Copied rows OK.

2016-08-29T14:58:04 Max rows for the rebuild_constraints method: 6168

Determining the method to update foreign keys...

2016-08-29T14:58:04   `baby`.`baby_account_billing_contract`: 21 rows; can use rebuild_constraints

2016-08-29T14:58:04   `baby`.`baby_account_tactics`: too many rows: 13308; must use drop_swap

2016-08-29T14:58:04 Drop-swapping tables...

2016-08-29T14:58:04 Analyzing new table...

2016-08-29T14:58:05 Dropped and swapped tables OK.

Not dropping old table because --no-drop-old-table was specified.

2016-08-29T14:58:05 Dropping triggers...

2016-08-29T14:58:05 Dropped triggers OK.

Successfully altered `baby`.`baby_account`.


發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章