數據抽象:
物理層:決定數據的存儲格式,即RDBMS在磁盤上如何組織文件;
邏輯層:描述DB存儲什麼數據,以及數據間存在什麼樣的關係,數據庫管理員看到的;
視圖層:描述DB中的部分數據;最終用戶實際看到的
mysql使用插件式存儲引擎:定義mysql組織數據數據的格式和磁盤存儲格式如何兼容起來。存儲管理器有多種實現版本,功能和特性可能均略有差別;用戶可根據需要靈活選擇;相比mysql:
(1) 更多的存儲的存儲引擎
MyISAM --> Aria(Mariadb上),不支持事務
InnoDB --> XtraDB
(2) 諸多擴展和新特性
(3) 提供了較多測試組件
(4) truly open source
存儲引擎是表級別的概念,每創建一張表都應該指明它的存儲引擎,對同一個數據庫而言,不建議交叉使用多個存儲引擎。
使用SHOW TABLE STATUS\G;查看數據庫用的存儲引擎 InnoDB:增強版是xtraDB,mariadb默認使用xtraDB 處理大量的短期事務; 表空間:每種存儲引擎可以唯一識別的文件系統 數據存儲於“表空間(table space)”中; (1) 所有InnoDB表的數據和索引放置於同一個表空間中哪怕不在同一個數據庫; 表空間文件:datadir定義的目錄下 數據文件:ibdata1, ibdata2, ... (2) 每個表單獨使用一個表空間存儲表的數據和索引;易於實現單表導入導出 innodb_file_per_table=ON MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE 'innodb_file_per%'; +-----------------------+-------+ | Variable_name | Value | +-----------------------+-------+ | innodb_file_per_table | OFF | +-----------------------+-------+ 1 row in set (0.00 sec) 創建t2表: MariaDB [mydb]> CREATE TABLE t2 (id int, name char(30)); Query OK, 0 rows affected (0.01 sec) 在系統的數據目錄下發現了t2表的相關數據 數據文件(存儲數據和索引):tbl_name.ibd, 表格式定義:tbl_name.frm [root@node1 mydb]# ls db.opt t1.frm t2.frm [root@node1 mydb]# pwd /var/lib/mysql/mydb 基於MVCC來支持高併發,支持所有的四個隔離級別,默認級別爲REPEATABLE READ; 間隙鎖防止幻讀; 使用聚集索引 支持“自適應hash索引” 鎖粒度:行級鎖 數據存儲:表空間 併發:MVCC, 間隙鎖 索引:聚集索引、輔助索引 性能:預計操作、自適應hash、插入緩存區 備份:支持熱備(xtrabacup)
MyISAM: 支持全文索引(FULLTEXT index),innodb不支持、壓縮、空間函數(GIS); 但不支持事務,且爲表級鎖,不支持行級鎖; 崩潰後無法安全恢復 適用場景:只讀(或者寫較少)、表較小(可以接受長時間進行修復操作) Aria:crash-safe 文件: tbl_name.frm: 表格式定義 tbl_name.MYD: 數據文件 tbl_name.MYI: 索引文件 特性: 加鎖和併發:表級鎖 修復:手工或自動修復、但可能丟失數據 索引:非聚集索引 延遲更新索引鍵: 壓縮表 行格式:dynamic動態, fixed固定, compressed, compact, redundent
其他存儲引擎 CSV:將普通的CSV(字段通過逗號分隔)作爲MySQL表使用; MRG_MYISAM:將多個MyISAM表合併成爲一個虛擬表; BLACKHOLE:類似於/dev/null,不真正存儲任何數據; MEMORY:所有數據都保存於內存中,內存表;支持hash索引;表級鎖;臨時表 PERFORMANCE_SCHEMA:僞存儲引擎; ARCHIVE:只支持SELECT和INSERT操作;支持行級鎖和專用緩存區; FEDERATED:用於訪問其它遠程MySQL服務器一個代理,它通過創建一個到遠程MySQL服務器的客戶端連接,並將查詢傳輸到遠程服務器執行,而後完成數據存取; 在MariaDB的上實現是FederatedX MariaDB支持的其它存儲引擎: OQGraph SphinxSE TokuDB Cassandra CONNECT SQUENCE
mariaDB鎖機制實現併發控制:
讀鎖:共享鎖
寫鎖:獨佔鎖
鎖粒度:
表級鎖
行級鎖
鎖類型:
隱式鎖:由存儲引擎自動施加鎖;
顯式鎖:手動施加鎖;
鎖策略:在鎖粒度及數據安全性尋求的平衡機制;
每種存儲引擎都可以自行實現其鎖策略和鎖粒度;
MySQL在服務器級也實現了鎖,表級鎖;用戶可顯式(手動)請求,看下面示例:
MariaDB [mydb]> LOCK TABLES t1 READ; Query OK, 0 rows affected (0.00 sec) 添加一個寫鎖: MariaDB [mydb]> UNLOCK TABLES; Query OK, 0 rows affected (0.00 sec) MariaDB [mydb]> LOCK TABLES t1 WRITE; Query OK, 0 rows affected (0.00 sec) 在另一個線程執行SELECT操作就會卡住,被阻塞: MariaDB [mydb]> SELECT * FROM t1;
事務:一組原子性的SQL查詢,或者說一個獨立工作單元。
ACID測試:
A:atomicity,原子性;整個事務中的所有操作要麼全部成功執行,要麼全部失敗後回滾;
C:consistency, 一致性;數據庫總是從一個一致性狀態轉換爲另一個一致性狀態;
I:Isolation隔離性;一個事務所做出的操作在提交之前,是不能爲其它所見;隔離有多種隔離級別;
D:durability: 持久性;一旦事務提交,其所做的修改會永久保存於數據庫中;
事務:
啓動事務:START TRANSACTION
結束事務:
(1) COMMIT:提交
(2) ROLLBACK: 回滾
流程:
注意:只有事務型存儲引擎(比如innodb)方能支持此類操作; 建議:顯式請求和提交事務,而不要使用“自動提交”功能;自動會降低系統性能 MariaDB [hellodb]> SHOW GLOBAL VARIABLES LIKE '%autoc%' -> ; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | autocommit | ON | +---------------+-------+ 1 row in set (0.00 sec) 關閉自動提交: MariaDB [hellodb]> SET SESSION autocommit=0; Query OK, 0 rows affected (0.00 sec) MariaDB [hellodb]> SHOW VARIABLES LIKE '%autoc%' -> ; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | autocommit | OFF | +---------------+-------+ 1 row in set (0.00 sec) 示例: 開啓事務: MariaDB [hellodb]> START TRANSACTION; Query OK, 0 rows affected (0.00 sec) 刪除兩行: MariaDB [hellodb]> DELETE FROM students WHERE StuID in (18,23); Query OK, 2 rows affected (0.00 sec) 查看發現被刪除了: MariaDB [hellodb]> SELECT * FROM students; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 7 | Xi Ren | 19 | F | 3 | NULL | | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 10 | Yue Lingshan | 19 | F | 3 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 12 | Wen Qingqing | 19 | F | 1 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 15 | Duan Yu | 19 | M | 4 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 19 | Xue Baochai | 18 | F | 6 | NULL | | 20 | Diao Chan | 19 | F | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 24 | Xu Xian | 27 | M | NULL | NULL | | 25 | Sun Dasheng | 100 | M | NULL | NULL | +-------+---------------+-----+--------+---------+-----------+ 23 rows in set (0.00 sec) 回滾 MariaDB [hellodb]> ROLLBACK; Query OK, 0 rows affected (0.00 sec) 查看發現剛剛刪除的馬超和華容又回來了: MariaDB [hellodb]> SELECT * FROM students; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 7 | Xi Ren | 19 | F | 3 | NULL | | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 10 | Yue Lingshan | 19 | F | 3 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 12 | Wen Qingqing | 19 | F | 1 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 15 | Duan Yu | 19 | M | 4 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | | 19 | Xue Baochai | 18 | F | 6 | NULL | | 20 | Diao Chan | 19 | F | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | | 24 | Xu Xian | 27 | M | NULL | NULL | | 25 | Sun Dasheng | 100 | M | NULL | NULL | +-------+---------------+-----+--------+---------+-----------+ 25 rows in set (0.00 sec) 事務相關命令幫助: MariaDB [(none)]> HELP Transactions You asked for help about help category: "Transactions" For more information, type 'help <item>', where <item> is one of the following topics: CHANGE MASTER TO DEALLOCATE PREPARE EXECUTE STATEMENT ISOLATION LOCK PREPARE PURGE BINARY LOGS RESET MASTER RESET SLAVE SAVEPOINT SET GLOBAL SQL_SLAVE_SKIP_COUNTER SET SQL_LOG_BIN START SLAVE START TRANSACTION STOP SLAVE
巧用SAVEPOINT:
MariaDB [hellodb]> INSERT INTO students (Name,Age,Gender) VALUEs ('Jinjiao King',100,'M'); Query OK, 1 row affected (0.00 sec) 創建回滾點: MariaDB [hellodb]> SAVEPOINT sp1; Query OK, 0 rows affected (0.00 sec) 查看:金角大王存在 MariaDB [hellodb]> SELECT * FROM students; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 7 | Xi Ren | 19 | F | 3 | NULL | | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 10 | Yue Lingshan | 19 | F | 3 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 12 | Wen Qingqing | 19 | F | 1 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 15 | Duan Yu | 19 | M | 4 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | | 19 | Xue Baochai | 18 | F | 6 | NULL | | 20 | Diao Chan | 19 | F | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | | 24 | Xu Xian | 27 | M | NULL | NULL | | 25 | Sun Dasheng | 100 | M | NULL | NULL | | 28 | Jinjiao King | 100 | M | NULL | NULL | +-------+---------------+-----+--------+---------+-----------+ 26 rows in set (0.00 sec) 刪除金角大王: MariaDB [hellodb]> DELETE FROM students WHERE StuID = 28; Query OK, 1 row affected (0.00 sec) 金角大王被刪除了: MariaDB [hellodb]> SELECT * FROM students; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 7 | Xi Ren | 19 | F | 3 | NULL | | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 10 | Yue Lingshan | 19 | F | 3 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 12 | Wen Qingqing | 19 | F | 1 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 15 | Duan Yu | 19 | M | 4 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | | 19 | Xue Baochai | 18 | F | 6 | NULL | | 20 | Diao Chan | 19 | F | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | | 24 | Xu Xian | 27 | M | NULL | NULL | | 25 | Sun Dasheng | 100 | M | NULL | NULL | +-------+---------------+-----+--------+---------+-----------+ 25 rows in set (0.00 sec) 回滾到回滾點1: MariaDB [hellodb]> ROLLBACK TO sp1; Query OK, 0 rows affected (0.00 sec) 金角大王又回來了: MariaDB [hellodb]> SELECT * FROM students; +-------+---------------+-----+--------+---------+-----------+ | StuID | Name | Age | Gender | ClassID | TeacherID | +-------+---------------+-----+--------+---------+-----------+ | 1 | Shi Zhongyu | 22 | M | 2 | 3 | | 2 | Shi Potian | 22 | M | 1 | 7 | | 3 | Xie Yanke | 53 | M | 2 | 16 | | 4 | Ding Dian | 32 | M | 4 | 4 | | 5 | Yu Yutong | 26 | M | 3 | 1 | | 6 | Shi Qing | 46 | M | 5 | NULL | | 7 | Xi Ren | 19 | F | 3 | NULL | | 8 | Lin Daiyu | 17 | F | 7 | NULL | | 9 | Ren Yingying | 20 | F | 6 | NULL | | 10 | Yue Lingshan | 19 | F | 3 | NULL | | 11 | Yuan Chengzhi | 23 | M | 6 | NULL | | 12 | Wen Qingqing | 19 | F | 1 | NULL | | 13 | Tian Boguang | 33 | M | 2 | NULL | | 14 | Lu Wushuang | 17 | F | 3 | NULL | | 15 | Duan Yu | 19 | M | 4 | NULL | | 16 | Xu Zhu | 21 | M | 1 | NULL | | 17 | Lin Chong | 25 | M | 4 | NULL | | 18 | Hua Rong | 23 | M | 7 | NULL | | 19 | Xue Baochai | 18 | F | 6 | NULL | | 20 | Diao Chan | 19 | F | 7 | NULL | | 21 | Huang Yueying | 22 | F | 6 | NULL | | 22 | Xiao Qiao | 20 | F | 1 | NULL | | 23 | Ma Chao | 23 | M | 4 | NULL | | 24 | Xu Xian | 27 | M | NULL | NULL | | 25 | Sun Dasheng | 100 | M | NULL | NULL | | 28 | Jinjiao King | 100 | M | NULL | NULL | +-------+---------------+-----+--------+---------+-----------+ 26 rows in set (0.00 sec)
事務隔離級別:
READ UNCOMMITTED 即使別人沒提交我們也能看到(讀未提交)---存在問題1,2,3
READ COMMITTED別人提交了我們才能看到 (讀提交)----存在問題2,3
REPEATABLE READ只要自己沒提交,看到的數據肯定是不變的,不管別人如何改變此數據 (可重讀)----存在問題3 默認級別
SERIALIZABILE (可串行化)-----存在問題4,併發能力最低,安全性最高
可能存在問題:
1.髒讀:讀到別人未提交的數據,別人一旦回滾,可能出現讀到的數據和別人真實擁有的不一致
2.不可重複讀:在別人提交前讀了一次,別人改變數據提交後又讀了一次,兩次不一致
3.幻讀:自欺欺人式讀法,即使別人改了數據,自己還認爲讀到的原來的是正確的
4.加鎖讀:數據被加鎖,別人無法讀到
更改隔離級別:
MariaDB [hellodb]> SHOW PROCESSLIST; +----+------+-----------+---------+---------+------+-------+------------------+----------+ | Id | User | Host | db | Command | Time | State | Info | Progress | +----+------+-----------+---------+---------+------+-------+------------------+----------+ | 5 | root | localhost | mydb | Sleep | 7952 | | NULL | 0.000 | | 8 | root | localhost | hellodb | Query | 0 | NULL | SHOW PROCESSLIST | 0.000 | +----+------+-----------+---------+---------+------+-------+------------------+----------+ 2 rows in set (0.03 sec) 定義mysql隔離級別的變量,可在session級別進行修改,默認是REPEATABLE READ: MariaDB [hellodb]> SHOW GLOBAL VARIABLES LIKE '%isola%'; +---------------+-----------------+ | Variable_name | Value | +---------------+-----------------+ | tx_isolation | REPEATABLE-READ | +---------------+-----------------+ 1 row in set (0.00 sec) 修改會話的隔離級別: MariaDB [hellodb]> SET tx_isolation='READ-UNCOMMITTED'; Query OK, 0 rows affected (0.00 sec) MariaDB [hellodb]> SHOW GLOBAL VARIABLES LIKE '%isola%'; +---------------+-----------------+ | Variable_name | Value | +---------------+-----------------+ | tx_isolation | REPEATABLE-READ | +---------------+-----------------+ 1 row in set (0.01 sec)
死鎖:兩個或多個事務在同一資源相互佔用,並請求鎖定對方佔用的資源的狀態;
事務日誌:磁盤上一段連續的空間記錄事務日誌,只要一提交,馬上寫入到事務日誌文件,完成後才寫進數據文件中,事務日誌一般應該與數據文件放置在不同磁盤,但是如果使用lvm做快照就只能放在同一個磁盤上
事務日誌的寫入類型爲“追加”,因此其操作爲“順序IO”;此日誌通常也被稱爲“預寫式日誌(write ahead logging)”;避免了直接寫入數據文件產生大量的隨機IO
修改以下三個數據庫參數時應關閉數據庫,通過修改配置文件修改
innodb_log_file_size 定義事務日誌文件大小,不建議太大
innodb_log_files_in_group 存儲位置
innodb_log_group_home_dir
MariaDB [hellodb]> SHOW GLOBAL VARIABLES LIKE 'innodb_log%' -> ; +---------------------------+---------+ | Variable_name | Value | +---------------------------+---------+ | innodb_log_block_size | 512 | | innodb_log_buffer_size | 8388608 | | innodb_log_file_size | 5242880 | | innodb_log_files_in_group | 2 | | innodb_log_group_home_dir | ./ | +---------------------------+---------+ 5 rows in set (0.00 sec)