首先介紹下MySQLdb、DBUtil、sqlobject:
(1)MySQLdb 是用於Python連接Mysql數據庫的接口,它實現了 Python 數據庫 API 規範 V2.0,基於 MySQL C API 上建立的。除了MySQLdb外,python還可以通過oursql, PyMySQL, myconnpy等模塊實現MySQL數據庫操作;
(2)DBUtil中提供了幾種連接池,用以提高數據庫的訪問性能,例如PooledDB,PesistentDB等
(3)sqlobject可以實現數據庫ORM映射的第三方模塊,可以以對象、實例的方式輕鬆操作數據庫中記錄。
爲測試這三者的性能,簡單做一個例子:50個併發訪問4000條記錄的單表,數據庫記錄如下:
測試代碼如下:
1、MySQLdb的代碼如下,其中在connDB()中把連接池相關代碼暫時做了一個註釋,去掉這個註釋既可以使用連接池來創建數據庫連接:
(1)DBOperator.py
import MySQLdb from stockmining.stocks.setting import LoggerFactory import connectionpool class DBOperator(object): def __init__(self): self.logger = LoggerFactory.getLogger('DBOperator') self.conn = None def connDB(self): self.conn=MySQLdb.connect(host="127.0.0.1",user="root",passwd="root",db="pystock",port=3307,charset="utf8") #當需要使用連接池的時候開啓 #self.conn=connectionpool.pool.connection() return self.conn def closeDB(self): if(self.conn != None): self.conn.close() def execute(self, sql): try: if(self.conn != None): cursor = self.conn.cursor() else: raise MySQLdb.Error('No connection') n = cursor.execute(sql) return n except MySQLdb.Error,e: self.logger.error("Mysql Error %d: %s" % (e.args[0], e.args[1])) def findBySQL(self, sql): try: if(self.conn != None): cursor = self.conn.cursor() else: raise MySQLdb.Error('No connection') cursor.execute(sql) rows = cursor.fetchall() return rows except MySQLdb.Error,e: self.logger.error("Mysql Error %d: %s" % (e.args[0], e.args[1]))
(2)測試代碼testMysql.py,做了50個併發,對獲取到的數據庫記錄做了個簡單遍歷。
import threading import time import DBOperator def run(): operator = DBOperator() operator.connDB() starttime = time.time() sql = "select * from stock_cash_tencent" peeps = operator.findBySQL(sql) for r in peeps: pass operator.closeDB() endtime = time.time() diff = (endtime - starttime)*1000 print diff def test(): for i in range(50): threading.Thread(target = run).start() time.sleep(1) if __name__ == '__main__': test()
2、連接池相關代碼:
(1)connectionpool.py
from DBUtils import PooledDB import MySQLdb import string maxconn = 30 #最大連接數 mincached = 10 #最小空閒連接 maxcached = 20 #最大空閒連接 maxshared = 30 #最大共享連接 connstring="root#root#127.0.0.1#3307#pystock#utf8" #數據庫地址 dbtype = "mysql" def createConnectionPool(connstring, dbtype): db_conn = connstring.split("#"); if dbtype=='mysql': try: pool = PooledDB.PooledDB(MySQLdb, user=db_conn[0],passwd=db_conn[1],host=db_conn[2],port=string.atoi(db_conn[3]),db=db_conn[4],charset=db_conn[5], mincached=mincached,maxcached=maxcached,maxshared=maxshared,maxconnections=maxconn) return pool except Exception, e: raise Exception,'conn datasource Excepts,%s!!!(%s).'%(db_conn[2],str(e)) return None pool = createConnectionPool(connstring, dbtype)
3、sqlobject相關代碼
(1)connection.py
from sqlobject.mysql import builder conn = builder()(user='root', password='root', host='127.0.0.1', db='pystock', port=3307, charset='utf8')
(2)StockCashTencent.py對應到數據庫中的表,50個併發並作了一個簡單的遍歷。(實際上,如果不做遍歷,只做count()計算,sqlobject性能是相當高的。)
import sqlobject import time from connection import conn import threading class StockCashTencent(sqlobject.SQLObject): _connection = conn code = sqlobject.StringCol() name = sqlobject.StringCol() date = sqlobject.StringCol() main_in_cash = sqlobject.FloatCol() main_out_cash = sqlobject.FloatCol() main_net_cash = sqlobject.FloatCol() main_net_rate= sqlobject.FloatCol() private_in_cash= sqlobject.FloatCol() private_out_cash= sqlobject.FloatCol() private_net_cash= sqlobject.FloatCol() private_net_rate= sqlobject.FloatCol() total_cash= sqlobject.FloatCol() def test(): starttime = time.time() query = StockCashTencent.select(True) for result in query: pass endtime = time.time() diff = (endtime - starttime)*1000 print diff if __name__ == '__main__': for i in range(50): threading.Thread(target = test).start() time.sleep(1)
測試結果如下:
MySQLdb平均(毫秒) | 99.63999271 |
DBUtil平均(毫秒) | 97.07998276 |
sqlobject平均(毫秒) | 343.2999897 |
結論:其實就測試數據而言,MySQLdb單連接和DBUtil連接池的性能並沒有很大的區別(100個併發下也相差無幾),相反sqlobject雖然具有的編程上的便利性,但是卻帶來性能上的巨大不足,在實際中使用哪個模塊就要斟酌而定了。
(馮智傑:PMP、信息系統項目管理師,大型企業高級開發經理,關注金融、供應鏈行業發展,擅長敏捷開發、項目管理、技術研發等)