Python抓取CSV文件中數據【V2】

前面 《Python抓取CSV文件中數據》是通過Python 的csv模塊直接調用。但這周自己在做一個one-person project的時候，發現這樣後面數據管理會有些不方便，就轉而選擇數據庫pysql了（安裝：pip install pymysql）。

第一步，安裝MySQL，通過Table Data Import wizard導入.csv文件。

第二步，MySQL測試鏈接。

第三步，Python實現與MySQL的鏈接：

mycon = pymysql.connect(host='127.0.0.1', user='*******', passwd='******', db='csvimages', port=3306, charset='utf8')
mycur =mycon.cursor()
mycur.execute('select * from imgs where(`imgs`.`OriginalURL` like "%http%" and Title like "%ic%")')

Test Result :

第四步，實現：

# -*- coding : utf-8 -*-
import pymysql
import os
import urllib2
import threading


class CsvReaderImageTwo(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def action(self):
            try:
                mycon = pymysql.connect(host='127.0.0.1', user='******', passwd='******',
                                        db='csvimages', port=3306, charset='utf8')
                mycur =mycon.cursor()
                mycur.execute('select * from imgs where(`imgs`.`OriginalURL` like "%http%" and Title like "%ic%")')
                mydata = mycur.fetchall()
                mycon.commit()
                for data in mydata:
                    url = data[2]
                    filename = url.split('/')[-1]
                    # print (filename)
                    print("Downloading:", url)
                    renum = 6
                    while os.path.exists(filename) == False and renum > 0:
                        try:
                            req = urllib2.Request(url)
                            web = urllib2.urlopen(req, timeout=3)
                            img = open(filename, 'wb')
                            img.write(web.read())
                            img.close()
                            break
                        except IOError as e:
                            print(e)
                        except Exception as e:
                            os.remove(filename)
                            print(e)
                        renum -= 1
            finally:
                mycon.close()
            
if __name__ == '__main__':
    for _ in range(3):
        D = CsvReaderImageTwo()
        D.action()

Downloading Result:

這個run 成功，後面one-person project就可以通過數據庫查詢、添加了，再通過PyQt加UI界面，一個實用小程序就OK了