[python爬蟲之路day7]：實戰之中國天氣網全國城市天氣情況爬取

通過今天的學習，我們將中國天氣網的所有城市天氣信息按照最低溫度的排序爬取出來，並將排名前10的城市可視化。
通過本次學習又溫習了以下：
1.sort函數，可以排序，但是數據必須是整型數據，
2.pyecharts的Bar庫，可以進行繪製表格。

代碼如下：

import requests
from bs4 import BeautifulSoup
from pyecharts import Bar

ALL_DATA = []


def parse_pade(url):
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36"}
    response = requests.get(url, headers=headers)
    text = response.content.decode('utf-8')
    soup = BeautifulSoup(text, "html5lib")
    conMidtab = soup.find('div', class_="conMidtab")
    tables = conMidtab.find_all("table")
    for table in tables:
        trs = table.find_all("tr")[2:]
        for index, tr in enumerate(trs):
            tds = tr.find_all("td")
            city_td = tds[-8]
            weather_td = tds[-7]
            city = list(city_td.stripped_strings)[0]
            weather = list(weather_td.stripped_strings)[0]
            low_td = tds[-2]
            low = list(low_td.stripped_strings)[0]
            high_td = tds[-5]
            high = list(high_td.stripped_strings)[0]
            ALL_DATA.append({"city": city, "weather": weather, "low_feel": int(low), "high_feel": high})


def main():
    urls = ["http://www.weather.com.cn/textFC/hb.shtml#", "http://www.weather.com.cn/textFC/db.shtml#",
            "http://www.weather.com.cn/textFC/hd.shtml#", "http://www.weather.com.cn/textFC/hz.shtml#",
            "http://www.weather.com.cn/textFC/hn.shtml#", "http://www.weather.com.cn/textFC/xb.shtml#",
            "http://www.weather.com.cn/textFC/xn.shtml#", "http://www.weather.com.cn/textFC/gat.shtml#"]
    for url in urls:
        parse_pade(url)
    # 按最低氣溫排序
    ALL_DATA.sort(key=lambda data: data['low_feel'])
    print(ALL_DATA)
    # 前十城市可視化
    chart = Bar("中國最低氣溫排行榜")
    data = ALL_DATA[0:10]
    cities = list(map(lambda x: x['city'], data))
    low_feels = list(map(lambda x: x['low_feel'], data))
    chart.add(" ", cities, low_feels)
    chart.render("weather.html")


if __name__ == '__main__':
    main()

運行結果：
C:\python38\python.exe “C:/python38/new project/2rd/day7.py”
[{‘city’: ‘大興安嶺’, ‘weather’: ‘晴’, ‘low_feel’: -27, ‘high_feel’: ‘-6’}, {‘city’: ‘呼倫貝爾’, ‘weather’: ‘晴’, ‘low_feel’: -24, ‘high_feel’: ‘-11’}, {‘city’: ‘阿勒泰’, ‘weather’: ‘小雪’, ‘low_feel’: -23, ‘high_feel’: ‘-5’}, {‘city’: ‘北屯’, ‘weather’: ‘小雪’, ‘low_feel’: -23, ‘high_feel’: ‘-5’}, {‘city’: ‘雙河’, ‘weather’: ‘小雪’, ‘low_feel’: -23, ‘high_feel’: ‘-5’}, {‘city’: ‘佳木斯’, ‘weather’: ‘多雲’, ‘low_feel’: -22, ‘high_feel’: ‘-2’}, {‘city’: ‘黑河’, ‘weather’: ‘多雲’, ‘low_feel’: -22, ‘high_feel’: ‘-8’}, {‘city’: ‘伊春’, ‘weather’: ‘晴’, ‘low_feel’: -22, ‘high_feel’: ‘-4’}, {‘city’: ‘齊齊哈爾’, ‘weather’: ‘晴’, ‘low_feel’: -21, ‘high_feel’: ‘-5’}, {‘city’: ‘錫林郭勒’, ‘weather’: ‘晴’, ‘low_feel’: -20, ‘high_feel’: ‘-7’}, {‘city’: ‘鶴崗’, ‘weather’: ‘多雲’, ‘low_feel’: -20, ‘high_feel’: ‘-3’}, {‘city’: ‘白山’, ‘weather’: ‘晴’, ‘low_feel’: -19, ‘high_feel’: ‘-1’}, {‘city’: ‘哈爾濱’, ‘weather’: ‘晴’, ‘low_feel’: -18, ‘high_feel’: ‘-3’}, 刪除部分結果**{‘city’: ‘七臺河’, ‘weather’: ‘多雲’, ‘low_feel’: -18, ‘high_feel’: ‘-4’}, {‘city’: ‘海北’, ‘weather’: ‘多雲’, ‘low_feel’: -18, ‘high_feel’: ‘3’}, {‘city’: ‘巴彥淖爾’, ‘weather’: ‘晴’, ‘low_feel’: -6, ‘high_feel’: ‘5’}, {‘city’: ‘葫蘆島’, ‘weather’: ‘多雲’, ‘low_feel’: -6, ‘high_feel’: ‘5’}, {‘city’: ‘延安’, ‘weather’: ‘晴’, ‘low_feel’: -6, ‘high_feel’: ‘10’}, {‘city’: ‘榆林’, ‘weather’: ‘晴’, ‘low_feel’: -6, ‘high_feel’: ‘7’}, {‘city’: ‘榮昌’, ‘weather’: ‘多雲’, ‘low_feel’: 6, ‘high_feel’: ‘14’}, {‘city’: ‘銅梁’, ‘weather’: ‘多雲’, ‘low_feel’: 6, ‘high_feel’: ‘14’}, {‘city’: ‘璧山’, ‘weather’: ‘多雲’, ‘low_feel’: 6, ‘high_feel’: ‘14’}, {‘city’: ‘豐都’, ‘weather’: ‘多雲’, ‘low_feel’: 6, ‘high_feel’: ‘13’}, {‘city’: ‘武隆’, ‘weather’: ‘多雲’, ‘low_feel’: 6, ‘high_feel’: ‘12’}, {‘city’: ‘綦江’, ‘weather’: ‘多雲’, ‘low_feel’: 6, ‘high_feel’: ‘12’}, {‘city’: ‘遵義’, ‘weather’: ‘多雲’, ‘low_feel’: 6, ‘high_feel’: ‘13’}, {‘city’: ‘曲靖’, ‘weather’: ‘多雲’, ‘low_feel’: 6, ‘high_feel’: ‘18’}, {‘city’: ‘楚雄’, ‘weather’: ‘晴’, ‘low_feel’: 6, ‘high_feel’: ‘19’}, {‘city’: ‘上海’, ‘weather’: ‘多雲’, ‘low_feel’: 7, ‘high_feel’: ‘11’}, {‘city’: ‘黃浦’, ‘weather’: ‘多雲’, ‘low_feel’: 7, ‘high_feel’: ‘11’},{‘city’: ‘貴港’, ‘weather’: ‘陰’, ‘low_feel’: 13, ‘high_feel’: ‘19’}, {‘city’: ‘防城港’, ‘weather’: ‘陰’, ‘low_feel’: 13, ‘high_feel’: ‘18’}, {‘city’: ‘深圳’, ‘weather’: ‘晴’, ‘low_feel’: 13, ‘high_feel’: ‘20’}, {‘city’: ‘珠海’, ‘weather’: ‘晴’, ‘low_feel’: 13, ‘high_feel’: ‘18’}, {‘city’: ‘西雙版納’, ‘weather’: ‘晴’, ‘low_feel’: 13, ‘high_feel’: ‘26’}, {‘city’: ‘香港’, ‘weather’: ‘多雲’, ‘low_feel’: 13, ‘high_feel’: ‘19’}, {‘city’: ‘澳門’, ‘weather’: ‘晴’, ‘low_feel’: 13, ‘high_feel’: ‘18’}, {‘city’: ‘欽州’, ‘weather’: ‘陰’, ‘low_feel’: 14, ‘high_feel’: ‘20’}, {‘city’: ‘儋州’, ‘weather’: ‘多雲’, ‘low_feel’: 14, ‘high_feel’: ‘23’}, {‘city’: ‘白沙’, ‘weather’: ‘多雲’, ‘low_feel’: 14, ‘high_feel’: ‘23’}, {‘city’: ‘五指山’, ‘weather’: ‘多雲’, ‘low_feel’: 14, ‘high_feel’: ‘22’}, {‘city’: ‘臺北’, ‘weather’: ‘多雲’, ‘low_feel’: 14, ‘high_feel’: ‘19’}, {‘city’: ‘北海’, ‘weather’: ‘多雲’, ‘low_feel’: 15, ‘high_feel’: ‘20’}, {‘city’: ‘茂名’, ‘weather’: ‘多雲’, ‘low_feel’: 15, ‘high_feel’: ‘23’}, {‘city’: ‘澄邁’, ‘weather’: ‘多雲’, ‘low_feel’: 15, ‘high_feel’: ‘22’}, {‘city’: ‘海口’, ‘weather’: ‘多雲’, ‘low_feel’: 17, ‘high_feel’: ‘20’}, {‘city’: ‘昌江’, ‘weather’: ‘多雲’, ‘low_feel’: 17, ‘high_feel’: ‘24’}, {‘city’: ‘瓊海’, ‘weather’: ‘多雲’, ‘low_feel’: 17, ‘high_feel’: ‘22’}, {‘city’: ‘保亭’, ‘weather’: ‘多雲’, ‘low_feel’: 17, ‘high_feel’: ‘25’}, {‘city’: ‘萬寧’, ‘weather’: ‘小雨’, ‘low_feel’: 17, ‘high_feel’: ‘23’}, {‘city’: ‘高雄’, ‘weather’: ‘多雲’, ‘low_feel’: 17, ‘high_feel’: ‘21’}, {‘city’: ‘三亞’, ‘weather’: ‘多雲’, ‘low_feel’: 18, ‘high_feel’: ‘25’}, {‘city’: ‘東方’, ‘weather’: ‘多雲’, ‘low_feel’: 18, ‘high_feel’: ‘24’}, {‘city’: ‘文昌’, ‘weather’: ‘多雲’, ‘low_feel’: 18, ‘high_feel’: ‘22’}, {‘city’: ‘陵水’, ‘weather’: ‘多雲’, ‘low_feel’: 18, ‘high_feel’: ‘23’}, {‘city’: ‘西沙’, ‘weather’: ‘多雲’, ‘low_feel’: 22, ‘high_feel’: ‘27’}, {‘city’: ‘中沙’, ‘weather’: ‘小雨’, ‘low_feel’: 24, ‘high_feel’: ‘29’}, {‘city’: ‘南沙’, ‘weather’: ‘小雨’, ‘low_feel’: 25, ‘high_feel’: ‘29’}]

Process finished with exit code 0
可視化結果如下：

本人將持續更新自己的python爬蟲之路，歡迎小夥伴關注,大家共同進步。