Python3進行詞頻統計

一、統計序列中元素的頻率
1.以序列中統計元素個數爲例

from random import randint
# 先用隨機庫生成有重複元素的序列
list = [randint(0,10) for _ in range(1,20)]
print(list)

[10,7,10,6,10,5,2,6,1,0,9,0,3,5,2,5,5,3,10]

方法①：新建空字典，循環遍歷做判斷

d={}
for i in list:
    if i not in d:
        d[i]=1
    else:
        d[i]+=1

d
{0:2,1:1,2:2,3:2,5:4,6:2,7:1,9:1,10:4}# 0:2表示0出現2次

方法②：新建一個以序列元素爲鍵，值爲0的字典

# 生成以序列list爲key，value全爲0的字典
c=dict.fromkeys(list,0)

c
{0:0,1:0,2:0,3:0,5:0,6:0,7:0,9:0,10:0}

for i in list:
    c[i]+=1 

c
{0:2,1:1,2:2,3:2,5:4,6:2,7:1,9:1,10:4} 

sorted(c.items(),key=lambda x:x[1],reverse=True)
[(10,4),(5,4),(6,2),(2,2)(0,2),(3,2),(7,1)(9,1)(1,1)]

方法③：使用Collections下的Counter對象

from collections import Counter
c1=Counter(list)

c1
Counter({0:2,1:1,2:2,3:2,5:4,6:2,7:1,9:1,10:4})

c1.most_common(5)
[(10,4),(5,4),(6,2),(2,2)(0,2)]

二、統計一段文本中單詞出現的頻率
1.先用正則表達把文本分割成單詞列表

import re
s='The Zen of Python,by Tim Peters
   Beautiful is better than ugly
   Simple is better than complex
   Sparse is better than dense'
data=re.split(r"\W+",s)
data
['The',
'Zen',
'of',
...
'than',
'dense']

2.實例化Counter對象

c2=Counter（data）
c2
Counter({'Beatuiful':1,
         'Complex':1,
         ...
         'better':3,
         'ugly':1})
c2.most_common(3)
[('better',3)('than',3)('is',3)]