對之前一節部分進行實踐，使用keras進行實現，keras關於循環神經網絡有多個方法。https://keras.io/zh/layers/recurrent/
- SimpleRNN
- LSTM+CNN
樣本
使用ai挑戰賽用戶評論信息，這裏僅用驗證集的數據（數據量少一些，運行快一些）
標籤取了用戶消費後感受字段。該字段標籤有1'正面情感', 0'中性情感', -1'負面情感', -2'情感傾向未提及'
| 評論內容 | 標籤 |
|---|---|
| 趁着國慶節，一家人在白天在山裏玩耍之後，晚上決定喫李記攪團。 | 1 |
模型
1.導入庫
import pandas as pd
import numpy as np
import json
import os
from keras.preprocessing.text import Tokenizer
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers.embeddings import Embedding
from keras.layers import Dropout, Conv1D, MaxPooling1D, LSTM, Dense, Bidirectional, SimpleRNN
from keras.utils import to_categorical
2.數據預處理
Tokenizer生成字典如下（截取部分）：
{"，": 1, "的": 2, "。": 3, "是": 4, "不": 5, " ": 6, "了": 7, "一": 8, "有": 9, "很": 10, "喫": 11, "\n": 12, "好": 13, "點": 14, "還": 15, "個": 16, "味": 17, "菜": 18, "就": 19, "來": 20, "我": 21, "這": 22, "也": 23, "\"": 24, "人": 25, "！": 26, "大": 27, "上": 28, "家": 29, "都": 30, "道": 31, "面": 32, "多": 33, "以": 34, "在": 35, "肉": 36, "店": 37, "沒": 38, "可": 39, "感": 40, "小": 41, "到": 42, "裏": 43, "覺": 44, "口": 45, "錯": 46, "過": 47, "去": 48, "說": 49, "服": 50, "和": 51, "比": 52, "下": 53, "務": 54, "要": 55, "後": 56, "得": 57, "餐": 58, "子": 59, "們": 60, "魚": 61, "會": 62, "次": 63, "時": 64}
處理主體部分。
data = pd.read_csv('data/sample.csv')
contents = data['content']
#取一列"本次消費感受"作爲標籤數據 2'正面情感', 1'中性情感', -1'負面情感', 0'情感傾向未提及'
labels = data['others_overall_experience']
#文字用數字表示
tokenizer = Tokenizer(filters='\t\n', char_level=True)
tokenizer.fit_on_texts(contents)
word_dict_file = 'build/dictionary.json'
#保存字典 在預測時爲了使用相同的映射這裏需要保存字典 預測時候直接使用該字典進行文字轉數字
if not os.path.exists(os.path.dirname(word_dict_file)):
os.makedirs(os.path.dirname(word_dict_file))
with open(word_dict_file, 'w',encoding='utf-8') as outfile:
json.dump(tokenizer.word_index, outfile, ensure_ascii=False)
num_words = len(tokenizer.word_index)+1
#將文字使用字典轉成數字
contents_tokenizer = tokenizer.texts_to_sequences(contents)
#將文本填充到相同的長度
x_processed = sequence.pad_sequences(contents_tokenizer, maxlen=512, value=0)
train_size = int(data.shape[0] * .75)
x_train, x_test = x_processed[:train_size], x_processed[train_size:]
labels = labels+2 #將-1 0 這樣標籤轉成正數 輸出再減掉即可
labels = to_categorical(labels)
y_train, y_test = labels[:train_size], labels[train_size:]
3.模型
- 使用雙向RNN
- Embedding詞嵌入層 將詞向量映射到維度更低且能有詞與詞之間關係。
- Bidirectional(SimpleRNN(units=512)) 雙向RNN
- Dropout 訓練中每次更新輸入單元的按比率隨機設置爲0 防止過擬合
- Dense 全連接層 輸出4個標籤的概率
- LSTM 長短期記憶網絡
- Conv1D一維卷積 MaxPooling1D 一維池化
- categorical_crossentropy 多分類損失函數 二分類使用binary_crossentropy 優化器adam
3.1 雙向RNN
model1 = Sequential()
model1.add(Embedding(num_words, 32, input_length=512))
model1.add(Dropout(0.2))
model1.add(Bidirectional(SimpleRNN(units=512)))
model1.add(Dropout(0.2))
model1.add(Dense(4, activation='sigmoid'))
model1.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model1.summary())
model1.fit(x_train, y_train, validation_split=0.25, epochs=20, batch_size=512)
網絡結構：
預測結果：
score, acc = model.evaluate(x_test, y_test, verbose=1, batch_size=1024)
print("Model Accuracy: {:0.2f}%".format(acc * 100))
# 66.16%
3.2 LSTM
model2 = Sequential()
model2.add(Embedding(num_words, 32, input_length=512))
model2.add(Dropout(0.2))
model2.add(LSTM(64, recurrent_dropout=0.5))
model2.add(Dropout(0.2))
model2.add(Dense(4, activation='sigmoid'))
model2.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model2.summary())
model2.fit(x_train, y_train, validation_split=0.25, epochs=20, batch_size=512)
網絡結構：
預測結果：
score, acc = model2.evaluate(x_test, y_test, verbose=1, batch_size=1024)
print("Model Accuracy: {:0.2f}%".format(acc * 100))
# 68.67%
3.3 LSTM+ CNN
model3 = Sequential()
model3.add(Embedding(num_words, 32, input_length=512))
model3.add(Dropout(0.2))
model3.add(Conv1D(64, 5, activation='relu'))
model3.add(MaxPooling1D(pool_size=4))
model3.add(LSTM(64, recurrent_dropout=0.2))
model3.add(Dropout(0.2))
model3.add(Dense(4, activation='sigmoid'))
model3.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model3.summary())
model3.fit(x_train, y_train, validation_split=0.25, epochs=20, batch_size=512)
網絡結構：
預測結果：
score, acc = model3.evaluate(x_test, y_test, verbose=1, batch_size=1024)
print("Model Accuracy: {:0.2f}%".format(acc * 100))
# 74.21%
3.4 小結一下
-
從LSTM和RNN對比看，LSTM的對序列模型學習能力更強（當然這裏也涉及到參數調優。是有優化點）
-
在序列模型中加入卷積操作能加快網絡訓練時間
模型調優是個有趣的過程，其他模型調試中~