sys.argv[]：從程序外部獲取參數。0是程序自身，其餘是外面輸入的
pycharm的使用方式：shift+alt+F10，edit configuration——Parameters，用空格分隔
train的運行參數：data/small_samples.csv training_config.json
predict的運行參數：trained_results_1590650940/ data/small_samples.csv
main() missing 1 required positional argument: ‘sampleN’
查看函數發現少個輸入參數，怕炸設了個100
猜測原因是，模型存在了trainded_results裏，但是restore的時候讀的checkpoints裏的。
saver.restore(sess, checkpoint_prefix + '-' + str(best_at_step))
把讀取改成了
saver.restore(sess, trained_dir + "best_model.ckpt")
似乎是通了，但是感覺checkpoint文件夾是空的肯定哪裏不對，這可悲的準確率是我sample太小了麼
原來train2是好的，神祕，那train1是幹啥用的
直接運行了一下，結果
D:\software\python37\project\Scripts\python.exe D:/software/python37/project/LogstashAI-master/B0_CNN-RNN-text/train2.py ./data/small_samples.csv ./training_config.json
loading data…
CRITICAL:root:The maximum length is 175
INFO:root:x_train: 8100, x_dev: 900, x_test: 1000
./trained_results_1591344355/
INFO:root:y_train: 8100, y_dev: 900, y_test: 1000
2020-06-05 16:05:55.083799: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
WARNING:tensorflow:From D:\software\python37\project\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From D:\software\python37\project\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:52: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:52: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:57: GRUCell.init (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.GRUCell, and will be replaced by that in Tensorflow 2.0.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:57: GRUCell.init (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.GRUCell, and will be replaced by that in Tensorflow 2.0.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:67: static_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use keras.layers.RNN(cell, unroll=True), which is equivalent to this API
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:67: static_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use keras.layers.RNN(cell, unroll=True), which is equivalent to this API
WARNING:tensorflow:From D:\software\python37\project\lib\site-packages\tensorflow\python\ops\rnn.py:1370: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From D:\software\python37\project\lib\site-packages\tensorflow\python\ops\rnn.py:1370: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:76: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:76: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:90: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.
See tf.nn.softmax_cross_entropy_with_logits_v2.
WARNING:tensorflow:From D:\software\python37\project\LogstashAI-master\B0_CNN-RNN-text\text_cnn_rnn.py:90: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.
See tf.nn.softmax_cross_entropy_with_logits_v2.
1 trainAccuracy 0.0234375
2 trainAccuracy 0.046875
INFO:root:Recall and precision of dev set: 0.0,0.0
INFO:root:Accuracy on dev set: 0.0033333333333333335
CRITICAL:root:Saved model ./checkpoints_1591344355/model-2 at step 2
CRITICAL:root:Best accuracy 0.0033333333333333335 at step 2
3 trainAccuracy 0.046875
4 trainAccuracy 0.0390625
INFO:root:Recall and precision of dev set: 0.0,0.0
INFO:root:Accuracy on dev set: 0.0033333333333333335
CRITICAL:root:Saved model ./checkpoints_1591344355/model-4 at step 4
CRITICAL:root:Best accuracy 0.0033333333333333335 at step 4
5 trainAccuracy 0.0390625
6 trainAccuracy 0.078125
INFO:root:Recall and precision of dev set: 0.0,0.0
INFO:root:Accuracy on dev set: 0.0033333333333333335
CRITICAL:root:Saved model ./checkpoints_1591344355/model-6 at step 6
CRITICAL:root:Best accuracy 0.0033333333333333335 at step 6
7 trainAccuracy 0.03125
8 trainAccuracy 0.09375
INFO:root:Recall and precision of dev set: 0.0,0.0
INFO:root:Accuracy on dev set: 0.0033333333333333335
CRITICAL:root:Saved model ./checkpoints_1591344355/model-8 at step 8
CRITICAL:root:Best accuracy 0.0033333333333333335 at step 8
9 trainAccuracy 0.1875
10 trainAccuracy 0.1328125
INFO:root:Recall and precision of dev set: 0.0015290519877675841,0.5
INFO:root:Accuracy on dev set: 0.0044444444444444444
CRITICAL:root:Saved model ./checkpoints_1591344355/model-10 at step 10
CRITICAL:root:Best accuracy 0.0044444444444444444 at step 10
11 trainAccuracy 0.140625
12 trainAccuracy 0.0859375
INFO:root:Recall and precision of dev set: 0.0045871559633027525,0.75
INFO:root:Accuracy on dev set: 0.006666666666666667
WARNING:tensorflow:From D:\software\python37\project\lib\site-packages\tensorflow\python\training\saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
WARNING:tensorflow:From D:\software\python37\project\lib\site-packages\tensorflow\python\training\saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
CRITICAL:root:Saved model ./checkpoints_1591344355/model-12 at step 12
CRITICAL:root:Best accuracy 0.006666666666666667 at step 12
13 trainAccuracy 0.171875
14 trainAccuracy 0.1328125
INFO:root:Recall and precision of dev set: 0.00764525993883792,0.8333333333333334
INFO:root:Accuracy on dev set: 0.008888888888888889
CRITICAL:root:Saved model ./checkpoints_1591344355/model-14 at step 14
CRITICAL:root:Best accuracy 0.008888888888888889 at step 14
15 trainAccuracy 0.2109375
16 trainAccuracy 0.171875
INFO:root:Recall and precision of dev set: 0.07339449541284404,0.9795918367346939
INFO:root:Accuracy on dev set: 0.056666666666666664
CRITICAL:root:Saved model ./checkpoints_1591344355/model-16 at step 16
CRITICAL:root:Best accuracy 0.056666666666666664 at step 16
17 trainAccuracy 0.1796875
18 trainAccuracy 0.234375
INFO:root:Recall and precision of dev set: 0.2492354740061162,0.9819277108433735
INFO:root:Accuracy on dev set: 0.18222222222222223
CRITICAL:root:Saved model ./checkpoints_1591344355/model-18 at step 18
CRITICAL:root:Best accuracy 0.18222222222222223 at step 18
19 trainAccuracy 0.3515625
20 trainAccuracy 0.2421875
INFO:root:Recall and precision of dev set: 0.7018348623853211,0.9935064935064936
INFO:root:Accuracy on dev set: 0.5111111111111111
CRITICAL:root:Saved model ./checkpoints_1591344355/model-20 at step 20
CRITICAL:root:Best accuracy 0.5111111111111111 at step 20
21 trainAccuracy 0.2578125
22 trainAccuracy 0.421875
INFO:root:Recall and precision of dev set: 0.9709480122324159,0.9937402190923318
INFO:root:Accuracy on dev set: 0.7055555555555556
CRITICAL:root:Saved model ./checkpoints_1591344355/model-22 at step 22
CRITICAL:root:Best accuracy 0.7055555555555556 at step 22
23 trainAccuracy 0.4296875
24 trainAccuracy 0.515625
INFO:root:Recall and precision of dev set: 0.9969418960244648,0.9939024390243902
INFO:root:Accuracy on dev set: 0.7244444444444444
CRITICAL:root:Saved model ./checkpoints_1591344355/model-24 at step 24
CRITICAL:root:Best accuracy 0.7244444444444444 at step 24
25 trainAccuracy 0.390625
26 trainAccuracy 0.5078125
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-26 at step 26
CRITICAL:root:Best accuracy 0.7255555555555555 at step 26
27 trainAccuracy 0.5
28 trainAccuracy 0.5546875
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-28 at step 28
CRITICAL:root:Best accuracy 0.7255555555555555 at step 28
29 trainAccuracy 0.578125
30 trainAccuracy 0.6328125
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-30 at step 30
CRITICAL:root:Best accuracy 0.7255555555555555 at step 30
31 trainAccuracy 0.609375
32 trainAccuracy 0.6640625
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-32 at step 32
CRITICAL:root:Best accuracy 0.7255555555555555 at step 32
33 trainAccuracy 0.6953125
34 trainAccuracy 0.7265625
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-34 at step 34
CRITICAL:root:Best accuracy 0.7255555555555555 at step 34
35 trainAccuracy 0.65625
36 trainAccuracy 0.59375
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-36 at step 36
CRITICAL:root:Best accuracy 0.7255555555555555 at step 36
37 trainAccuracy 0.765625
38 trainAccuracy 0.7421875
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-38 at step 38
CRITICAL:root:Best accuracy 0.7255555555555555 at step 38
39 trainAccuracy 0.671875
40 trainAccuracy 0.65625
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-40 at step 40
CRITICAL:root:Best accuracy 0.7255555555555555 at step 40
41 trainAccuracy 0.6875
42 trainAccuracy 0.7421875
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-42 at step 42
CRITICAL:root:Best accuracy 0.7255555555555555 at step 42
43 trainAccuracy 0.7421875
44 trainAccuracy 0.7734375
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-44 at step 44
CRITICAL:root:Best accuracy 0.7255555555555555 at step 44
45 trainAccuracy 0.7109375
46 trainAccuracy 0.7421875
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-46 at step 46
CRITICAL:root:Best accuracy 0.7255555555555555 at step 46
47 trainAccuracy 0.765625
48 trainAccuracy 0.6171875
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-48 at step 48
CRITICAL:root:Best accuracy 0.7255555555555555 at step 48
49 trainAccuracy 0.71875
50 trainAccuracy 0.734375
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-50 at step 50
CRITICAL:root:Best accuracy 0.7255555555555555 at step 50
51 trainAccuracy 0.78125
52 trainAccuracy 0.6640625
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-52 at step 52
CRITICAL:root:Best accuracy 0.7255555555555555 at step 52
53 trainAccuracy 0.78125
54 trainAccuracy 0.7109375
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-54 at step 54
CRITICAL:root:Best accuracy 0.7255555555555555 at step 54
55 trainAccuracy 0.7421875
56 trainAccuracy 0.6796875
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-56 at step 56
CRITICAL:root:Best accuracy 0.7255555555555555 at step 56
57 trainAccuracy 0.8125
58 trainAccuracy 0.765625
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-58 at step 58
CRITICAL:root:Best accuracy 0.7255555555555555 at step 58
59 trainAccuracy 0.7109375
60 trainAccuracy 0.734375
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-60 at step 60
CRITICAL:root:Best accuracy 0.7255555555555555 at step 60
61 trainAccuracy 0.671875
62 trainAccuracy 0.75
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-62 at step 62
CRITICAL:root:Best accuracy 0.7255555555555555 at step 62
63 trainAccuracy 0.7265625
64 trainAccuracy 0.8611111
INFO:root:Recall and precision of dev set: 0.9984709480122325,0.9939117199391172
INFO:root:Accuracy on dev set: 0.7255555555555555
CRITICAL:root:Saved model ./checkpoints_1591344355/model-64 at step 64
CRITICAL:root:Best accuracy 0.7255555555555555 at step 64
CRITICAL:root:Training is complete, testing the best model on x_test and y_test
WARNING:tensorflow:From D:\software\python37\project\lib\site-packages\tensorflow\python\training\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From D:\software\python37\project\lib\site-packages\tensorflow\python\training\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
INFO:tensorflow:Restoring parameters from ./checkpoints_1591344355/model-64
INFO:tensorflow:Restoring parameters from ./checkpoints_1591344355/model-64
CRITICAL:root:Accuracy on test set: 0.734
Process finished with exit code 0
predict2 報錯
研究半天好像是代碼裏把severity轉化成0/1了，但是讀的還是crit啥的？
把df.loc的三行註釋掉能跑出結果了
預測結果全是debug，肯定不對，浩哥說是富採樣，自己去改訓練集去
果然是訓練集的問題？debug類的太多了，大概每個拎了200個出來以後再跑，正確率就有80%了
然後就是想辦法提高了