stream_kws_cnn

原創: [email protected]
時間: 2020/06/17
code: google-research/kws_streaming/
paper: Streaming keyword spotting on mobile devices
關於stream kws 代碼上的一些探索

0x00 parser 超參傳入

base_parser.py —> base_parser()

parser = base_parser.base_parser()

python3 train/model_train_eval.py \
--data_dir ../data \
--lr_schedule linear \
--background_volume 0.1 \
--l2_weight_decay 0.0 \
--background_frequency 0.8 \
--split_data 1 \
--time_shift_ms 100.0 \
--how_many_training_steps 10000,5000 \
--learning_rate 0.0001,0.00005 \
--batch_size 100 \
--train_dir ./train_model/cnn/ \
--feature_type 'mfcc_tf' \
--start_checkpoint '' \
--resample 0.15 \
--train 1 \
--window_size_ms 40 \
--window_stride_ms 20 \
--dct_num_features 20 \
--preprocess mfcc \
--feature_type mfcc_tf \
--mel_lower_edge_hertz 20.0 \
--mel_upper_edge_hertz 7000 \
--fft_magnitude_squared 0 \
--mel_num_bins 40 \
cnn \
--cnn_filters 64,64,64,64,128,64,128 \
--cnn_kernel_size '(3,3),(5,3),(5,3),(5,3),(5,2),(5,1),(3,1)' \
--cnn_dilation_rate '(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1)'

---

cnn 模型相關參數傳入
cnn.py —> model_parameters()

parser_cnn = subparsers.add_parser('cnn')
cnn.model_parameters(parser_cnn)

---

繼續更新參數

model_flags.py —> update_flags(FLAGS)

flags = model_flags.update_flags(FLAGS)

• label_count = 14
• desired_samples = 16000
• window_size_samples = 640
• window_stride_samples = 320
• spectrogram_length = 49
• fingerprint_width = 40
• fft_magnitude_squared = False
• summaries_dir = os.path.join(flags.train_dir, ‘logs/’)

0x01 數據預處理

audio_processor = input_data.AudioProcessor(flags)	

不多贅述，大神解釋的很清楚了: 使用Tensorflow識別語音關鍵詞

只不過該項目在之前的基礎上新增了raw和micro 的特徵處理

---

打印flags日誌：

logging.info(flags)

0x02 model 此處選擇 CNN

空洞卷積覈計算方式： $k_{new} = (k - 1) * d + 1$

2.1 輸入( 此處 flags.preprocess = mfcc)

• train:
  • TRAINING
• inference:
  • STREAM_INTERNAL_STATE_INFERENCE
  • STREAM_EXTERNAL_STATE_INFERENCE
  • NON_STREAM_INFERENCE

# cnn.py
input_audio = tf.keras.layers.Input(
        shape=modes.get_input_data_shape(flags, modes.Modes.TRAINING),
        batch_size=flags.batch_size)

• non-streaming and training

  shape = (flags.spectrogram_length, flags.dct_num_features,)
  

  • shape = (49, 20)

• streaming

   shape = (1, flags.dct_num_features,)
  

  • shape = (1, 20)

• 輸入分兩種：

  • 當flags.preprocess爲 raw 時，net = input_audio
  • 當flags.preprocess爲 mfcc or micro 時,

  # it is a self contained model, user need to feed raw audio only
  net = speech_features.SpeechFeatures(
          speech_features.SpeechFeatures.get_params(flags))(
              net)
  

2.2 輸入增維

爲了增加chanel

# input shape = （100，49， 20， 1）
net = tf.keras.backend.expand_dims(net)

2.3 構建網絡

for + zip 簡直就是神來之筆

for filters, kernel_size, activation, dilation_rate, strides in zip(
      parse(flags.cnn_filters), parse(flags.cnn_kernel_size),
      parse(flags.cnn_act), parse(flags.cnn_dilation_rate),
      parse(flags.cnn_strides)):
    net = Stream(
        cell=tf.keras.layers.Conv2D(
            filters=filters,
            kernel_size=kernel_size,
            activation=activation,
            dilation_rate=dilation_rate,
            strides=strides))(
                net)

---

2.3.1 stream()函數

靈魂拷問：

1. 爲什麼conv要加stream()
2. 爲什麼flattern 要加stream()
3. 爲什麼stream() 有效果
4. 爲什麼使用 build()

回答：

1. 爲了產生self.state_shape 這一個變量。
2. 在訓練（卷積）的過程生成每一層的self.state_shape變量，在推理時候，如果是Modes.STREAM_EXTERNAL_STATE_INFERENCE會生成一個新的self.input_state的輸入，如果是Modes.STREAM_INTERNAL_STATE_INFERENCE的時候會生成一個不可訓練的權重文件
4. 當定義網絡時不知道網絡的維度時可以重寫build()函數，用獲得的shape構建網絡

1. __init__()

   - super().\_\_init\_\_(**kwargs)
   
   - self.cell = tf.keras.layers.Conv2D(filters=filters, kernel_size=kernel_size,
                activation=activation, dilation_rate=dilation_rate, strides=strides)
   
   - self.mode = Modes.TRAINING
   
   - self.pad_time_dim = False
   
   - self.state_shape = None
   
   - self.ring_buffer_size_in_time_dim = None
   
   - strides = (1,1)
   
   - dilation_rate = (1,1)
   
   - kernel_size = (3,3)
   
   - self.ring_buffer_size_in_time_dim = 3  # 時間維度上的核大小
   

    # conv
    self.ring_buffer_size_in_time_dim = dilation_rate[0] * (kernel_size[0] - 1) + 1
    # Flattern 
    self.ring_buffer_size_in_time_dim = None
   

2. build() ：

   build()是在__init__()調用之後被調用

     self.state_shape = [
               self.inference_batch_size, self.ring_buffer_size_in_time_dim
           ] + input_shape.as_list()[2:]
   

   self.state_shape = [1, 3, 20, 1]

     # Modes.STREAM_INTERNAL_STATE_INFERENCE
     # 增加一個變量， state 變爲 weight
     self.states = self.add_weight(name='states', shape=self.state_shape, trainable=False,
         initializer=tf.zeros_initializer)
   

3. call() 前向傳播

   • 訓練的時候，no padding，call()如下：

     self.cell(inputs)
     
     # 即 tf.keras.layers.Conv2D()(inputs)
     

   • 推理的時候，假定mode = STREAM_INTERNAL_STATE_INFERENCE

     輸入的第一維必須是1，移除第一幀，補上最後一幀，作爲網絡的輸入

       def _streaming_internal_state(self, inputs):
         # The time dimenstion always has to equal 1 in streaming mode.
         if inputs.shape[1] != 1:
           raise ValueError('inputs.shape[1]: %d must be 1 ' % inputs.shape[1])
     
         # remove latest row [batch_size, (memory_size-1), feature_dim, channel]
         memory = self.states[:, 1:self.ring_buffer_size_in_time_dim, :]
     
         # add new row [batch_size, memory_size, feature_dim, channel]
         memory = tf.keras.backend.concatenate([memory, inputs], 1)
     
         assign_states = self.states.assign(memory)
     
         with tf.control_dependencies([assign_states]):
           return self.cell(memory)
     

2.3.2 網絡結構模型 CNN

logging.info(model.summary())

Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   

=================================================================
input_1 (InputLayer)         [(100, 49, 20)]           0         
_________________________________________________________________
tf_op_layer_ExpandDims (Tens [(100, 49, 20, 1)]        0         
_________________________________________________________________
stream (Stream)                (100, 47, 18, 64)         640       
_________________________________________________________________
stream_1 (Stream)            (100, 43, 16, 64)         61504     
_________________________________________________________________
stream_2 (Stream)            (100, 35, 14, 64)         61504     
_________________________________________________________________
stream_3 (Stream)            (100, 31, 12, 64)         61504     
_________________________________________________________________
stream_4 (Stream)            (100, 23, 11, 128)        82048     
_________________________________________________________________
stream_5 (Stream)            (100, 19, 11, 64)         41024     
_________________________________________________________________
stream_6 (Stream)            (100, 1, 11, 128)         82048     
_________________________________________________________________
stream_7 (Stream)            (100, 1408)        	  	    0         
_________________________________________________________________
dropout (Dropout)            (100, 1408)     		          0         
_________________________________________________________________
dense (Dense)                     (100, 128)                   180352    
_________________________________________________________________
dense_1 (Dense)                (100, 256)                    33024     
_________________________________________________________________
dense_2 (Dense)                (100, 14)                       3598      

=================================================================
Total params: 607,246
Trainable params: 607,246
Non-trainable params: 0

2.3.3 保存網絡結構模型和參數到 txt

這個函數的代碼可以抄襲，很有借鑑意義

utils.save_model_summary(model, flags.train_dir)

def save_model_summary(model, path, file_name='model_summary.txt'):
  """Saves model topology/summary in text format.

  Args:
    model: Keras model
    path: path where to store model summary
    file_name: model summary file name
  """
  with open(os.path.join(path, file_name), 'wt') as fd:
    stringlist = []
    # print_fn 默認是 print ，傳入的是函數
    model.summary(print_fn=lambda x: stringlist.append(x))  # pylint: disable=unnecessary-lambda
    model_summary = '\n'.join(stringlist)
    fd.write(model_summary)

  # save model and data flags
  with open(os.path.join(flags.train_dir, 'flags.txt'), 'wt') as f:
    # stream 流向f，默認是屏幕
    pprint.pprint(flags, stream=f)

0x03 訓練

訓練的過程一直是 non-stream，不過每一層會產生self.state_shape 這個值

3.1 優化器等參數配置

# 設定loss
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
# optimizer = tf.keras.optimizers.Adam(epsilon=flags.optimizer_epsilon)

# 設定優化器
if flags.optimizer == 'adam':
  optimizer = tf.keras.optimizers.Adam(epsilon=flags.optimizer_epsilon)
elif flags.optimizer == 'momentum':
  optimizer = tf.keras.optimizers.SGD(momentum=0.9)
else:
  raise ValueError('Unsupported optimizer:%s' % flags.optimizer)

model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy'])

# 指定一個文件用來保存圖
train_writer = tf.summary.FileWriter(flags.summaries_dir + '/train',
                                     sess.graph)
validation_writer = tf.summary.FileWriter(flags.summaries_dir + '/validation')

# 初始化所有變量
sess.run(tf.global_variables_initializer())

start_step = 1

logging.info('Training from step: %d ', start_step)

# Save graph.pbtxt.
# 保存圖, 但不保存variable
tf.train.write_graph(sess.graph_def, flags.train_dir, 'graph.pbtxt')

# Save list of words.
# 保存標籤
with tf.io.gfile.GFile(os.path.join(flags.train_dir, 'labels.txt'), 'w') as f:
  f.write('\n'.join(audio_processor.words_list))

best_accuracy = 0.0

# prepare parameters for exp learning rate decay
training_steps_max = np.sum(training_steps_list)
lr_init = learning_rates_list[0]
# exp_rate 與後面的各個階段的學習率有關
exp_rate = -np.log(learning_rates_list[-1] / lr_init)/training_steps_max

3.2 開始訓練

獲取輸入、學習率設定

訓練方式：

result = model.train_on_batch(train_fingerprints, train_ground_truth)

3.3 訓練結束

會生成如下文件

./speech_commands_train
├── accuracy_last.txt  # 訓練結束後的測試集的準確率
├── best_weights.data-00000-of-00002  # 最高準確率的權重保存
├── best_weights.data-00001-of-00002  # 最高準確率的權重保存
├── best_weights.index  # 最高準確率的權重保存
├── flags.txt  # flags 參數保存
├── graph.pbtxt  # 網絡結構圖保存，The model architecture, and training configuration
├── labels.txt  # 標籤保存
├── last_weights.data-00000-of-00002  # 最後一次的權重保存？？
├── last_weights.data-00001-of-00002  # 最後一次的權重保存？？
├── last_weights.index  # 最後一次的權重保存？？
├── logs  # tensorlfow 可視化工具
│   ├── train
│   │   ├── events.out.tfevents.1591857342.ubuntu
│   │   ├── events.out.tfevents.1591857494.ubuntu
│   │   ├── events.out.tfevents.1591870751.RT-AI
│   │   └── events.out.tfevents.1591875621.ubuntu
│   └── validation
│       ├── events.out.tfevents.1591857855.ubuntu
│       └── events.out.tfevents.1591870970.RT-AI
├── model_summary.txt  # 模型結構及參數保存
└── train  # 每經過400次 step 保存一個權重文件，name: acc+weights+steps
    ├── 0weights_400.data-00000-of-00002
    ├── 0weights_400.data-00001-of-00002
    ├── 0weights_400.index
    ├── ...
    ├── 9027weights_15000.data-00000-of-00002
    ├── 9027weights_15000.data-00001-of-00002
    ├── 9027weights_15000.index
    └── checkpoint

---

• 查看權重詳細數值（tensorflow1)

• 查看了之後存在一個問題，所有的權重文件數值都是一樣的，文件的大小也是一樣的，不知道原因

  from tensorflow.python import pywrap_tensorflow
  
  checkpoint_path = 'your saved weight file path'
  # Read data from checkpoint file
  reader = pywrap_tensorflow.NewCheckpointReader(checkpoint_path)
  var_to_shape_map = reader.get_variable_to_shape_map()
  # Print tensor name and values
  for key in var_to_shape_map:
      print("tensor_name: ", key)
      print(reader.get_tensor(key))
  

0x04 保存model

4.1 保存non-stream model

# convert to SavedModel
test.convert_model_saved(flags, 'non_stream', Modes.NON_STREAM_INFERENCE)

# 在convert_model_saved() 函數中使用了這麼一行代碼
# convert_model_saved(flags, folder, mode, weights_name='best_weights')
utils.model_to_saved(model, flags, path_model, mode)

model_to_saved() 的功能是保存網絡結構到txt 和保存網絡結構圖，權重文件保存在variables下面

./non_stream
├── assets  # 未知
├── model_summary.txt
├── saved_model.pb
└── variables
    ├── variables.data-00000-of-00002
    ├── variables.data-00001-of-00002
    └── variables.index

4.2 保存 stream model

test.convert_model_saved(flags, 'stream_state_internal',
                         Modes.STREAM_INTERNAL_STATE_INFERENCE)

同樣調用utils.model_to_saved()這個函數，不同的是mode = Modes.STREAM_INTERNAL_STATE_INFERENCE，

在保存之前，多了這麼一步，這一步完成之後纔是保存網絡結構到txt 和保存網絡結構圖:

# convert non streaming Keras model to Keras streaming model, internal state
model = to_streaming_inference(model_non_stream, flags, mode)

上面的函數的功能是將之前的訓練好的模型轉換成 streaming model

怎麼轉換呢？繼續深入源碼

發現其中有兩個比較重要的函數：

model = _set_mode(model, mode)
new_model = _clone_model(model, input_tensors)

第一個函數是將model.layers 中的所有mode 從TRAINING 改爲STREAM_INTERNAL_STATE_INFERENCE；

第二個函數可就厲害了，功能是克隆網絡結構和configs，

寫的代碼，調用的tensorflow 的API在市面上沒有任何的流通資料，一頭抓瞎，心態裂開，源碼如下：

def _clone_model(model, input_tensors):
  """Clone model with configs, except of weights."""
  new_input_layers = {}  # Cache for created layers.
  # pylint: disable=protected-access
  if input_tensors is not None:
    # Make sure that all input tensors come from a Keras layer.
    input_tensors = tf.nest.flatten(input_tensors)
    for i, input_tensor in enumerate(input_tensors):
      if not tf.keras.backend.is_keras_tensor(input_tensor):
        raise ValueError('Expected keras tensor but get', input_tensor)
      
      # 新舊Input 保存成一個字典 dict[舊] = 新
      original_input_layer = model._input_layers[i]
      newly_created_input_layer = input_tensor._keras_history.layer
      new_input_layers[original_input_layer] = newly_created_input_layer

  # 將原來的第一層 Input層改爲現在的Input層
  # shape [1, 25, 10] --> [1, 1, 10]
  # 如果不是第一層，就copy 各層的config
  model_config, created_layers = models._clone_layers_and_model_config(
      model, new_input_layers, models._clone_layer)
  # pylint: enable=protected-access

  # Reconstruct model from the config, using the cloned layers.
  input_tensors, output_tensors, created_layers = (
      functional.reconstruct_from_config(
          model_config, created_layers=created_layers))

  new_model = tf.keras.Model(input_tensors, output_tensors, name=model.name)
  return new_model

---

此處引發了關於tf版本的一個問題，需要tf_nightly==2.3.0.dev20200515 # was validated on tf_nightly-2.3.0.dev20200515-cp36-cp36m-manylinux2010_x86_64.whl ，未完待續

4.3 最終的文件目錄

⇒  tree ./train_model/cnn
./train_model/cnn
├── accuracy_last.txt
├── best_weights.data-00000-of-00002
├── best_weights.data-00001-of-00002
├── best_weights.index
├── flags.json
├── flags.txt
├── graph.pbtxt
├── labels.txt
├── last_weights.data-00000-of-00002
├── last_weights.data-00001-of-00002
├── last_weights.index
├── logs
│   ├── train
│   │   ├── events.out.tfevents.1591857342.ubuntu
│   │   ├── events.out.tfevents.1591857494.ubuntu
│   │   ├── events.out.tfevents.1591870751.RT-AI
│   │   └── events.out.tfevents.1591875621.ubuntu
│   └── validation
│       ├── events.out.tfevents.1591857855.ubuntu
│       └── events.out.tfevents.1591870970.RT-AI
├── model_summary.txt
├── non_stream
│   ├── assets
│   ├── model_summary.txt
│   ├── saved_model.pb
│   └── variables
│       ├── variables.data-00000-of-00002
│       ├── variables.data-00001-of-00002
│       └── variables.index
├── quantize_opt_for_size_tflite_non_stream
│   ├── model_summary.txt
│   ├── non_stream.tflite
│   └── tflite_non_stream_model_accuracy.txt
├── quantize_opt_for_size_tflite_stream_state_external
│   ├── model_summary.txt
│   ├── stream_state_external.tflite
│   ├── tflite_stream_state_external_model_accuracy_reset0.txt
│   └── tflite_stream_state_external_model_accuracy_reset1.txt
├── stream_state_internal
│   ├── assets
│   ├── model_summary.txt
│   ├── saved_model.pb
│   └── variables
│       ├── variables.data-00000-of-00002
│       ├── variables.data-00001-of-00002
│       └── variables.index
├── tf
│   ├── model_summary_non_stream.png
│   ├── model_summary_non_stream.txt
│   ├── model_summary_stream_state_external.png
│   ├── model_summary_stream_state_external.txt
│   ├── model_summary_stream_state_internal.png
│   ├── model_summary_stream_state_internal.txt
│   ├── stream_state_external_model_accuracy_sub_set_reset0.txt
│   ├── stream_state_external_model_accuracy_sub_set_reset1.txt
│   ├── tf_non_stream_model_accuracy.txt
│   ├── tf_non_stream_model_sampling_stream_accuracy.txt
│   └── tf_stream_state_internal_model_accuracy_sub_set.txt
├── tflite_non_stream
│   ├── model_summary.txt
│   ├── non_stream.tflite
│   └── tflite_non_stream_model_accuracy.txt
├── tflite_stream_state_external
│   ├── model_summary.txt
│   ├── stream_state_external.tflite
│   ├── tflite_stream_state_external_model_accuracy_reset0.txt
│   └── tflite_stream_state_external_model_accuracy_reset1.txt
└── train
    ├── 0weights_400.data-00000-of-00002
    ├── 0weights_400.data-00001-of-00002
    ├── 0weights_400.index
    ├── ...
    ├── 9027weights_15000.data-00000-of-00002
    ├── 9027weights_15000.data-00001-of-00002
    ├── 9027weights_15000.index
    └── checkpoint

0x05 Other 問題集收錄

1 stride_ms 40 & dct_num_features 10 報錯

    ValueError: Negative dimension size caused by subtracting 5 from 4 for '{{node stream_4/conv2d_4/Conv2D}} = Conv2D[T=DT_FLOAT, 
data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true]
(stream_4/conv2d_4/SpaceToBatchND, stream_4/conv2d_4/Conv2D/ReadVariableOp)' with input shapes: [200,4,2,64], [5,2,64,128]

第五層卷積核過大，重新設置參數：

--cnn_kernel_size (3,3),(5,3),(5,3),(5,3),(5,2),(5,1),(3,1) \
--cnn_dilation_rate (1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1) \

出現新的報錯：

    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Name object at 0x7f8ca813a630>

原因：

--units2 128,256 \
--act2 linear,relu

參數傳入有問題，刪除即可

class MyLayer(layers.Layer):
    def __init__(self, input_dim=32, unit=32):
        super(MyLayer, self).__init__()
        self.weight = self.add_weight(shape=(input_dim, unit),
                                     initializer=keras.initializers.RandomNormal(),
                                     trainable=True)
        self.bias = self.add_weight(shape=(unit,),
                                   initializer=keras.initializers.Zeros(),
                                   trainable=True)
    
    def call(self, inputs):
        return tf.matmul(inputs, self.weight) + self.bias

2. 最開始訓練時傳入參數有個坑

末尾需要補上’\’， 否則保存的權重文件會出現在神奇的位置

# 保存訓練的權重
model.save_weights(flags.train_dir + 'train/' +
                         str(int(best_accuracy * 10000)) + 'weights_' +
                         str(training_step))

# 保存每次驗證的最優的權重
model.save_weights(flags.train_dir + 'best_weights')

# 保存最後一次測試的權重
model.save_weights(flags.train_dir + 'last_weights')

3. 模型訓練好之後，non-stream to streaming

WARNING: failed to convert to SavedModel: module 'tensorflow.python.keras.engine.node' has no attribute '_CONSTANT_VALUE'

在 /home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow/python/keras/engine/node.py中，少了一行定義，增加在第25行

_CONSTANT_VALUE = '_CONSTANT_VALUE'

---

下面的解決思想產生了點小偏差，花了一天多一點的時間，最後在下班之前發現是這個原因：

/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/node.py

版本不匹配，包括上面的_CONSTANT_VALUE 值的問題，戰術繞頭，有點懵逼。

在github master分支上的node.py 有output、_CONSTANT_VALUE兩個屬性的，但是在tf2.1 和 2.2 中的node.py 就不存在，所以會報錯

======== update 2020/06/17 版本問題 =========

作者給出的意見，更新到版本 tf_nightly==2.3.0.dev20200515 # was validated on tf_nightly-2.3.0.dev20200515-cp36-cp36m-manylinux2010_x86_64.whl，

問題看上去解決了

未完待續…

（一些迷了路的想法）

後續繼續引發問題：

WARNING: failed to convert to SavedModel: 'Node' object has no attribute 'outputs'
        
# 上述問題代碼出錯在這一行：
(model_config, created_layers) = models._clone_layers_and_model_config(
      model, new_input_layers, models._clone_layer)

問題原因在於在clone的過程中，丟失了每一層的output這個參數：

正常是這樣的：

# 往下面找，發現問題在這
Layer stream has no inbound nodes.

在調用output 屬性之前，有一個叫做inbound_nodes 的東西，這個不存在則導致input & output 不存在

這些inbound nodes是什麼？
A Node describes the connectivity between two layers. Each time a layer is connected to some new input, a node is added to layer._inbound_nodes. Each time the output of a layer is used by another layer, a node is added to layer._outbound_nodes.

4. 在模型訓練好之後轉換成tflite的過程中

tensorflow.python.framework.errors_impl.InvalidArgumentError: Matrix size-incompatible: In[0]: [100,128], In[1]: [1408,128]
	 [[{{node dense/BiasAdd}}]]

這個是傳入的參數不對，訓練時的參數dct是20，測試的時候傳入10，因此報錯

---

新的問題：[在tf版本更新之後，該問題小時了]

I0615 15:00:39.572662 140162058372864 model_train_eval.py:164] run TF non streaming model accuracy evaluation
I0615 15:01:34.501143 140162058372864 test.py:80] TF Final test accuracy on non stream model = 89.95% (N=5600)
I0615 15:02:19.639499 140162058372864 test.py:80] TF Final test accuracy on non stream model = 89.18% (N=5600)
...
I0615 15:02:20.397487 140162058372864 utils.py:216] None
W0615 15:02:20.405573 140162058372864 stream.py:103] There is no need to use Stream on time dim with size 1
W0615 15:02:20.410688 140162058372864 test.py:582] FAILED to convert to mode NON_STREAM_INFERENCE, tflite: 'Node' object has no attribute 'outputs'
Traceback (most recent call last):
  File "train/model_train_eval.py", line 331, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "train/model_train_eval.py", line 202, in main
    test.tflite_non_stream_model_accuracy(flags, folder_name, file_name)
  File "/home/lebhoryi/WakeUp-Xiaorui/kws_streaming/train/test.py", line 498, in tflite_non_stream_model_accuracy
    model_path=os.path.join(path, tflite_model_name))
  File "/home/lebhoryi/anaconda3/envs/tf2/lib/python3.6/site-packages/tensorflow/lite/python/interpreter.py", line 204, in __init__
    model_path, self._custom_op_registerers))
ValueError: Mmap of '/tmp/speech_commands_train/tflite_non_stream/non_stream.tflite' failed.

5. 在non-stream to streaming 過程中

在運行這一行代碼的時候：

  (model_config, created_layers) = models._clone_layers_and_model_config(
      model, new_input_layers, models._clone_layer)

警告如下：

There is no need to use Stream on time dim with size 1

這是因爲在傳入參數的時候，在streaming_7 層時，傳入的self.stae_shape = [1, 1, 11, 128],

if isinstance(self.cell, tf.keras.layers.Flatten):
      # effective kernel size in time dimension
      if self.state_shape:
        self.ring_buffer_size_in_time_dim = self.state_shape[1]
        
if self.ring_buffer_size_in_time_dim == 1:
      logging.warn('There is no need to use Stream on time dim with size 1')