TFLite outputs different results for the same input

The model was converted successfully, but the output differs for each execution. The same does not happen for the non-converted TensorFlow model, which always yields the same results. I've attached a notebook that shows how to reproduce the problem and how to detect it. After a few runs, the output gets more consistent. It only happens when using LSTMs and seems to get even worse when using return_sequences=True.

Notebook Link: https://github.com/BernardoGO/tflite_convert_bug/blob/main/BiLSTM%20Conversion%20Test.ipynb

My model:

new_model = Sequential()  new_model.add(Input(name='input', batch_size=1, shape=(925, 3)) )  new_model.add(Dense(8))  new_model.add(LSTM(64, return_sequences=True))  new_model.add(LSTM(64, return_sequences=True))  new_model.add(LSTM(64, return_sequences=True))  new_model.add(LSTM(64, return_sequences=True))  new_model.add(Activation('softmax', name='softmax'))    new_model.summary()  

Command used to run the converter

converter = tf.lite.TFLiteConverter.from_keras_model(new_model)  #converter.experimental_new_converter = True  converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]  #converter.allow_custom_ops=True  tflite_model = converter.convert()    open("tf_test.tflite", "wb").write(tflite_model)  

The output from the converter invocation

INFO:tensorflow:Assets written to: /tmp/tmp4xjrqwpb/assets  Out[9]: 476052  

• I've used this function to check if the output is the same after each execution:

def equal(tensor1, tensor2):      for i, j in zip(tensor1.reshape(-1), tensor2.reshape(-1)):          if abs(i - j) > 0.001:              return False      return True  

System information

• OS Platform: Ubuntu 20.04
• TensorFlow installed Binary
• TensorFlow version: 2.3.1

https://stackoverflow.com/questions/65855602/tflite-outputs-different-results-for-the-same-input January 23, 2021 at 12:00PM