I have a keras model, preprocessing a string in the first layer.
Either with "tf.strings.unicode_decode" or with "tf.io.decode_raw". Unfortunately, both operations are not supported by tf lite.
My question is, is there a way to decode the string tensor "by hand"
(with simple tensorflow and algebra ops)?
What is the structure of a string tensor?
Does it simply contain a pointer to the first char? If so, how do I process this?
I have no experience with tf lite custom ops but maybe it would be the better approach to implement the decoding operation in this way.
My first attempt was to enter the string directly in the form of a char (uint8) array. But tf lite requires that the input shape be "None" only in the first dimension. This means that I have to set the maximum char array length when creating the model. Therefore, I hope that there is a better implementation.
This is the code of my keras model:
def text_preprocess(x):
b = tf.io.decode_raw(x,tf.uint8)
#do tensor operations
return b
input_text = tf.keras.layers.Input( dtype=tf.string, shape=(1,))
embedding = tf.keras.layers.Lambda(text_preprocess)(input_text)
model = tf.keras.Model(inputs=[input_text], outputs=embedding)
Of course, converting this model leads to the warning:
Converting unsupported operation: DecodeRaw