用于Keras模型上的预处理/特征工程的Tensorflow 2文档似乎很令人困惑并且不是很友好。
目前,我有一个简单的Keras N层模型,其中TF特征列作为密集层提供。为了进行培训,我使用tf.dataset
API读取了CSV文件,并且编写了一个功能工程功能,该功能可以使用dataset.map
函数创建新功能。
def feature_engg_features(features):
#Add new features
features['nodlgrbyvpatd'] = features['NODLGR'] / features['VPATD']
return(features)
我可以使用tf.keras.models.save_model
方法轻松保存模型。但是,我在弄清楚如何在服务功能中附加feature_engineering
步骤时遇到了麻烦。
要求:现在,我想采用上面相同的功能工程功能并将其附加到我的serving function
上,以便在通过tensorflow_model_server
输入的JSON中,执行相同的功能工程步骤应用。我知道Keras中的lambda Layer选项,但是我想通过saved_model
方法做到这一点,但是这里有很多困难。
例如,下面的代码给出错误:
def feature_engg_features(features):
#Add new features
features['nodlgrbyvpatd'] = features['NODLGR'] / features['VPATD']
return(features)
@tf.function
def serving(data):
data = tf.map_fn(feature_engg_features, data, dtype=tf.float32)
# Predict
predictions = m_(data)
version = "1"
tf.keras.models.save_model(
m_,
"./exported_model/" + version,
overwrite=True,
include_optimizer=True,
save_format=None,
signatures=serving,
options=None
)
错误:
Only `tf.functions` with an input signature or concrete functions can be used as a signature.
上述错误是因为我没有提供我的Keras模型的InputSignature,但是我无法理解我有13个输入字段,这是期望的输入签名。
所以我想知道是否有人知道解决这个问题的最短方法。这是一个非常基本的要求,对于Keras Tensorflow模型服务,Tensorflow似乎使这一问题变得非常复杂。
编辑: 我对其进行了修复,因此必须为每个功能生成并传递TensorSpec,并且还必须在服务函数中调用model()。
@tf.function
def serving(WERKS, DIFGRIRD, SCENARIO, TOTIRQTY, VSTATU, EKGRP, TOTGRQTY, VPATD, EKORG, NODLGR, DIFGRIRV, NODLIR, KTOKK):
##Feature engineering
nodlgrbyvpatd = tf.cast(NODLGR / VPATD, tf.float32)
payload = {
'WERKS': WERKS,
'DIFGRIRD': DIFGRIRD,
'SCENARIO': SCENARIO,
'TOTIRQTY': TOTIRQTY,
'VSTATU': VSTATU,
'EKGRP': EKGRP,
'TOTGRQTY': TOTGRQTY,
'VPATD': VPATD,
'EKORG': EKORG,
'NODLGR': NODLGR,
'DIFGRIRV': DIFGRIRV,
'NODLIR': NODLIR,
'KTOKK': KTOKK,
'nodlgrbyvpatd': nodlgrbyvpatd,
}
## Predict
##IF THERE IS AN ERROR IN NUMBER OF PARAMS PASSED HERE OR DATA TYPE THEN IT GIVES ERROR, "COULDN'T COMPUTE OUTPUT TENSOR"
predictions = m_(payload)
return predictions
serving = serving.get_concrete_function(WERKS=tf.TensorSpec([None,], dtype= tf.string, name='WERKS'),
DIFGRIRD=tf.TensorSpec([None,], name='DIFGRIRD'),
SCENARIO=tf.TensorSpec([None,], dtype= tf.string, name='SCENARIO'),
TOTIRQTY=tf.TensorSpec([None,], name='TOTIRQTY'),
VSTATU=tf.TensorSpec([None,], dtype= tf.string, name='VSTATU'),
EKGRP=tf.TensorSpec([None,], dtype= tf.string, name='EKGRP'),
TOTGRQTY=tf.TensorSpec([None,], name='TOTGRQTY'),
VPATD=tf.TensorSpec([None,], name='VPATD'),
EKORG=tf.TensorSpec([None,], dtype= tf.string, name='EKORG'),
NODLGR=tf.TensorSpec([None,], name='NODLGR'),
DIFGRIRV=tf.TensorSpec([None,], name='DIFGRIRV'),
NODLIR=tf.TensorSpec([None,], name='NODLIR'),
KTOKK=tf.TensorSpec([None,], dtype= tf.string, name='KTOKK')
)
version = "1"
tf.saved_model.save(
m_,
"./exported_model/" + version,
signatures=serving
)
答案 0 :(得分:0)
因此正确的方法就在这里,可以通过以下选项通过serving_default
方法进行要素工程和预处理。我通过Tensorflow服务对其进行了进一步的测试。
@tf.function
def serving(WERKS, DIFGRIRD, SCENARIO, TOTIRQTY, VSTATU, EKGRP, TOTGRQTY, VPATD, EKORG, NODLGR, DIFGRIRV, NODLIR, KTOKK):
##Feature engineering
nodlgrbyvpatd = tf.cast(NODLGR / VPATD, tf.float32)
payload = {
'WERKS': WERKS,
'DIFGRIRD': DIFGRIRD,
'SCENARIO': SCENARIO,
'TOTIRQTY': TOTIRQTY,
'VSTATU': VSTATU,
'EKGRP': EKGRP,
'TOTGRQTY': TOTGRQTY,
'VPATD': VPATD,
'EKORG': EKORG,
'NODLGR': NODLGR,
'DIFGRIRV': DIFGRIRV,
'NODLIR': NODLIR,
'KTOKK': KTOKK,
'nodlgrbyvpatd': nodlgrbyvpatd,
}
## Predict
##IF THERE IS AN ERROR IN NUMBER OF PARAMS PASSED HERE OR DATA TYPE THEN IT GIVES ERROR, "COULDN'T COMPUTE OUTPUT TENSOR"
predictions = m_(payload)
return predictions
serving = serving.get_concrete_function(WERKS=tf.TensorSpec([None,], dtype= tf.string, name='WERKS'),
DIFGRIRD=tf.TensorSpec([None,], name='DIFGRIRD'),
SCENARIO=tf.TensorSpec([None,], dtype= tf.string, name='SCENARIO'),
TOTIRQTY=tf.TensorSpec([None,], name='TOTIRQTY'),
VSTATU=tf.TensorSpec([None,], dtype= tf.string, name='VSTATU'),
EKGRP=tf.TensorSpec([None,], dtype= tf.string, name='EKGRP'),
TOTGRQTY=tf.TensorSpec([None,], name='TOTGRQTY'),
VPATD=tf.TensorSpec([None,], name='VPATD'),
EKORG=tf.TensorSpec([None,], dtype= tf.string, name='EKORG'),
NODLGR=tf.TensorSpec([None,], name='NODLGR'),
DIFGRIRV=tf.TensorSpec([None,], name='DIFGRIRV'),
NODLIR=tf.TensorSpec([None,], name='NODLIR'),
KTOKK=tf.TensorSpec([None,], dtype= tf.string, name='KTOKK')
)
version = "1"
tf.saved_model.save(
m_,
"./exported_model/" + version,
signatures=serving
)