Question

我正在使用基于this tuorial的Tensorflow进行深度神经网络回归。当我尝试使用tf.estimator export_savemodel保存模型时，我收到以下错误：

 raise ValueError('Feature {} is not in features dictionary.'.format(key))
 ValueError: Feature ad_provider is not in features dictionary.

我需要将其导出才能部署模型以支持Google Cloud Platform中的预测。

这是我定义列的位置：

CSV_COLUMNS = [
"ad_provider", "device", "split_group","gold", "secret_areas",
 "scored_enemies", "tutorial_sec", "video_success"
]

FEATURES = ["ad_provider", "device", "split_group","gold", "secret_areas",
 "scored_enemies", "tutorial_sec"]

LABEL = "video_success"

ad_provider = tf.feature_column.categorical_column_with_vocabulary_list(
    "ad_provider", ["Organic","Apple Search Ads","googleadwords_int",
"Facebook Ads","website"]  )

split_group = tf.feature_column.categorical_column_with_vocabulary_list(
    "split_group", [1,2,3,4])

device = tf.feature_column.categorical_column_with_hash_bucket(
    "device", hash_bucket_size=100)


secret_areas = tf.feature_column.numeric_column("secret_areas")
gold = tf.feature_column.numeric_column("gold")
scored_enemies = tf.feature_column.numeric_column("scored_enemies")
finish_tutorial_sec = tf.feature_column.numeric_column("tutorial_sec")
video_success = tf.feature_column.numeric_column("video_success")


feature_columns = [
tf.feature_column.indicator_column(ad_provider),
tf.feature_column.embedding_column(device, dimension=8),
tf.feature_column.indicator_column(split_group),
tf.feature_column.numeric_column(key="gold"),
tf.feature_column.numeric_column(key="secret_areas"),
tf.feature_column.numeric_column(key="scored_enemies"),
tf.feature_column.numeric_column(key="tutorial_sec"),
]

之后，我创建了一个函数以在JSON词典中导出我的模型。我不确定如果我的服务功能很好。

def json_serving_input_fn():
  """Build the serving inputs."""
  inputs = {}
  for feat in feature_columns:
    inputs[feat.name] = tf.placeholder(shape=[None], dtype= feat.dtype if     
hasattr(feat, 'dtype') else tf.string)

features = {
  key: tf.expand_dims(tensor, -1)
  for key, tensor in inputs.items()
}
  return tf.contrib.learn.InputFnOps(features, None, inputs)

以下是我的其余代码：

def main(unused_argv):

  #Normalize columns 'Gold' and 'tutorial_sec' for Traininig Set
  train_n = training_set
  train_n['gold'] = (train_n['gold'] - train_n['gold'].mean()) / (train_n['gold'].max() - train_n['gold'].min())
  train_n['tutorial_sec'] = (train_n['tutorial_sec'] - train_n['tutorial_sec'].mean()) / (train_n['tutorial_sec'].max() - train_n['tutorial_sec'].min())
  train_n['scored_enemies'] = (train_n['scored_enemies'] - train_n['scored_enemies'].mean()) / (train_n['scored_enemies'].max() - train_n['scored_enemies'].min())

  test_n = test_set
  test_n['gold'] = (test_n['gold'] - test_n['gold'].mean()) / (test_n['gold'].max() - test_n['gold'].min())
  test_n['tutorial_sec'] = (test_n['tutorial_sec'] - test_n['tutorial_sec'].mean()) / (test_n['tutorial_sec'].max() - test_n['tutorial_sec'].min())
  test_n['scored_enemies'] = (test_n['scored_enemies'] - test_n['scored_enemies'].mean()) / (test_n['scored_enemies'].max() - test_n['scored_enemies'].min())

  train_input_fn = tf.estimator.inputs.pandas_input_fn(
    x=train_n,
    y=pd.Series(train_n[LABEL].values),
    batch_size=100,
    num_epochs=None,
    shuffle=True)

  test_input_fn = tf.estimator.inputs.pandas_input_fn(
    x=test_n,
    y=pd.Series(test_n[LABEL].values),
    batch_size=100,
    num_epochs=1,      
   shuffle=False)


  regressor = tf.estimator.DNNRegressor(feature_columns=feature_columns,
                                      hidden_units=[40, 30, 20],
                                      model_dir="model1",
                                      optimizer='RMSProp'
                                      )


  # Train

  regressor.train(input_fn=train_input_fn, steps=5)

  regressor.export_savedmodel("test",json_serving_input_fn)

  #Evaluate loss over one epoch of test_set.
  #For each step, calls `input_fn`, which returns one batch of data.
  ev = regressor.evaluate(
    input_fn=test_input_fn)
  loss_score = ev["loss"]
  print("Loss: {0:f}".format(loss_score))
  for key in sorted(ev):
      print("%s: %s" % (key, ev[key]))


  # Print out predictions over a slice of prediction_set.
  y = regressor.predict(
    input_fn=test_input_fn)
  # Array with prediction list!
  predictions = list(p["predictions"] for p in y)

  #real = list(p["real"] for p in pd.Series(training_set[LABEL].values))
  real = test_set[LABEL].values
  diff = np.subtract(real,predictions)

  diff = np.absolute(diff)
  diff = np.mean(diff)
  print("Mean Square Error of Test Set = ",diff*diff)

Answer 1

除了您提到的问题之外，我预计您还会遇到其他多个问题：

您正在使用TensorFlow 1.3中引入的tf.estimator.DnnRegressor。 CloudML Engine仅正式支持TF 1.2。
您正在规范熊猫数据框中的功能，这不会在服务时发生（除非您在客户端执行此操作）。这会引入偏差，你会得到糟糕的预测结果。

因此，让我们首先使用tf.contrib.learn.DNNRegressor，这只需要进行细微更改：

regressor = tf.estimator.DNNRegressor(
    feature_columns=feature_columns,
    hidden_units=[40, 30, 20],
    model_dir="model1",
    optimizer='RMSProp'
)
regressor.fit(input_fn=train_input_fn, steps=5)
regressor.export_savedmodel("test",json_serving_input_fn)

请注意fit而不是train。

（ NB：您的json_serving_inputfn实际上已经为TF 1.2编写并且与TF 1.3不兼容。现在这很好。

现在，您看到的错误的根本原因是列/功能ad_provider不在输入和功能列表中（但您确实有ad_provider_indicator）。这是因为您正在遍历feature_columns而不是通过原始输入列列表。解决这个问题的方法是迭代实际输入而不是特征列;但是，我们也需要知道类型（仅用几列简化）：

CSV_COLUMNS = ["ad_provider", "gold", "video_success"] 
FEATURES = ["ad_provider", "gold"] 
TYPES = [tf.string, tf.float32] 
LABEL = "video_success" 

def json_serving_input_fn(): 
  """Build the serving inputs.""" 
  inputs = {} 
  for feat, dtype in zip(FEATURES, TYPES): 
    inputs[feat] = tf.placeholder(shape=[None], dtype=dtype) 

  features = {
    key: tf.expand_dims(tensor, -1)
    for key, tensor in inputs.items()
  }
  return tf.contrib.learn.InputFnOps(features, None, inputs)

最后，为了规范化您的数据，您可能希望在图表中执行此操作。您可以尝试使用tf.transform，或者编写一个执行转换的自定义估算器，委派实际的模型实现DNNRegressor。

使用tf Estimator和export_savedmodel函数

1 个答案: