Question

我尝试使用tensorflow联合学习工具来处理我的数据。我有两个从csv文件获得的数据集（数据集和数据集2），其中前15列是要素，最后一列是标签。我将熊猫数据框转换为张量流数据集。但是，在迭代器中，存在一个奇怪的类型错误。我是tensrflow的新手，并发送了代码：任何帮助将不胜感激。预先感谢。

from __future__ import absolute_import, division, print_function
from sklearn.preprocessing import MinMaxScaler
from keras.models import Model

import collections
import numpy as np
import tensorflow as tf
import tensorflow_federated as tff
from numpy import loadtxt
from keras.models import Sequential
from keras.layers import Dense
from numpy import loadtxt
from keras.models import Sequential
from keras.layers import Dense
import pandas as pd


X_train= pd.read_csv('./daily_frames_HR.csv')



values = X_train.values

values = values.astype('float32')
# normalize features
scaler = MinMaxScaler(feature_range=(0, 1))
scaled = scaler.fit_transform(values)
# frame as supervised learning

train = values[:, :]
# split into input and outputs
X, y = train[:, :-2], train[:, -1]


def create_compiled_keras_model():
  model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(
      12, activation=tf.nn.softmax, kernel_initializer='zeros', input_dim=15)])

  return model

def model_fn():
  keras_model = create_compiled_keras_model()

  keras_model.compile(loss='binary_crossentropy', optimizer='sgd', metrics= 
   ['SparseCategoricalAccuracy'])
  X_train = pd.read_csv('./daily_frames_HR.csv')

  values = X_train.values

  values = values.astype('float32')
  # normalize features
  scaler = MinMaxScaler(feature_range=(0, 1))
  scaled = scaler.fit_transform(values)
  # frame as supervised learning

  train = values[:, :]
  # split into input and outputs
  X, y = train[:, :-2], train[:, -1]

  sample_batch = collections.OrderedDict([('x', X), ('y', y)])
  return tff.learning.from_compiled_keras_model(keras_model, sample_batch)


  iterative_process = tff.learning.build_federated_averaging_process(model_fn)

  state = iterative_process.initialize()

  X2_train= pd.read_csv('./lab_frames_HR.csv')
  values2 = X2_train.values

  values2 = values2.astype('float32')
  # normalize features
  scaler = MinMaxScaler(feature_range=(0, 1))
  scaled = scaler.fit_transform(values2)
  # frame as supervised learning

  train2 = values2[:, :]
  # split into input and outputs
  X2, y2 = train2[:, :-2], train2[:, -1]

  X2=pd.DataFrame(X2)
  y2=pd.DataFrame(y2)

  X=pd.DataFrame(X)
  y=pd.DataFrame(y)


dataset = tf.data.Dataset.from_tensor_slices((X2.values, y2.values))

dataset2= tf.data.Dataset.from_tensor_slices((X.values, y.values))


list = [dataset, dataset2]

state, metrics = iterative_process.next(state, list)
print('round  1, metrics={}'.format(metrics))

错误消息如下：

回溯（最近通话最近）：在第117行的文件“ /home/affectech/Desktop/Fed_son/Fed_son.py” 状态，指标= iterative_process.next（状态，列表）在调用中的第666行中的文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/utils/function_utils.py” arg = pack_args（self._type_signature.parameter，args，kwargs，context）在pack_args中的第424行，文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/utils/function_utils.py” 上下文）文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/utils/function_utils.py”，行346，在pack_args_into_anonymous_tuple中 result_elements.append（（name，context.ingest（arg_value，elem_type））））摄入的文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/reference_executor.py”，第629行返回to_representation_for_type（arg，type_spec，_handle_callable）文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/reference_executor.py”，行241，在to_representation_for_type中价值v 在第241行的文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/reference_executor.py”中价值v 文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/reference_executor.py”，第200行，位于to_representation_for_type 价值v 在第200行的文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/reference_executor.py”中价值v 文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/reference_executor.py”，第192行，位于to_representation_for_type callable_handler）文件“ /home/affectech/Desktop/Fed_son/venv/lib/python3.6/site-packages/tensorflow_federated/python/core/impl/reference_executor.py”，行165，在to_representation_for_type中 '类型说明{}。'。format（inferred_type_spec，type_spec）） TypeError：值表示形式的张量类型float32 [15]与规范spec float32 [？，15]的类型不匹配。

以退出代码1完成的过程

Answer 1

好像对iterative_process.next(state, list)的调用期望数据集列表（list）是 batched 数据集的列表。如果您不希望每个批次有多个示例，则批次大小甚至可以为1。

batch_size = 1
list = [dataset.batch(batch_size), dataset2.batch(batch_size)]

使用print(iterative_process.next.type_signature)和tf.data.experimental.get_structure (dataset)可以打印不同对象的类型。

Answer 2

我看到的一个问题是

X, y = train[:, :-2], train[:, -1]

您在这里丢失了最后一个功能列，应该是

X, y = train[:, :-1], train[:, -1]

此脚本令人困惑的是在函数体内重新定义了train，X和y。该程序永远都不会出现您发布的错误，因为X2尚未在使用时定义。我建议通常不要重用变量名，这将使程序更易于调试。

然后，当您对某物的形状有所抱怨时，在错误发生之前打印它（或一部分），这将有助于诊断出问题所在。

联合学习时的张量类型错误

2 个答案: