Protobuf in python during message deserialization complains about 'Unexpected end-group tag.'

Question

I'm developing zmq/protobuf application and I have a problem with deserialization of messages sent from C++ to python. I easily handle messages from python to C++ however in the other direction I have a problem.

Protobuf library in python client application complains that it detected:
File "C:\Python27\lib\site-packages\google\protobuf\internal\python_message.py", line 844, in MergeFromString raise message_mod.DecodeError('Unexpected end-group tag.')

I presume there is a problem between C++ serizalization and python deserialization. I'm wondering if there is some problem with null terminator in C/C++.

I'm using RaspberryPi running Raspian for C++ code and x64 CPU running Windows 7 for python code.

This is my C++ serialization code..

// Test Code.
// Try to send some 'demo' response back.
RPiProtocol::Message response;
std::string response_string;
response.set_type(RPiProtocol::Message::RESPONSE);
response.set_command(RPiProtocol::Message::GET_SYS_INFO);
response.set_version(0);

// Serialize ZMQ message to string.
if (response.SerializeToString(&response_string))
{
    //  Send response message back to the client.
    zmq::message_t reply(response_string.length());
    memcpy((void *)reply.data(), (void *)&response_string, response_string.length());
    socket.send(reply);
}

This is my python deserialization code..

#  Get the reply.
message = socket.recv()
response = rpi_protocol_pb2.Message()

# This line fails.
response.ParseFromString(str(message))

I debugged that deserialization fails in this function \google\protobuf\internal\python_message.py

  def InternalParse(self, buffer, pos, end):
    self._Modified()
    field_dict = self._fields
    unknown_field_list = self._unknown_fields
    while pos != end:
      (tag_bytes, new_pos) = local_ReadTag(buffer, pos)
      field_decoder, field_desc = decoders_by_tag.get(tag_bytes, (None, None))
      if field_decoder is None:
        value_start_pos = new_pos
        new_pos = local_SkipField(buffer, new_pos, end, tag_bytes)
        if new_pos == -1: # HERE I HAVE -1 !!!
          return pos
        if not unknown_field_list:
          unknown_field_list = self._unknown_fields = []
        unknown_field_list.append((tag_bytes, buffer[value_start_pos:new_pos]))
        pos = new_pos
      else:
        pos = field_decoder(buffer, new_pos, end, self, field_dict)
        if field_desc:
          self._UpdateOneofState(field_desc)
    return pos
  cls._InternalParse = InternalParse

Could you help me with enabling my application?

Answer 1

我在用Python反序列化probbuff数据时也遇到了问题。原型数据已使用“ C”原型代码进行了序列化。

简短的答案是使用binascii.unhexlify（）。

在python上序列化一个原型，然后将其发送到反序列化的“ C”代码中就可以了。但是相反，直到我执行以下操作后，情况才成立：binstring = binascii.unhexlify（hexstring）。然后protoBuff.ParseFromString（binstring）正常工作。

Answer 2

最后我找到了错误的代码。我在C ++服务器的这行中出错：

memcpy((void *)reply.data(), (void *)&response_string, response_string.length());

而不是上面的错误代码应该是：

memcpy((void *)reply.data(), (void *)response_string.data(), response_string.length());

我理解如何将C ++字符串转换为ZMQ字符串，因为我在网上找到了这个函数：

//  Convert string to 0MQ string and send to socket    
static bool s_send (zmq::socket_t & socket, const std::string & string) {

    zmq::message_t message(string.size());
    memcpy (message.data(), string.data(), string.size());

    bool rc = socket.send (message);
    return (rc);
}

以下是 zhelpers.hpp 头文件的链接，其中包含上面粘贴的功能以及基于C ++ ZMQ的应用程序的许多其他有用功能： https://github.com/imatix/zguide/blob/master/examples/C%2B%2B/zhelpers.hpp