从gRPC服务向RESTful客户端返回响应流时出现问题

时间：2019-09-27 14:20:37

标签： java rest spring-boot grpc-java

我正在尝试编写一组服务，该服务将为外部客户端提供REST接口，同时使用gRPC / protobuf在内部进行服务间通信。 Spring Boot被用于帮助后台基础架构支持。大约一周前，我才开始学习和使用Google protobuf和gRPC。

我已经能够成功地使更简单的处理从头到尾工作，但是这些返回的是单个对象（即不流式传输）。我在通话中遇到了问题，该通话旨在流式传输响应对象的集合。要注意的另一件事是，到目前为止，我尝试的所有操作都是GET请求。

下面的示例仅显示了我遇到问题的相关部分。

详细信息：

Protobuf定义：

syntax = "proto3";

package experiment;

message BOM {
    PartSpec part_spec = 1;
    string description = 2;
    string category = 3;
}

message PartSpec {
    string part_number = 1;
    string cage_code = 2;
}

service BOMService {
    rpc getBOMList(google.protobuf.Empty) returns (stream BOM) {}
}

“域”服务：

实施.proto文件定义的服务。响应“网关”服务的请求。

@GrpcService
public class BOMDomainService extends BOMServiceImplBase {

  private final Logger logger = LogManager.getLogger();

  private Random       rand   = new Random(1);

  @Override
  public void getBOMList(Empty request, StreamObserver<BOM> responseObserver) {
    for (int i = 1; i <= rand.nextInt(40); i++) {
      PartSpec part = PartSpec.newBuilder().setPartNumber("part-1").setCageCode("cage-1").build();
      BOM bom = BOM.newBuilder().setPartSpec(part).setDescription("some part").build();
      // Stream BOM response.
      responseObserver.onNext(bom);
    }

    // Complete server response.
    responseObserver.onCompleted();
  }
}

“网关”

网关服务：

通过gRPC调用（阻止存根）与“域”服务进行交互。

@Service
public class BOMGatewayService {

  private final Logger           logger = LogManager.getLogger();

  @GrpcClient("bom-service")
  private BOMServiceBlockingStub bomSvcStub;

  public List<BOM> getBOMList() {
    List<BOM> result = new ArrayList<>();

    try {
      Iterator<BOM> iter = bomSvcStub.getBOMList(Empty.getDefaultInstance());

      while (iter.hasNext()) {
        result.add(iter.next());
      }
    } catch (StatusRuntimeException e) {
      logger.warn("RPC failed: {}", e.getStatus());
    }

    return result;
  }
}

网关控制器：

@RestController
@RequestMapping("/alisn/bom")
public class BOMController {

  @Autowired
  private BOMGatewayService service;

  @GetMapping(value = "/getBOMList", produces = { MediaType.APPLICATION_JSON_VALUE })
  public DeferredResult<ResponseEntity<List<BOM>>> getBOMList() {
    DeferredResult<ResponseEntity<List<BOM>>> result = new DeferredResult<>();
    List<BOM> response = service.getBOMList();
    result.setResult(ResponseEntity.ok(response));
    return result;
  }
}

外部“客户”服务：

通过REST调用与“网关”服务交互。不使用gRPC，但是使用从.proto文件生成的消息类。

注意：我不是在这里获取“ BOM”的列表。如果我尝试使用ResponseEntity<List<BOM>>，则getForEntity()方法会引发编译器错误。我不确定我应该在这里做什么，也许这就是问题的一部分。

@Service
public class ClientService {

  private final Logger logger = LogManager.getLogger();

  @Autowired
  private RestTemplate restTemplate;

  @Scheduled(fixedDelay = 5000)
  public void getBOMList() {
    ResponseEntity<BOM> response = restTemplate
        .getForEntity("http://localhost:7090/alisn/bom/getBOMList", BOM.class);
    logger.info("Received:\n{}", response.getBody());
  }
}

我已尝试通过两种方式进行锻炼：

  1. 从使用RestTemplate的客户端Java Spring Boot服务中 进行REST调用（上面显示的“客户端”）。
  2. 进行HTTP 来自邮递员的GET请求。

两种方法的响应似乎大致相同。

Java客户端方法：

客户端控制台：

[2019-09-27 10:01:16.373] scheduling-1 ERROR: support.TaskUtils$LoggingErrorHandler:96 - Unexpected error occurred in scheduled task.
org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 null
    at org.springframework.web.client.HttpServerErrorException.create(HttpServerErrorException.java:79) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:124) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:102) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:778) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:736) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:670) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.getForEntity(RestTemplate.java:338) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at edu.mit.ll.alisn.exp.client.service.ClientService.getBOMList(ClientService.java:26) ~[classes/:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_201]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_201]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_201]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]

“网关”控制台：

[2019-09-27 10:01:16.356] http-nio-7090-exec-3 ERROR: [/].[dispatcherServlet]:175 - Servlet.service() for servlet [dispatcherServlet] threw exception
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["defaultInstanceForType"])
    at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContentsUsing(CollectionSerializer.java:171) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:116) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1392) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[jackson-databind-2.9.9.jar:2.9.9]
    at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:287) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:103) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters(AbstractMessageConverterMethodProcessor.java:290) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.HttpEntityMethodProcessor.handleReturnValue(HttpEntityMethodProcessor.java:225) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:82) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:122) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114) ~[spring-boot-actuator-2.1.6.RELEASE.jar:2.1.6.RELEASE]
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104) ~[spring-boot-actuator-2.1.6.RELEASE.jar:2.1.6.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.doDispatch(ApplicationDispatcher.java:633) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:601) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.AsyncContextImpl$AsyncRunnable.run(AsyncContextImpl.java:566) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.AsyncContextImpl.doInternalDispatch(AsyncContextImpl.java:355) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.connector.CoyoteAdapter.asyncDispatch(CoyoteAdapter.java:235) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProcessor.dispatch(AbstractProcessor.java:241) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:53) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:853) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]

邮递员：

{
    "timestamp": "2019-09-27T14:03:38.530+0000",
    "status": 500,
    "error": "Internal Server Error",
    "message": "Type definition error: [simple type, class com.google.protobuf.UnknownFieldSet]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM[\"unknownFields\"]->com.google.protobuf.UnknownFieldSet[\"defaultInstanceForType\"])",
    "path": "/alisn/bom/getBOMList"
}

对于我在做什么错的任何建议，或者对进行此类处理的更好方法的建议，将不胜感激。

更新：

我还包括用于客户端和网关服务的Bean配置。

客户：

@Configuration
@EnableScheduling
@ComponentScan(XXXXXX)
public class Configurer {
  @Bean
  ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufJsonFormatHttpMessageConverter();
  }

  @Bean
  RestTemplate restTemplate(ProtobufHttpMessageConverter hmc) {
    return new RestTemplate(Arrays.asList(hmc));
  }
}

网关：

@Configuration
@ComponentScan(XXXXXX)
public class Configurer {
  @Bean
  ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufJsonFormatHttpMessageConverter();
  }
}

更新：

我做了一个更简单的实验，创建了一个新服务，该服务仅返回一个BOM对象。这样就可以了。

我不明白为什么这样做，而List<BOM>情况却不可行。抛出异常中的信息似乎指向生成的BOM类中的内部结构。

网关控制台：

[2019-10-02 08:37:25.960] http-nio-7090-exec-3 ERROR: [/].[dispatcherServlet]:175 - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class com.google.protobuf.UnknownFieldSet$Parser]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])] with root cause
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])

我对java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"]的引用似乎表明BOM类的内部结构有问题，但是我可以毫无问题地返回一个BOM。

2 个答案:

答案 0 :(得分：0)

根据网关日志，看来jackson在将原型转换为json时遇到问题。

您可以尝试com.googlecode.protobuf.format.JsonFormat吗？我不是spring用户，但是应该有用于自定义序列化的API。或者，您可以使用JsonFormat向杰克逊添加自定义序列化，但这听起来有点奇怪。

答案 1 :(得分：0)

在可能的情况下，我有： .... response.setMessageCode（0）.setResponseMessage（“字符串消息”）; .... 永远不会回到客户身上。

我将其更改为： ... APIResponse响应= APIResponse.newBuilder（）。setResponseMessage（localidad.toString（））。build（）; ...

没有“ .setMessageCode（0）” ...，就可以了！

相关问题
最新问题