Question

我在我的spring-boot应用程序中使用WebClient和自定义BodyExtractor类

WebClient webLCient = WebClient.create();
webClient.get()
   .uri(url, params)
   .accept(MediaType.APPLICATION.XML)
   .exchange()
   .flatMap(response -> {
     return response.body(new BodyExtractor());
   })

BodyExtractor.java

@Override
public Mono<T> extract(ClientHttpResponse response, BodyExtractor.Context context) {
  Flux<DataBuffer> body = response.getBody();
  body.map(dataBuffer -> {
    try {
      JaxBContext jc = JaxBContext.newInstance(SomeClass.class);
      Unmarshaller unmarshaller = jc.createUnmarshaller();

      return (T) unmarshaller.unmarshal(dataBuffer.asInputStream())
    } catch(Exception e){
       return null;
    }
  }).next();
}

上面的代码使用小的有效负载而不是大的有效负载，我认为这是因为我只用next读取单个通量值，我不确定如何组合和读取所有{{1 }}

我是反应堆的新手，所以我不知道很多使用flux / mono的技巧。

Answer 1

这确实没有其他答案所暗示的那么复杂。

如@ jin-kwon所建议的那样，流传输数据而不将其全部缓存在内存中的唯一方法是使用管道。但是，使用Spring的BodyExtractors和DataBufferUtils实用程序类可以非常简单地完成此操作。

示例：

private InputStream readAsInputStream(String url) throws IOException {
    PipedOutputStream osPipe = new PipedOutputStream();
    PipedInputSteam isPipe = new PipedInputStream(osPipe);

    ClientResponse response = webClient.get().uri(url)
        .accept(MediaType.APPLICATION.XML)
        .exchange()
        .block();
    final int statusCode = response.rawStatusCode();
    // check HTTP status code, can throw exception if needed
    // ....

    Flux<DataBuffer> body = response.body(BodyExtractors.toDataBuffers())
        .doOnError(t -> {
            log.error("Error reading body.", t);
            // close pipe to force InputStream to error,
            // otherwise the returned InputStream will hang forever if an error occurs
            try(isPipe) {
              //no-op
            } catch (IOException ioe) {
                log.error("Error closing streams", ioe);
            }
        })
        .doFinally(s -> {
            try(osPipe) {
              //no-op
            } catch (IOException ioe) {
                log.error("Error closing streams", ioe);
            }
        });

    DataBufferUtils.write(body, osPipe)
        .subscribe(DataBufferUtils.releaseConsumer());

    return isPipe;
}

如果您不关心检查响应代码或为故障状态代码引发异常，则可以使用

跳过block()调用和中间ClientResponse变量。

flatMap(r -> r.body(BodyExtractors.toDataBuffers()))

相反。

Answer 2

首先重建InputStream会破坏使用WebClient的目的，因为在collect操作完成之前不会发出任何内容。对于大流，这可能是一个很长的时间。反应模型不处理单个字节，而是处理字节块（如Spring DataBuffer）。请在此处查看我的答案，以获得更优雅的解决方案：https://stackoverflow.com/a/48054615/839733

Answer 3

这是其他答案的另一个变体。而且它仍然对内存不友好。

static Mono<InputStream> asStream(WebClient.ResponseSpec response) {
    return response.bodyToFlux(DataBuffer.class)
        .map(b -> b.asInputStream(true))
        .reduce(SequenceInputStream::new);
}

static void doSome(WebClient.ResponseSpec response) {
    asStream(response)
        .doOnNext(stream -> {
            // do some with stream
        })
        .block();
}

Answer 4

有一种更干净的方法可以直接使用底层的反应堆净值HttpClient来代替使用WebClient。组成层次结构如下：

WebClient -uses-> HttpClient -uses-> TcpClient

比解释容易显示代码：

HttpClient.create()
    .get()
    .responseContent() // ByteBufFlux
    .aggregate() // ByteBufMono
    .asInputStream() // Mono<InputStream>
    .block() // We got an InputStream, yay!

但是，正如我已经指出的那样，使用InputStream是一种阻塞操作，它违反了使用非阻塞HTTP客户端的目的，更不用说汇总整个响应了。有关Java NIO与IO的比较，请参见this。

Answer 5

我可以使用Flux#collect和SequenceInputStream

使其有效

@Override
public Mono<T> extract(ClientHttpResponse response, BodyExtractor.Context context) {
  Flux<DataBuffer> body = response.getBody();
  return body.collect(InputStreamCollector::new, (t, dataBuffer)-> t.collectInputStream(dataBuffer.asInputStream))
    .map(inputStream -> {
      try {
        JaxBContext jc = JaxBContext.newInstance(SomeClass.class);
        Unmarshaller unmarshaller = jc.createUnmarshaller();

        return (T) unmarshaller.unmarshal(inputStream);
      } catch(Exception e){
        return null;
      }
  }).next();
}

InputStreamCollector.java

public class InputStreamCollector {
  private InputStream is;

  public void collectInputStream(InputStream is) {
    if (this.is == null) this.is = is;
    this.is = new SequenceInputStream(this.is, is);
  }

  public InputStream getInputStream() {
    return this.is;
  }
}

Answer 6

Bk Santiago的答案的稍微修改后的版本使用了reduce()而不是collect()。非常相似，但是不需要额外的类：

Java：

body.reduce(new InputStream() {
    public int read() { return -1; }
  }, (s: InputStream, d: DataBuffer) -> new SequenceInputStream(s, d.asInputStream())
).flatMap(inputStream -> /* do something with single InputStream */

或者科特林：

body.reduce(object : InputStream() {
  override fun read() = -1
}) { s: InputStream, d -> SequenceInputStream(s, d.asInputStream()) }
  .flatMap { inputStream -> /* do something with single InputStream */ }

此方法相对于使用collect()的好处是，您无需使用其他类来收集信息。

我创建了一个新的空InputStream()，但是如果该语法令人困惑，您也可以将其替换为ByteArrayInputStream("".toByteArray())，而不是创建一个空的ByteArrayInputStream作为初始值。

Answer 7

您可以使用管道。

public static <R> R pipeBodyToStreamAndApply(
        final int pipeSize,
        final Flux<DataBuffer> dataBuffers,
        final Executor taskExecutor,
        final Function<? super InputStream, ? extends R> streamFunction)
        throws IOException {
    final PipedOutputStream output = new PipedOutputStream();
    final PipedInputStream input = new PipedInputStream(output, pipeSize);
    final Flux<DataBuffer> mapped = dataBuffers.map(b -> {
        final byte[] d = new byte[b.readableByteCount()];
        b.read(d);
        try {
            output.write(d);
        } catch (final IOException ioe) {
            throw new RuntimeException("failed to write", ioe);
        }
        return b;
    });
    taskExecutor.execute(() -> {
        flux.map(DataBufferUtils::release).blockLast();
        try {
            output.flush();
            output.close();
        } catch (final IOException ioe) {
            throw new RuntimeException(
                "failed to flush and close the piped output stream", ioe);
        }
    });
    return streamFunction.apply(input);
}

这里是频道。

public static <R> R pipeBodyToChannelAndApply(
        final Flux<DataBuffer> dataBuffers,
        final Executor taskExecutor,
        final Function<? super ReadableByteChannel, ? extends R> channelFunction)
        throws IOException {
    final Pipe pipe = Pipe.open();
    final Flux<DataBuffer> flux = dataBuffers.map(b -> {
        for (final ByteBuffer s = b.asByteBuffer(); s.hasRemaining(); ) {
            try {
                final int written = pipe.sink().write(s);
            } catch (final IOException ioe) {
                throw new RuntimeException("failed to write", ioe);
            }
        }
        return b;
    });
    taskExecutor.execute(() -> {
        flux.map(DataBufferUtils::release).blockLast();
        try {
            pipe.sink().close();
        } catch (final IOException ioe) {
            throw new RuntimeException("failed to close the pipe.sink", ioe);
        }
    });
    return channelFunction.apply(pipe.source());
}

如何正确读取Flux <databuffer>并将其转换为单个inputStream

7 个答案: