Question

我正在尝试以小块的形式将文本写入流中，当我在输出流中指向XML流编写器时（它立即开始发送数据）然后我尝试编写一些文本然后刷新它在流关闭之前不发送任何内容。

(defn data
  "Download a 5MB file and parse it"
  []
  (-> "http://www.cs.washington.edu/research/xmldatasets/data/tpc-h/orders.xml"
      URL.
      .openStream
      xml/parse))

(defn send-stuff [request]
  (condp = (:uri request)
    "/text" (response/response
             (ring-io/piped-input-stream
              #(let [w (io/make-writer % {:encoding "UTF-8"})]
                 (.write w "start\n")
                 (.flush w)
                 (Thread/sleep 1000)
                 (.write w "done\n")
                 (.flush w))))
    "/xml"  (response/response
             (ring-io/piped-input-stream
              #(->> (io/make-writer % {:encoding "UTF-8"})
                    (xml/emit (data))
                    .flush)))))

(comment
  (def server (jetty/run-jetty #'send-stuff {:port 8888 :join? false}))
  (.stop server))

使用curl测试它，如下所示：

curl localhost:8888/text

静静地坐在那里一秒钟，然后返回

start
done

我希望看到“开始”然后一秒钟后“完成”，而不是一秒钟延迟，然后是两者。

并使用

curl localhost:8888/xml

立即启动流式处理眼睛的XML（抱歉个人偏见在那里偷偷摸摸; - ）

- 编辑我已经确认问题在于jetty输出缓冲区，因为如果我将缓冲区设置得很小就会消失：

(def server (jetty/run-jetty #'send-stuff {:output-buffer-size 1 :port 8888 :join? false}))

当然，在许多情况下，将输出缓冲区设置为1是一个坏主意。

Answer 1

您正在呼叫的.flush不在用于HTTP响应的流上，而是在output stream of the piped streams pair上。

当您查看source code of PipedOutputStream.flush()时，您会注意到它只通知所有等待从连接的PipedInputStream读取的线程，并不意味着刷新到底层的HTTP响应流。

行为的差异是由响应数据大小引起的。如果您将示例更改为使用小型XML数据，则行为将是相同的：

(defn data
  []
  (-> "<?xml version=\"1.0\" encoding=\"UTF-8\"?><a>1</a>"
      (.getBytes)
      (ByteArrayInputStream.)
      (xml/parse)))


(defn send-stuff [request]
  (condp = (:uri request)
    "/text" (response/response
              (ring-io/piped-input-stream
                #(let [w (io/make-writer % {:encoding "UTF-8"})]
                   (.write w "start\n")
                   (.flush w)
                   (Thread/sleep 1000)
                   (.write w "done\n")
                   (.flush w))))
    "/xml"  (response/response
              (ring-io/piped-input-stream
                #(let [w (io/make-writer % {:encoding "UTF-8"})]
                   (xml/emit (data) w)
                   (.flush w)
                   (Thread/sleep 1000)
                   (xml/emit (data) w)
                   (.flush w))))))

调用curl localhost:8888/xml将在一秒钟后仅显示整个响应：

<?xml version="1.0" encoding="UTF-8"?><a>1</a><?xml version="1.0" encoding="UTF-8"?><a>1</a>

您可以使用不同的流机制来控制刷新HTTP响应流，例如使用阻塞队列：

(ns so43769408
  (:require [ring.adapter.jetty :as jetty]
            [clojure.java.io :as io]
            [ring.util.response :as response]
            [ring.core.protocols :as protocols])
  (:import (java.io OutputStream)
           (java.util.concurrent LinkedBlockingQueue)))

(extend-protocol protocols/StreamableResponseBody
  LinkedBlockingQueue
  (write-body-to-stream [output-queue _ ^OutputStream output-stream]
    (with-open [out (io/writer output-stream)]
      (loop [chunk (.take output-queue)]
        (when-not (= chunk ::EOF)
          (.write out (str chunk))
          (.flush out)
          (recur (.take output-queue)))))))


(defn send-stuff [request]
  (response/response
    (let [output-queue (LinkedBlockingQueue.)]
      (future
        (.put output-queue "start\n")
        (Thread/sleep 1000)
        (.put output-queue "end\n")
        (.put output-queue ::EOF))
      output-queue)))

(comment
  (def server (jetty/run-jetty #'send-stuff {:port 8888 :join? false}))
  (.stop server))

为什么xml / emit会立即流，而直接写入流不会

1 个答案: