apache thrift监视一个hdfs文件

时间:2017-10-07 22:12:02

标签: thrift spark-thriftserver

我计划使用Apache Thrift监控本地更改的任何更改,并将该数据推送到客户端(更改的数据)。

当我检查节俭文档时,我看到多个传输层,但不明白我应该使用哪个传输层

http://thrift-tutorial.readthedocs.io/en/latest/thrift-stack.html

Tranport Layer
The transport layer is responsible for reading from and writing to the wire. Thrift supports the following:

TSocket - Uses blocking socket I/O for transport.
TFramedTransport - Sends data in frames, where each frame is preceded by a length. This transport is required when using a non-blocking server.
TFileTransport - This transport writes to a file. While this transport is not included with the Java implementation, it should be simple enough to implement.
TMemoryTransport - Uses memory for I/O. The Java implementation uses a simple ByteArrayOutputStream internally.
TZlibTransport - Performs compression using zlib. Used in conjunction with another transport. Not available in the Java implementation.

1 个答案:

答案 0 :(得分:0)

有两种传输方式:

  • 端点传输
  • 分层运输

前者是您需要(其中一个)在线路上写入和读取数据的前者。例如,这可能是TSocket。

后者另外使用,在某些情况下甚至可以组合使用。例如,TFramedTransport为数据添加了一个特殊层,以使内存分配和I / O更高效。 zlib传输可用于压缩数据。

一个例子可能是:

+------------------------------------+
|  Application code                  |
+------------------------------------+
|  TBinaryProtocol                   |
+------------------------------------+
|  TZLibTransport                    |
+------------------------------------+
|  TFramedTransport                  |
+------------------------------------+
|  TSocket transport                 |
+------------------------------------+

另一个没有分层传输的人:

+------------------------------------+
|  Application code                  |
+------------------------------------+
|  TBinaryProtocol                   |
+------------------------------------+
|  TSocket transport                 |
+------------------------------------+

PS:您链接的不是官方文档,这是由一些第三方人员设置的,Apache Thrift项目对该网站的质量没有影响。

强烈推荐the forthcoming Manning book from Randy Abernethy。他是一个节俭提交者,这本书提供了宝贵的见解。不,我没有得到任何建议。