Redis - 解析远程服务器提供的数据流

时间:2017-01-02 12:03:20

标签: python redis data-stream redis-py

我已经在本地设置了Redis,我想连接到一个远程服务器,该服务器以<形式提供合成数据流。 ID,值>。到目前为止,我设法使用套接字连接到上面提到的服务器,读取数据流并打印它。相反,我想将这些对存储在一个哈希数据结构中(我将在以后存储关于每个ID的更多信息)。问题是我不知道如何解析数据流以便使用hget以及如何连续使用它。在更高级别,我希望能够将传入数据流中的Name和Value作为hget的参数传递。忘了提我正在使用Python API。 到目前为止:

import socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('xx.xx.xx.xxx', 1337))
while 1:
        data = client_socket.recv(512)
        print data

数据流的示例:

'AMZN,780.6758\n'
'TSLA,197.1802\n'
'CSCO,29.7491\n'
'GOOG,761.3758\n'
'AAPL,112.4122\n'
'GRPN,4.5848\n'
'FB,121.1232\n'
'MSFT,60.3529\n'
'INTC,35.9056\n'
'NVDA,94.473\n'
'QCOM,68.7389\n'
'AMZN,780.6761\n'
'TSLA,197.1798\n'
'CSCO,29.7486\n'
'GOOG,761.3755\n'
'AAPL,112.4122\n'
'GRPN,4.5848\n'
'FB,121.1237\n'
'MSFT,60.353\n'
'INTC,35.9054\n'
'NVDA,94.473\n'
'QCOM,68.7391\n'

我不确定是否保证所有线路都经过完全格式化,但让它们成为现实。

1 个答案:

答案 0 :(得分:1)

将单个非空行解析为键/值对非常简单:

key, value = line.strip().split(",", 1)

假设您的数据可能不完整(未终止的记录)并且它是标记记录结束的换行符,您可以将不完整的记录存储在缓冲区中并在解析之前将其添加回来,这样您的函数可能会看起来如此像这样的东西:

 def run(client_socket):
    buffer = ""
    while True:
        data = client_socket.recv(512)
        # not sure the following lines makes sense - 
        # you may actually want to handle exceptions 
        # or whatever
        if not data:
            break

        # add the buffer back 
        data = buffer + data
        # split on newlines
        lines = data.splitlines()
        # check if we have an incomplete record
        # (if it doesn't end with a newline)
        if data[-1] !=  '\n':
            # incomplete record, store it back so
            # we process it next time
            buffer = lines.pop()
        else:
            # all records complete for this call, 
            # empty the buffer for next turn
            buffer = "" 

        # now handle our records:    
        for line in filter(None, lines):
            k, v = line.split(",", 1)
            do_something_with(k, v)

do_something_with(k, v)的实施留给读者一个练习。