Question

我的文本文件中有以下示例数据（只是一部分）。我试图将时间戳，dataFrame和rssi三个键提取到csv文件中。

packet"{\"test\":{\"id\":1479238177559,\"deveui\":\"0000000033035032\",\"timestamp\":\"2016-11-15T19:29:37.559Z\",\"dataFrame\":\"ABzuPdVNxrSEAV8=\",\"fcnt\":81,\"port\":5,\"rssi\":6,\"snr\":9.5,\"sf_used\":10,\"cr_used\":\"4/5\",\"device_redundancy\":0,\"time_on_air_ms\":288.76800000000003,\"decrypted\":true}}"
Received message in at 2016-11-15 14:29:43.611000
packet"{\"test\":{\"id\":1479238184069,\"deveui\":\"0000000033035032\",\"timestamp\":\"2016-11-15T19:29:44.069Z\",\"dataFrame\":\"ABzuPdVNxrSEAV8=\",\"fcnt\":82,\"port\":5,\"rssi\":6,\"snr\":8.5,\"sf_used\":10,\"cr_used\":\"4/5\",\"device_redundancy\":0,\"time_on_air_ms\":288.76800000000003,\"decrypted\":true}}"
Received message in at 2016-11-15 14:29:49.225000
packet"{\"test\":{\"id\":1479238189685,\"deveui\":\"0000000033035032\",\"timestamp\":\"2016-11-15T19:29:49.685Z\",\"dataFrame\":\"ABzuPdVNxrSEAV8=\",\"fcnt\":83,\"port\":5,\"rssi\":7,\"snr\":9.5,\"sf_used\":10,\"cr_used\":\"4/5\",\"device_redundancy\":0,\"time_on_air_ms\":288.76800000000003,\"decrypted\":true}}"
Received message in at 2016-11-15 14:29:56.410000
packet"{\"testl\":{\"id\":1479238196868,\"deveui\":\"0000000033035032\",\"timestamp\":\"2016-11-15T19:29:56.868Z\",\"dataFrame\":\"ABzuPdVNxrSEAV8=\",\"fcnt\":84,\"port\":5,\"rssi\":3,\"snr\":9.8,\"sf_used\":10,\"cr_used\":\"4/5\",\"device_redundancy\":0,\"time_on_air_ms\":288.76800000000003,\"decrypted\":true}}"

Answer 1

这显然是偶然被JSON编码两次的数据，因此它可以被解码两次以获得一个好的字典：

SessionConfiguration config = new SessionConfiguration.Builder()
.setClientId("<CLIENT_ID>")
.setClientSecret("<CLIENT_SECRET>")
.setEnvironment(Environment.SANDBOX)
.setScopes(Arrays.asList(Scope.PROFILE, Scope.REQUEST))
.setRedirectUri("<REDIRECT_URI>")
.build();

 OAuth2Credentials credentials = new OAuth2Credentials.Builder()
.setSessionConfiguration(config)
.build();

 String authorizationUrl = credentials.getAuthorizationUrl();

输出：

import json

with open('log.txt') as infile:
    packet = []
    for line in infile:
        if line.startswith('packet"{'):
            # Remove 'packet' prefix
            line = line[len('packet'):]
            packet = json.loads(json.loads(line))
            print('Packet:')
            print(packet)
            packet = packet.values()[0]
            print('Values:')
            print(packet['timestamp'], packet['dataFrame'], packet['rssi'])

Answer 2

您可以尝试这种方法：

import pandas as pd

content = []
for c in open('log.txt').readlines():
    if c.startswith('packet"{'):
        content.append(c[7:-2].decode('string_escape'))

df = pd.concat([pd.read_json(line, orient='index') for line in content])

df[['dataFrame', 'rssi', 'timestamp']].to_csv('out.csv', index=False, header=None)

out.csv文件中的数据是：

ABzuPdVNxrSEAV8=,6,2016-11-15 19:29:37.559
ABzuPdVNxrSEAV8=,6,2016-11-15 19:29:44.069
ABzuPdVNxrSEAV8=,7,2016-11-15 19:29:49.685
ABzuPdVNxrSEAV8=,3,2016-11-15 19:29:56.868

将文本文件解析为csv

2 个答案: