使用简单的json解析设备数据并在hdfs中构建逗号分隔文件

时间:2016-03-07 18:47:07

标签: java json hadoop

我的问题是如何从设备的api解析json响应。 json看起来如下。

  

[{“name”:“Device 1”,“label”:“Switch   状态“,”值“:”关“},{”名称“:”设备2“,”标签“:”切换   状态”, “值”: “上”}]

public class Fubar{

public static void main(String[] args) throws Exception {
    boolean a = true;
    while (a) {
        Configuration conf = new Configuration();
        FileSystem hdfs = FileSystem.get(conf);
        String fileName = "filethingy" + "-" + new Date().getTime() + ".txt";

        boolean b = true;
        while(b){
            String connString = "url";
            URL url = new URL(connString);
            HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            conn.setRequestMethod("GET");
            BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
            String inputLine;
            out.writeBytes("TimeStamp"+",");
            out.writeBytes("DeviceName"+",");
            out.writeBytes("DeviceLabel"+",");
            out.writeBytes("DeviceStatus"+",");
            out.writeBytes("\n");
            while ((inputLine = in.readLine()) != null) {
                JSONParser jsonParser = new JSONParser();
                JSONObject jsonObject = (JSONObject) jsonParser.parse(inputLine);

                String date = new java.sql.Timestamp(System.currentTimeMillis()).toString();
                out.write(date.getBytes());
                out.write(",".getBytes());
                out.writeBytes((String) jsonObject.get("name"));
                out.write(",".getBytes());
                out.writeBytes((String) jsonObject.get("label"));
                out.write(",".getBytes());
                out.writeBytes((String) jsonObject.get("value"));
                //out.write(inputLine.getBytes());
                out.write("\n".getBytes());
                out.hflush();
            }

            in.close();
            Thread.sleep(30000);
            out.hflush();

        }
        out.close();
        hdfs.close();
        a=false;
        b=false;
    }

这不起作用我知道,因为我无法将JSONArray转换为JSONObject。我无法确定如何正确地拆分JSON,尽管最终构建一个看起来像TimeStamp,DeviceName,DeviceLabel,DeviceStatus,01012001,Device1,Switch Status,On的csv。当我们完成时,有人会在数据之上放置一个Hive表。任何帮助将不胜感激。

1 个答案:

答案 0 :(得分:0)

回答了我自己的问题。

names = c('A','A','A','B','B','B')
datetime = c('2016-01-11 02:00:04 PST','2016-01-11 02:10:04 PST','2016-01-11 02:20:04 PST','2016-01-02 03:55:58 PST','2016-01-02 03:59:58 PST','2016-01-02 04:10:58 PST')
latitude = c(40.07054,39.82271579,39.83636098, 40.009918,39.94432271,39.91651225)
longitude = c(-76.288572, -77.2300438, -77.2061907,-75.188196, -76.5933571, -77.1641403)
mode = c('Bike','Bike','Bike','Car','Car','Car')
test = data.frame(names,datetime,latitude,longitude,mode)

library("ggmap")
m <- get_map(location=c(lon=median(test$longitude), lat=median(test$latitude)), zoom=8)
ggmap(m) + geom_point(aes(x=longitude, y=latitude, color=mode), data=test) + 
  geom_line(aes(x=longitude, y=latitude, color=mode), data=test)