Bash-解析JSON以与InfluxDB一起使用

时间：2018-08-07 17:41:08

标签： json bash jq influxdb

我已经成功地完成了一些简单的脚本，以解析来自json的数据以导入到InfluxDB中（并在Grafana中查看），但这比我过去要复杂得多。 Cisco UCCX / Finesse的VoiceCSQDetailsS​​tats API看起来像下面的json。我想做的是让一个循环解析下面的json，并使用底部的curl将单个用户数据（基于每个id的agentId，agentName，agentState和agentState Duration轮询）轮询到InfluxDB。我将如何使用bash完成此任务？

将以下数据另存为以下变量：

jsonAgentId="id"
jsonAgentName="agentName"
jsonAgentState="agentState"
jsonAgentStateDuration="agentStateDuration"

VoiceCSQDetailsS​​tats json：

{
  "id": "muser",
  "operation": "UPDATE",
  "VoiceCSQDetailsStats": {
    "agentId": "muser",
    "agentName": "My User",
    "agentState": "Not Ready",
    "skillGroup": "",
    "agentStateDuration": 982761,
    "reason": "Break",
    "AgentVoiceCSQNames": [
      {
        "agentVoiceCSQName": "Dispatcher"
      },
      {
        "agentVoiceCSQName": "NOCEscalation"
      },
      {
        "agentVoiceCSQName": "NOCHelpdesk"
      }
    ]
  }
}
{
  "id": "yuser",
  "operation": "UPDATE",
  "VoiceCSQDetailsStats": {
    "agentId": "yuser",
    "agentName": "Your User",
    "agentState": "Talking",
    "skillGroup": "",
    "agentStateDuration": 626160,
    "reason": "",
    "AgentVoiceCSQNames": [
      {
        "agentVoiceCSQName": "Dispatcher"
      },
      {
        "agentVoiceCSQName": "NOCHelpdesk"
      }
    ]
  }
}
{
  "id": "euser",
  "operation": "UPDATE",
  "VoiceCSQDetailsStats": {
    "agentId": "euser",
    "agentName": "Everyones User",
    "agentState": "Ready",
    "skillGroup": "",
    "agentStateDuration": 203631,
    "reason": "",
    "AgentVoiceCSQNames": [
      {
        "agentVoiceCSQName": "NOCHelpdesk"
      },
      {
        "agentVoiceCSQName": "NOCEscalation"
      }
    ]
  }
}
{
  "id": "duser",
  "operation": "UPDATE",
  "VoiceCSQDetailsStats": {
    "agentId": "duser",
    "agentName": "Dumb User",
    "agentState": "Not Ready",
    "skillGroup": "",
    "agentStateDuration": 175342,
    "reason": "Call Not Answered",
    "AgentVoiceCSQNames": [
      {
        "agentVoiceCSQName": "NOCEscalation"
      },
      {
        "agentVoiceCSQName": "NOCHelpdesk"
      }
    ]
  }
}
{
  "id": "fuser",
  "operation": "UPDATE",
  "VoiceCSQDetailsStats": {
    "agentId": "fuser",
    "agentName": "Foolish User",
    "agentState": "Not Ready",
    "skillGroup": "",
    "agentStateDuration": 1057520,
    "reason": "Offhook",
    "AgentVoiceCSQNames": [
      {
        "agentVoiceCSQName": "NOCEscalation"
      },
      {
        "agentVoiceCSQName": "NOCHelpdesk"
      }
    ]
  }
}
{
  "id": "druser",
  "operation": "UPDATE",
  "VoiceCSQDetailsStats": {
    "agentId": "druser",
    "agentName": "Drug User",
    "agentState": "Talking ( from CSQ: NOCHelpdesk )",
    "skillGroup": "NOCHelpdesk",
    "agentStateDuration": 167914,
    "reason": "Offhook",
    "AgentVoiceCSQNames": [
      {
        "agentVoiceCSQName": "NOCEscalation"
      },
      {
        "agentVoiceCSQName": "NOCHelpdesk"
      }
    ]
  }
}

发送到InfluxDB：

curl -i -XPOST "$influxdbIP:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=$jsonAgentId value=$jsonAgentName"
curl -i -XPOST "$influxdbIP:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=$jsonAgentId value=$jsonAgentState"
curl -i -XPOST "$influxdbIP:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=$jsonAgentId value=$jsonAgentStateDuration"

我曾经考虑过使用jq做下面的事情，但是我仍然需要它遍历用户并在jq -c'。[n]'返回null时停止。 

curl 'http://10.10.66.16:9080/realtime/VoiceCSQDetailsStats' | jq -c '.[6]' | grep -oP '(?<="id":")[^."]*'
druser

curl 'http://10.10.66.16:9080/realtime/VoiceCSQDetailsStats' | jq -c '.[6]' | grep -oP '(?<="agentName":")[^."]*'
Drug User

curl 'http://10.10.66.16:9080/realtime/VoiceCSQDetailsStats' | jq -c '.[6]' | grep -oP '(?<="agentState":")[^."]*'
Ready

curl 'http://10.10.66.16:9080/realtime/VoiceCSQDetailsStats' | jq -c '.[6]' | grep -oP '(?<="agentStateDuration":)[^.,]*'
167914

curl 'http://10.10.66.16:9080/realtime/VoiceCSQDetailsStats' | jq -c '.[6]' | grep -oP '(?<="reason":")[^."]*'
Offhook

2 个答案:

答案 0 :(得分：0)

您可以使用jq提取所需字段并将值分配给bash变量：

<file jq -r '.VoiceCSQDetailsStats | [ .agentId,.agentName,.agentState,.agentStateDuration ] | @tsv' \
| while IFS=$'\t' read jsonAgentId jsonAgentName jsonAgentState jsonAgentStateDuration; do
    echo -e "jsonAgentId=${jsonAgentId}\njsonAgentName=${jsonAgentName}\njsonAgentState=${jsonAgentState}\njsonAgentStateDuration=${jsonAgentStateDuration}\n";
done

@tsv运算符允许使用制表符\t分隔所有值，然后通过相应地设置输入字段分隔符IFS，shell可以轻松地对其进行解析。

如果您想直接解析JSON数据，则可能希望将<file替换为curl ... |。

答案 1 :(得分：0)

我的解决方案是让jq为您编写curl命令：

从简单开始：只需获取具有所需数据的对象

$ jq -c '.VoiceCSQDetailsStats | { id: .agentId, name: .agentName, state: .agentState, stateDuration: .agentStateDuration } | select(.id and .name and .state and .stateDuration)' < eptesicus.json
{"id":"muser","name":"My User","state":"Not Ready","stateDuration":982761}
{"id":"yuser","name":"Your User","state":"Talking","stateDuration":626160}
{"id":"euser","name":"Everyones User","state":"Ready","stateDuration":203631}
{"id":"duser","name":"Dumb User","state":"Not Ready","stateDuration":175342}
{"id":"fuser","name":"Foolish User","state":"Not Ready","stateDuration":1057520}
{"id":"druser","name":"Drug User","state":"Talking ( from CSQ: NOCHelpdesk )","stateDuration":167914}

对此进行扩展：让jq编写您的curl语句

$ influxdbIP=1.2.3.4 jq -r '.VoiceCSQDetailsStats | { id: .agentId, name: .agentName, state: .agentState, stateDuration: .agentStateDuration } | select(.id and .name and .state and .stateDuration) | .id as $id | ( .name, .state, .stateDuration) | "curl -i -XPOST \"\( $ENV | .influxdbIP):8086/write?db=nocdb\" --data-binary \"uccxstats,host=uccx,user=\($id) value=\(.)\""' < eptesicus.json
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=muser value=My User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=muser value=Not Ready"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=muser value=982761"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=yuser value=Your User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=yuser value=Talking"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=yuser value=626160"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=euser value=Everyones User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=euser value=Ready"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=euser value=203631"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=duser value=Dumb User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=duser value=Not Ready"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=duser value=175342"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=fuser value=Foolish User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=fuser value=Not Ready"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=fuser value=1057520"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=druser value=Drug User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=druser value=Talking ( from CSQ: NOCHelpdesk )"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=druser value=167914"

整理

如果其中任何一个字段缺失或为空，则select(…)将生成否 curl请求。

如果愿意，可以这样扩展：

$ influxdbIP=1.2.3.4 jq -r '.VoiceCSQDetailsStats | { id: .agentId, name: .agentName, state: .agentState, stateDuration: .agentStateDuration } | select(.id and .name and .state and .stateDuration and ( .state != "Not Ready" ) ) | .id as $id | ( .name, .state, .stateDuration) | "curl -i -XPOST \"\( $ENV | .influxdbIP):8086/write?db=nocdb\" --data-binary \"uccxstats,host=uccx,user=\($id) value=\(.)\""' < eptesicus.json
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=yuser value=Your User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=yuser value=Talking"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=yuser value=626160"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=euser value=Everyones User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=euser value=Ready"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=euser value=203631"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=druser value=Drug User"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=druser value=Talking ( from CSQ: NOCHelpdesk )"
curl -i -XPOST "1.2.3.4:8086/write?db=nocdb" --data-binary "uccxstats,host=uccx,user=druser value=167914"

请注意，对于处于“未就绪”状态的座席，它不会产生curl个请求。

相关问题
最新问题