使用jq

时间:2018-09-28 15:05:50

标签: json shell jq categorization

由于我正在为Check_MK(以前称为Nagios)开发自定义检查,而我的应用程序(带有WebUI API的qBittorrent)返回JSON字符串,因此我正在学习如何在Linux中将jq与shell一起使用。

目前,仅使用一个简单的jq length就可以计算种子的总数。现在,我要计算当前正在下载,播种或暂停的种子的数量。我只对state感兴趣,所以如果我有6个种子,我的JSON可能像这样:

[
  {
    "state": "uploading"
  },
  {
    "state": "downloading"
  },
  {
    "state": "downloading"
  },
  {
    "state": "downloading"
  },
  {
    "state": "pauseDL"
  },
  {
    "state": "pauseUP"
  }
]

在这里,jq length返回6。我需要怎么做才能获取详细信息,例如3正在下载,1正在上传,2暂停和0错误?

这是我的实际脚本:

#!/bin/sh
curl -s http://localhost:8080/query/torrents -o /tmp/torrents.json
count=$(jq length /tmp/torrents.json)
echo "0 qbt_Nb_torrents - $count"

Check_MK需要echo的语法(如here所述)。

我已经阅读了多个有关过滤器的示例,但是当我们通过顶级属性进行过滤时,它们似乎都可以正常工作。在这里,我的最高级别基本上是[0],...,[5],因此它与我在手册中找到的示例不兼容。

其他信息

WebUI API表示存在12种可能的状态。这就是我打算将它们分开的方式:

downloading: queuedDL, checkingDL, downloading 
uploading: queuedUP, checkingUP, uploading 
pause: pausedUP, pausedDL 
error: error 
stalled: stalledUP, stalledDL, metaDL

按照CheckMK语法,我基本上需要输出以下内容:

  

0 qbt_Nb_torrents-总共6个,下载3个,播种1个,暂停2个,停顿0个,错误0个

开头的第一个0表示CheckMK的OK状态。如果有任何停止的洪流,我希望该状态变为1,并且如果有任何洪流错误,则状态变为2。示例:

  

2个qbt_Nb_torrents-总计8个,下载3个,播种1个,暂停2个,停顿1个,错误1个

3 个答案:

答案 0 :(得分:4)

对于其他有相关问题但未共享OP特定要求的人:请参阅编辑历史记录!在此答案的先前迭代中,还有其他一些相关的建议,包括group_by的使用。


如果您需要所有 值的条目,甚至没有出现的值,则可以考虑:

jq -r '
  def filterStates($stateMap):
    if $stateMap[.] then $stateMap[.] else . end;

  def errorLevel:
    if (.["error"] > 0) then 2 else
      if (.["stalled"] > 0) then 1 else
        0
      end
    end;

  {"queuedDL": "downloading", 
   "checkingDL": "downloading",
   "queuedUP": "uploading", 
   "checkingUP": "uploading",
   "pausedUP": "pause", 
   "pausedDL": "pause",
   "stalledUP": "stalled", 
   "stalledDL": "stalled", 
   "metaDL": "stalled"} as $stateMap |

  # initialize an output array since we want 0 outputs for everything
  {"pause": 0,  "stalled": 0, "error": 0, "downloading": 0, "uploading": 0} as $counts |

  # count number of items which filter to each value
  reduce (.[].state | filterStates($stateMap)) as $state ($counts; .[$state]+=1) |

  # actually format an output string
  "\(. | errorLevel) qbt_Nb_torrents - \(values | add) total, \(.["downloading"]) downloading, \(.["uploading"]) seeding, \(.["pause"]) on pause, \(.["stalled"]) stalled, \(.["error"]) error"
' /tmp/torrents.json

答案 1 :(得分:1)

这里是@CharlesDuffy解决方案的模块化程度更高的版本。主要兴趣点可能是通用的“单词袋”过滤器:

# bag of words
def bow(init; s): reduce s as $word (init; .[$word] += 1) ;

还要注意初始化功能:

# initialize an output object since we minimally want 0s
def init:
  {} | {pause,stalled,error,downloading,uploading} | map_values(0);

有了这些附加的抽象,“主”程序就变成了两行代码。

  def filterStates($stateMap):
    if $stateMap[.] then $stateMap[.] else . end ;

  def errorLevel:
    if .error > 0 then 2
    elif .stalled > 0 then 1
    else 0
    end ;

  def stateMap:
    {"queuedDL": "downloading", 
     "checkingDL": "downloading",
     "queuedUP": "uploading", 
     "checkingUP": "uploading",
     "pausedUP": "pause", 
     "pausedDL": "pause",
     "stalledUP": "stalled", 
     "stalledDL": "stalled", 
     "metaDL": "stalled"} ;

“主要”

  # count number of items which map to each value
  bow(init; .[].state | filterStates(stateMap))
  # format an output string
  | "\(errorLevel) qbt_Nb_torrents - \(values | add) total, \(.downloading) downloading, \(.uploading) seeding, \(.pause) on pause, \(.stalled) stalled, \(.error) error"

答案 2 :(得分:0)

以防万一有人怀疑我遵循Charles Duffy的出色答案到底使用了什么,下面是完整的/usr/lib/check_mk_agent/local/qbittorrent shell脚本,该脚本允许Check_MK(原始1.5.0)获得我认为最相关的内容有关在服务器上的专用VM中运行的qBittorrent应用程序(qBittorrent v3.3.7 Web UI)的信息:

#!/bin/sh
curl -s http://localhost:8080/query/transferInfo -o /tmp/transferInfo.json
curl -s http://localhost:8080/query/torrents -o /tmp/torrents.json

if [ -e /tmp/transferInfo.json ]
then
 dwl=$(jq .dl_info_speed /tmp/transferInfo.json)
 dwl_MB=$(bc <<< "scale=2;$dwl/1048576")
 upl=$(jq .up_info_speed /tmp/transferInfo.json)
 upl_MB=$(bc <<< "scale=2;$upl/1048576")
 echo "0 qbt_Global_speed download=$dwl_MB|upload=$upl_MB Download: $dwl_MB MB/s, Upload: $upl_MB MB/s"
 rm -f /tmp/transferInfo.json
else
 echo "3 qbt_Global_speed download=0|upload=0 Can't get the information from qBittorrent WebUI API"
fi

if [ -e /tmp/torrents.json ]
then    
 jq -r '
   def filterStates($stateMap):
    if $stateMap[.] then $stateMap[.] else . end;

   {"queuedDL": "downloading",
   "checkingDL": "downloading",
   "queuedUP": "uploading",
   "checkingUP": "uploading",
   "pausedUP": "pause",
   "pausedDL": "pause",
   "stalledUP": "stalled",
   "stalledDL": "stalled",
   "metaDL": "stalled"} as $stateMap |

   # initialize an output array since we want 0 outputs for everything
   {"pause": 0,  "stalled": 0, "error": 0, "downloading": 0, "uploading": 0} as $counts |

   # count number of items which filter to each value
   reduce (.[].state | filterStates($stateMap)) as $state ($counts; .[$state]+=1) |

   # output string
   "P qbt_Nb_torrents total=\(values|add)|downloading=\(.["downloading"])|seeding=\(.["uploading"])|pause=\(.["pause"])|stalled=\(.["stalled"]);0|error=\(.["error"]);0;0 total is \(values|add), downloading is \(.["downloading"]), seeding is \(.["uploading"]), pause is \(.["pause"])"
' /tmp/torrents.json

 rm -f /tmp/torrents.json
else
 echo "3 qbt_Nb_torrents total=0|downloading=0|seeding=0|pause=0|stalled=0;0|error=0;0;0 Can't get the information from qBittorrent WebUI API"
fi

以下是带有1个停顿的torrent的输出:

0 qbt_Global_speed download=0|upload=0 Download: 0 MB/s, Upload: 0 MB/s
P qbt_Nb_torrents total=1|downloading=0|seeding=0|pause=0|stalled=1;0|error=0;0;0 total is 1, downloading is 0, seeding is 0, pause is 0

由于Check_MK的工作原理,我认为不需要的errorLevel(请参阅Charles的答案)不是必需的。指定警告和临界值后,它将通过the metric parameter自行处理阈值。

这是Check_MK中的样子:

Check_MK view

注意Check_MK如何自动添加停滞和错误的种子。这是因为指定了警告和/或严重阈值。总体而言(带有或不带有阈值)的指标对于提供详细的相关图表很重要。