与“ --stream”选项一起使用时,对小文件的转换失败(由于文件大小而需要)

时间:2019-04-09 17:38:29

标签: json jq

JQ播放片段:https://jqplay.org/s/D5-FZl8wOs

我正在使用jq展平用于sql的json数组。

json:

{
"0123":[
    {"i":0,"p":"file 1","l":100},
    {"i":1,"p":"file 2","l":200}
    ],
"0234":[
    {"i":0,"p":"file 1","l":100},
    {"i":1,"p":"file 2","l":200}
    ]
}

jq:

jq -r to_entries[] | {hash: .key, val: .value[]} | [.hash, .val.i, .val.p, .val.l]

所需的输出:

[
  "0123",
  0,
  "file 1",
  100
]
[
  "0123",
  1,
  "file 2",
  200
]
[
  "0234",
  0,
  "file 1",
  100
]
[
  "0234",
  1,
  "file 2",
  200
]

以上仅在文件较小时起作用,但是现在我遇到内存错误/操作系统将其增大时将其杀死。

如果传递--stream参数,则会出现错误:

jq: error (at <stdin>:9): Cannot index array with string "i"

我该如何解决?

2 个答案:

答案 0 :(得分:2)

以下操作将对示例输入有效。

foreach inputs as $pv ([[],[]]; # [A, B]
  if ($pv|length) == 2          # if pv is a path-value pair
  then .[0] |= if . == []       # if A is empty
    then . + [$pv[0][0],$pv[1]] # add first key from path definition and the value located at path to A
    else . + [$pv[1]] end       # add value to A
  else [[],.[0]] end;           # move A to B's place, leave A empty
  if .[0] == [] and .[1] != []  # if A is empty but B is not
  then .[1] else empty end      # print B
)

调用:

jq --stream -n 'foreach inputs as $pv ([[],[]]; if ($pv|length) == 2 then (.[0] |= if . == [] then . + [$pv[0][0],$pv[1]] else . + [$pv[1]] end) else [[],.[0]] end; if .[0] == [] and .[1] != [] then .[1] else empty end)' file

jqplay:https://jqplay.org/s/Q81EZahkjG

答案 1 :(得分:1)

  

我需要一种方法来使to_entries []与流媒体一起使用

这是一个可以做到的def:

def atomize(s):
    fromstream(foreach s as $in ( {previous:null, emit: null};
      if ($in | length == 2) and ($in|.[0][0]) != .previous and .previous != null
      then {emit: [[.previous]], previous: $in|.[0][0]}
      else { previous: ($in|.[0][0]), emit: null}
      end;
      (.emit // empty), $in) ) ;

有了这个定义,您可以通过在atomize(inputs)前面加上-n来假设使用同时调用--streamatomize(inputs) | to_entries[] | {hash: .key, val: .value[]} | [.hash, .val.i, .val.p, .val.l] 选项的jq来使用过滤器。也就是说,您的主要过滤器为:

atomize(inputs)
| to_entries[]
| .value[] as $value
| [.key, $value[]]

替代

如果JSON是完全常规的(如示例中所示),则可以编写:

switch (animanum) {
   case 1:
    animateCSS('.abarcador', 'swing');
   break;
   case 2:
    animateCSS('.abarcador', 'pulse');
   break;
   case 3:
    animateCSS('.abarcador', 'bounceInLeft');
   break;
   case 4:
    animateCSS('.abarcador', 'headShake');
   break;
   case 5:
    animateCSS('.abarcador', 'flipInX');
   break;
}

animanum++;

if (animanum >= 6) {
    animanum=1;
}