jq: shell 脚本中JSON创建、解析工具
0x01 背景
在Linux下开发监控、自动化shell脚本是运维的日常工作之一，JSON是通用且流行的数据交换格式，在Web API中使用很普遍
0x02 问题/诉求
如何在Linux shell脚本中创建与解析JSON呢？
0x03 土办法
1. 创建JSON
变量嵌入
#!/bin/bash
FILENAME="/tmp/xxxx"
curl -i -X POST -H "Content-Type: application/json" -d '{"type": "test", "params":{"item":{"file":"'"$FILENAME"'"}}}' http://192.168.0.1/api#!/bin/bash
header="Content-Type: application/json"
FILENAME="/tmp/xxxx"
request_body=$(< <(cat <<EOF
{
"type": "test",
"params": {
"item": {
"file": "$FILENAME"
}
}
}
EOF
))
curl -i -X POST -H "$header" -d "$request_body" http://192.168.0.1/api循环拼接
#!/bin/bash
arr=""
while read var; do
arr="\"${var}\",${arr}"
done < list
json=`echo ${arr} | sed 's/,$//g'`
echo "[${json}]"文本内容：
$ cat list
aa
bb
cc
dd
123
r4
t55
5t
gg
1
rgs
g
f
gs
脚本输出：
$ sh t.sh
["gs","f","g","rgs","1","gg","5t","t55","r4","123","dd","cc","bb","aa"]
2. 解析JSON
awk
没有具体尝试过，可参考stackoverflow上的一则回答
0x04 完美解法 —— jq
jq is a lightweight and flexible command-line JSON processor.
$ jq -h
jq - commandline JSON processor [version 1.5-1-a5b5cbe]
Usage: jq [options] <jq filter> [file...]
jq is a tool for processing JSON inputs, applying the
given filter to its JSON text inputs and producing the
filter's results as JSON on standard output.
The simplest filter is ., which is the identity filter,
copying jq's input to its output unmodified (except for
formatting).
For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq
Some of the options include:
-c compact instead of pretty-printed output;
-n use `null` as the single input value;
-e set the exit status code based on the output;
-s read (slurp) all inputs into an array; apply filter to it;
-r output raw strings, not JSON texts;
-R read raw strings, not JSON texts;
-C colorize JSON;
-M monochrome (don't colorize JSON);
-S sort keys of objects on output;
--tab use tabs for indentation;
--arg a v set variable $a to value <v>;
--argjson a v set variable $a to JSON value <v>;
--slurpfile a f set variable $a to an array of JSON texts read from <f>;
See the manpage for more options.1. 创建JSON
变量嵌入
$ FILENAME="/tmp/xxxx"
$ jq -n -c -M --arg v "$FILENAME" '{"type":"test","params":{"item":{"file": $v}}}'
{"type":"test","params":{"item":{"file":"/tmp/xxxx"}}}
// 去掉 -c 参数可格式化输出json
$ jq -n -M --arg v "$FILENAME" '{"type":"test","params":{"item":{"file": $v}}}'
{
"type": "test",
"params": {
"item": {
"file": "/tmp/xxxx"
}
}
}循环拼接
$ jq -R -M -c -s 'split("\n")' < list
["aa","bb","cc","dd","123","r4","t55","5t","gg","1","rgs","g","f","gs",""]数组末尾的空字符串，可以另想办法处理下。
2. 解析JSON
这块可直接学习jq官方教程，由浅入深，讲的很详细。
jq '.foo'
Input {"foo": 42, "bar": "less interesting data"}
Output 42
jq '.foo'
Input {"notfoo": true, "alsonotfoo": false}
Output null
jq '.["foo"]'
Input {"foo": 42}
Output 42
jq '.[0]'
Input [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
Output {"name":"JSON", "good":true}
jq '.[2]'
Input [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]
Output null
jq '.[2:4]'
Input ["a","b","c","d","e"]
Output ["c", "d"]
jq '.[2:4]'
Input "abcdefghi"
Output "cd"
jq '.[:3]'
Input ["a","b","c","d","e"]
Output ["a", "b", "c"]
jq '.[-2:]'
Input ["a","b","c","d","e"]
Output ["d", "e"]
jq '.[-2]'
Input [1,2,3]
Output 2
