unexpected loop cycles with jq

问题

I'm trying to print a table in bash to stdout from json with jq:

[
    {
      "key": "name",
      "doc_count": 1000,
      "values_over_time": {
        "buckets": [
          {
            "key_as_string": "2019-05-01 11:00:00.000",
            "key": 1556708400000,
            "doc_count": 50
          },
          {
            "key_as_string": "2019-05-02 12:00:00.000",
            "key": 1556798400000,
            "doc_count": 40
          },
          {
            "key_as_string": "2019-05-02 13:00:00.000",
            "key": 1556802000000,
            "doc_count": 30
          }
        ]
      }
    }
]

With jq -r '(.[].key + " " + .[].values_over_time[][].key_as_string) + " " + (.[].values_over_time[][].doc_count|tostring)' I get these results:

"name 2019-05-01 11:00:00.000 50"
"name 2019-05-02 12:00:00.000 50"
"name 2019-05-02 13:00:00.000 50"
"name 2019-05-01 11:00:00.000 40"
"name 2019-05-02 12:00:00.000 40"
"name 2019-05-02 13:00:00.000 40"
"name 2019-05-01 11:00:00.000 30"
"name 2019-05-02 12:00:00.000 30"
"name 2019-05-02 13:00:00.000 30"

There is something strange with additional loop level as I expect to see only 3 lines:

"name 2019-05-01 11:00:00.000 50"
"name 2019-05-02 12:00:00.000 40"
"name 2019-05-02 13:00:00.000 30"

Read jq docs but wasn't able to understand the right way to iterate in a proper in a neat way. Do you have any clues?

JQplay code sample

回答1:

You're expanding values_over_time twice, thus 3 * 3 = 9 outputs are generated. Do something like this instead:

.[] | .key + " " + (.values_over_time.buckets[] | "\(.key_as_string) \(.doc_count)")

Compare these two commands to see the difference clearly:

$ jq -nc '[1,2,3] | [.[]*.[]]'
[1,2,3,2,4,6,3,6,9]

$ jq -nc '[1,2,3] | [.[]|.*.]'
[1,4,9]

回答2:

The technical reason is that you're combining expressions that generate multiple results. And the way jq operates, it will generate results for every combination of those subexpressions.

Looking at your filter and your input, here's how many results are generated by each subexpression:

(.[].key + " " + .[].values_over_time[][].key_as_string) + " " + (.[].values_over_time[][].doc_count|tostring)
(1         1     3)                                        1     (3                                 | 1)
(1 * 1 * 3 = 3)                                            1     (3 * 1 = 3)
 3 * 1 * 3 = 9

This is why you must always take care in using expressions that generate multiple results (e.g., []).

As Oguz points out, you need to write it such that the input is expanded only once in the expression. Usually right at the beginning of the filter is best.

This could be done in many ways, I would write it like so:

.[] | "\(.key) \(.values_over_time.buckets[] | "\(.key_as_string) \(.doc_count)")"
1   |   (1)     (3                           |   (1)               (1))
1   |   (1)     (3                           | 1 * 1 = 1)
1   |   (1)     (3 * 1 = 3)
1   | (1 * 3 = 3)
1 * 3 = 3

来源：https://stackoverflow.com/questions/56065822/unexpected-loop-cycles-with-jq

标签

json