path: root/docs/content/3.manual/manual.yml

headline: jq Manual
body: |

  A jq program is a "filter": it takes an input, and produces an
  output. There are a lot of builtin filters for extracting a
  particular field of an object, or converting a number to a string,
  or various other standard tasks.

  Filters can be combined in various ways - you can pipe the output of
  one filter into another filter, or collect the output of a filter
  into an array.
  
  Some filters produce multiple results, for instance there's one that
  produces all the elements of its input array. Piping that filter
  into a second runs the second filter for each element of the
  array. Generally, things that would be done with loops and iteration
  in other languages are just done by gluing filters together in jq.

  It's important to remember that every filter has an input and an
  output. Even literals like "hello" or 42 are filters - they take an
  input but always produce the same literal as output. Operations that
  combine two filters, like addition, generally feed the same input to
  both and combine the results. So, you can implement an averaging
  filter as `add / length` - feeding the input array both to the `add`
  filter and the `length` filter and dividing the results.

  But that's getting ahead of ourselves. :) Let's start with something
  simpler:
  
sections:
  - title: Invoking jq
    body: |
      
      jq filters run on a stream of JSON data. The input to jq is
      parsed as a sequence of whitespace-separated JSON values which
      are passed through the provided filter one at a time. The
      output(s) of the filter are written to standard out, again as a
      sequence of whitespace-separated JSON data.

      You can affect how jq reads and writes its input and output
      using some command-line options:

      * `--slurp`/`-s` 

        Instead of running the filter for each JSON object in the
        input, read the entire input stream into a large array and run
        the filter just once.
      
      * `--raw-input`/`-R` 
      
        Don't parse the input as JSON. Instead, each line of text is
        passed to the filter as a string. If combined with `--slurp`,
        then the entire input is passed to the filter as a single long
        string.

      * `--null-input`/`-n`

        Don't read any input at all! Instead, the filter is run once
        using `null` as the input. This is useful when using jq as a
        simple calculator or to construct JSON data from scratch.

      * `--compact-output` / `-c`

        By default, jq pretty-prints JSON output. Using this option
        will result in more compact output by instead putting each
        JSON object on a single line.

      * `--ascii-output` / `-a`

        jq usually outputs non-ASCII Unicode codepoints as UTF-8, even
        if the input specified them as escape sequences (like
        "\u03bc"). Using this option, you can force jq to produce pure
        ASCII output with every non-ASCII character replaced with the
        equivalent escape sequence.

      * `--raw-output` / `-r`

        With this option, if the filter's result is a string then it
        will be written directly to standard output rather than being
        formatted as a JSON string with quotes. This can be useful for
        making jq filters talk to non-JSON-based systems.

  - title: Basic filters
    entries:
      - title: "`.`"
        body: |
          
          The absolute simplest (and least interesting) filter
          is `.`. This is a filter that takes its input and
          produces it unchanged as output.

          Since jq by default pretty-prints all output, this trivial
          program can be a useful way of formatting JSON output from,
          say, `curl`.

        examples:
          - program: '.'
            input: '"Hello, world!"'
            output: ['"Hello, world!"']

      - title: "`.foo`"
        body: |
          
          The simplest *useful* filter is .foo. When given a
          JSON object (aka dictionary or hash) as input, it produces
          the value at the key "foo", or null if there\'s none present.

        examples:
          - program: '.foo'
            input: '{"foo": 42, "bar": "less interesting data"}'
            output: [42]
          - program: '.foo'
            input: '{"notfoo": true, "alsonotfoo": false}'
            output: ['null']

      - title: "`.[foo]`"
        body: |
          
          You can also look up fields of an object using syntax like
          `.["foo"]` (.foo above is a shorthand version of this). This
          one works for arrays as well, if the key is an
          integer. Arrays are zero-based (like javascript), so .[2]
          returns the third element of the array.

        examples:
          - program: '.[0]'
            input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}]'
            output: ['{"name":"JSON", "good":true}']

          - program: '.[2]'
            input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}]'
            output: ['null']

      - title: "`.[]`"
        body: |
          
          If you use the `.[foo]` syntax, but omit the index
          entirely, it will return *all* of the elements of an
          array. Running `.[]` with the input `[1,2,3]` will produce the
          numbers as three seperate results, rather than as a single
          array.

        examples:
          - program: '.[]'
            input: '[{name":"JSON", "good":true}, {"name":"XML", "good":false}]'
            output:
              - '{"name":"JSON", "good":true}'
              - '{"name":"XML", "good":false}'

          - program: '.[]'
            input: '[]'
            output: []

      - title: "`,`"
        body: |
          
          If two filters are separated by a comma, then the
          input will be fed into both and there will be multiple
          outputs: first, all of the outputs produced by the left
          expression, and then all of the outputs produced by the
          right. For instance, filter `.foo, .bar`, produces
          both the "foo" fields and "bar" fields as separate outputs.

        examples:
          - program: '.foo, .bar'
            input: '{"foo": 42, "bar": "something else", "baz": true}'
            output: ['42', '"something else"']

          - program: ".user, .projects[]"
            input: '{"user":"stedolan", "projects": ["jq", "wikiflow"]}'
            output: ['"stedolan"', '"jq"', '"wikiflow"']            
            
          - program: '.[4,2]'
            input: '["a","b","c","d","e"]'
            output: ['"d"', '"c"']
          
      - title: "`|`"
        body: |
          The | operator combines two filters by feeding the output(s) of
          the one on the left into the input of the one on the right. It\'s
          pretty much the same as the Unix shell\'s pipe, if you\'re used to
          that. 

          If the one on the left produces multiple results, the one on
          the right will be run for each of those results. So, the
          expression `.[] | .foo` retrieves the "foo" field of each
          element of the input array.

        examples:
          - program: '.[] | .name'
            input: '[{name":"JSON", "good":true}, {"name":"XML", "good":false}]'
            output: ['"JSON"', '"XML"']

  - title: Types and Values
    body: |
      
      jq supports the same set of datatypes as JSON - numbers,
      strings, booleans, arrays, objects (which in JSON-speak are
      hashes with only string keys), and "null".

      Booleans, null, strings and numbers are written the same way as
      in javascript. Just like everything else in jq, these simple
      values take an input and produce an output - `42` is a valid jq
      expression that takes an input, ignores it, and returns 42
      instead.

    entries:
      - title: Array construction - `[]`
        body: |
        
          As in JSON, `[]` is used to construct arrays, as in
          `[1,2,3]`. The elements of the arrays can be any jq
          expression. All of the results produced by all of the
          expressions are collected into one big array. You can use it
          to construct an array out of a known quantity of values (as
          in `[.foo, .bar, .baz]`) or to "collect" all the results of a
          filter into an array (as in `[.items[].name]`)
        
          Once you understand the "," operator, you can look at jq\'s array
          syntax in a different light: the expression [1,2,3] is not using a
          built-in syntax for comma-separated arrays, but is instead applying
          the `[]` operator (collect results) to the expression 1,2,3 (which
          produces three different results).

          If you have a filter `X` that produces four results,
          then the expression `[X]` will produce a single result, an
          array of four elements.

        examples:
          - program: "[.user, .projects[]]"
            input: '{"user":"stedolan", "projects": ["jq", "wikiflow"]}'
            output: ['["stedolan", "jq", "wikiflow"]']
      - title: Objects - `{}`
        body: |