☰

# std/json

Theme:

- Manual
- Standard library
- Index
- Compiler docs
- Fusion docs
- devel, stable

Search:
Source Edit

This module implements a simple high performance JSON parser. JSON (JavaScript Object Notation) is a lightweight data-interchange format that is easy for humans to read and write (unlike XML). It is easy for machines to parse and generate. JSON is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999.

# See also

- std/parsejson
- std/jsonutils
- std/marshal
- std/jscore

# Overview

## Parsing JSON

JSON often arrives into your program (via an API or a file) as a `string`. The first step is to change it from its serialized form into a nested object structure called a `JsonNode`.

The `parseJson` procedure takes a string containing JSON and returns a `JsonNode` object. This is an object variant and it is either a `JObject`, `JArray`, `JString`, `JInt`, `JFloat`, `JBool` or `JNull`. You check the kind of this object variant by using the `kind` accessor.

For a `JsonNode` who's kind is `JObject`, you can access its fields using the `[]` operator. The following example shows how to do this:

    import std/json

    let jsonNode = parseJson("""{"key": 3.14}""")

    doAssert jsonNode.kind == JObject
    doAssert jsonNode["key"].kind == JFloat

## Reading values

Once you have a `JsonNode`, retrieving the values can then be achieved by using one of the helper procedures, which include:

- `getInt`
- `getFloat`
- `getStr`
- `getBool`

To retrieve the value of `"key"` you can do the following:

    import std/json

    let jsonNode = parseJson("""{"key": 3.14}""")

    doAssert jsonNode["key"].getFloat() == 3.14

Important: The `[]` operator will raise an exception when the specified field does not exist.

## Handling optional keys

By using the `{}` operator instead of `[]`, it will return `nil` when the field is not found. The `get`-family of procedures will return a type's default value when called on `nil`.

    import std/json

    let jsonNode = parseJson("{}")

    doAssert jsonNode{"nope"}.getInt() == 0
    doAssert jsonNode{"nope"}.getFloat() == 0
    doAssert jsonNode{"nope"}.getStr() == ""
    doAssert jsonNode{"nope"}.getBool() == false

## Using default values

The `get`-family helpers also accept an additional parameter which allow you to fallback to a default value should the key's values be `null`:

    import std/json

    let jsonNode = parseJson("""{"key": 3.14, "key2": null}""")

    doAssert jsonNode["key"].getFloat(6.28) == 3.14
    doAssert jsonNode["key2"].getFloat(3.14) == 3.14
    doAssert jsonNode{"nope"}.getFloat(3.14) == 3.14 # note the {}

## Unmarshalling

In addition to reading dynamic data, Nim can also unmarshal JSON directly into a type with the `to` macro.

Note: Use Option for keys sometimes missing in json responses, and backticks around keys with a reserved keyword as name.

    import std/json
    import std/options

    type
      User = object
        name: string
        age: int
        `type`: Option[string]

    let userJson = parseJson("""{ "name": "Nim", "age": 12 }""")
    let user = to(userJson, User)
    if user.`type`.isSome():
      assert user.`type`.get() != "robot"

# Creating JSON

This module can also be used to comfortably create JSON using the `%*` operator:

    import std/json

    var hisName = "John"
    let herAge = 31
    var j = %*
      [
        { "name": hisName, "age": 30 },
        { "name": "Susan", "age": herAge }
      ]

    var j2 = %* {"name": "Isaac", "books": ["Robot Dreams"]}
    j2["details"] = %* {"age":35, "pi":3.1415}
    echo j2

See also: std/jsonutils for hookable json serialization/deserialization of arbitrary types.

Example:

    import std/json
    ## Note: for JObject, key ordering is preserved, unlike in some languages,
    ## this is convenient for some use cases. Example:
    type Foo = object
      a1, a2, a0, a3, a4: int
    doAssert $(%* Foo()) == """{"a1":0,"a2":0,"a0":0,"a3":0,"a4":0}"""

# Imports

hashes, tables, strutils, lexbase, streams, macros, parsejson, options, since, syncio, assertions, formatfloat

# Types

    JsonNode = ref JsonNodeObj

  JSON node Source Edit

    JsonNodeKind = enum
      JNull, JBool, JInt, JFloat, JString, JObject, JArray

  possible JSON node types Source Edit

    JsonNodeObj {.acyclic.} = object
      case kind*: JsonNodeKind
      of JString:
        str*: string
      of JInt:
        num*: BiggestInt
      of JFloat:
        fnum*: float
      of JBool:
        bval*: bool
      of JNull:
        nil
      of JObject:
        fields*: OrderedTable[string, JsonNode]
      of JArray:
        elems*: seq[JsonNode]

  Source Edit

# Procs

    proc `$`(node: JsonNode): string {....raises: [], tags: [], forbids: [].}

  Converts `node` to its JSON Representation on one line. Source Edit

    proc `%`(b: bool): JsonNode {....raises: [], tags: [], forbids: [].}

  Generic constructor for JSON data. Creates a new `JBool JsonNode`. Source Edit

    proc `%`(keyVals: openArray[tuple[key: string, val: JsonNode]]): JsonNode {.
        ...raises: [], tags: [], forbids: [].}

  Generic constructor for JSON data. Creates a new `JObject JsonNode` Source Edit

    proc `%`(n: BiggestInt): JsonNode {....raises: [], tags: [], forbids: [].}

  Generic constructor for JSON data. Creates a new `JInt JsonNode`. Source Edit

    proc `%`(n: BiggestUInt): JsonNode {....raises: [], tags: [], forbids: [].}

  Generic constructor for JSON data. Creates a new `JInt JsonNode`. Source Edit

    proc `%`(n: float): JsonNode {....raises: [], tags: [], forbids: [].}

  Generic constructor for JSON data. Creates a new `JFloat JsonNode`.

  Example:

      assert $(%[NaN, Inf, -Inf, 0.0, -0.0, 1.0, 1e-2]) == """["nan","inf","-inf",0.0,-0.0,1.0,0.01]"""
      assert (%NaN).kind == JString
      assert (%0.0).kind == JFloat

  Source Edit

    proc `%`(n: int): JsonNode {....raises: [], tags: [], forbids: [].}

  Generic constructor for JSON data. Creates a new `JInt JsonNode`. Source Edit

    proc `%`(n: uint): JsonNode {....raises: [], tags: [], forbids: [].}

  Generic constructor for JSON data. Creates a new `JInt JsonNode`. Source Edit

    proc `%`(o: enum): JsonNode

  Construct a JsonNode that represents the specified enum value as a string. Creates a new `JString JsonNode`. Source Edit

    proc `%`(o: ref object): JsonNode

  Generic constructor for JSON data. Creates a new `JObject JsonNode` Source Edit

    proc `%`(s: string): JsonNode {....raises: [], tags: [], forbids: [].}

  Generic constructor for JSON data. Creates a new `JString JsonNode`. Source Edit

    proc `%`[T: object](o: T): JsonNode

  Construct JsonNode from tuples and objects. Source Edit

    proc `%`[T](elements: openArray[T]): JsonNode

  Generic constructor for JSON data. Creates a new `JArray JsonNode` Source Edit

    proc `%`[T](opt: Option[T]): JsonNode

  Generic constructor for JSON data. Creates a new `JNull JsonNode` if `opt` is empty, otherwise it delegates to the underlying value. Source Edit

    proc `%`[T](table: Table[string, T] | OrderedTable[string, T]): JsonNode

  Generic constructor for JSON data. Creates a new `JObject JsonNode`. Source Edit

    proc `==`(a, b: JsonNode): bool {.noSideEffect, ...raises: [], tags: [RootEffect],
                                      forbids: [].}

  Check two nodes for equality Source Edit

    proc `[]`(node: JsonNode; index: BackwardsIndex): JsonNode {.inline, ...raises: [],
        tags: [], forbids: [].}

  Gets the node at `array.len-i` in an array through the `^` operator.

  i.e. `j[^i]` is a shortcut for `j[j.len-i]`.

  Example:

      let
        j = parseJson("[1,2,3,4,5]")

      doAssert j[^1].getInt == 5
      doAssert j[^2].getInt == 4

  Source Edit

    proc `[]`(node: JsonNode; index: int): JsonNode {.inline, ...raises: [], tags: [],
        forbids: [].}

  Gets the node at `index` in an Array. Result is undefined if `index` is out of bounds, but as long as array bound checks are enabled it will result in an exception. Source Edit

    proc `[]`(node: JsonNode; name: string): JsonNode {.inline, ...raises: [KeyError],
        tags: [], forbids: [].}

  Gets a field from a `JObject`, which must not be nil. If the value at `name` does not exist, raises KeyError. Source Edit

    proc `[]`[U, V](a: JsonNode; x: HSlice[U, V]): JsonNode

  Slice operation for JArray.

  Returns the inclusive range `[a[x.a], a[x.b]]`:

  Example:

      import std/json
      let arr = %[0,1,2,3,4,5]
      doAssert arr[2..4] == %[2,3,4]
      doAssert arr[2..^2] == %[2,3,4]
      doAssert arr[^4..^2] == %[2,3,4]

  Source Edit

    proc `[]=`(obj: JsonNode; key: string; val: JsonNode) {.inline, ...raises: [],
        tags: [], forbids: [].}

  Sets a field from a `JObject`. Source Edit

    proc add(father, child: JsonNode) {....raises: [], tags: [], forbids: [].}

  Adds `child` to a JArray node `father`. Source Edit

    proc add(obj: JsonNode; key: string; val: JsonNode) {....raises: [], tags: [],
        forbids: [].}

  Sets a field from a `JObject`. Source Edit

    proc contains(node: JsonNode; key: string): bool {....raises: [], tags: [],
        forbids: [].}

  Checks if `key` exists in `node`. Source Edit

    proc contains(node: JsonNode; val: JsonNode): bool {....raises: [],
        tags: [RootEffect], forbids: [].}

  Checks if `val` exists in array `node`. Source Edit

    proc copy(p: JsonNode): JsonNode {....raises: [], tags: [], forbids: [].}

  Performs a deep copy of `p`. Source Edit

    proc delete(obj: JsonNode; key: string) {....raises: [KeyError], tags: [],
        forbids: [].}

  Deletes `obj[key]`. Source Edit

    proc escapeJson(s: string): string {....raises: [], tags: [], forbids: [].}

  Converts a string `s` to its JSON representation with quotes. Source Edit

    proc escapeJson(s: string; result: var string) {....raises: [], tags: [],
        forbids: [].}

  Converts a string `s` to its JSON representation with quotes. Appends to `result`. Source Edit

    proc escapeJsonUnquoted(s: string): string {....raises: [], tags: [], forbids: [].}

  Converts a string `s` to its JSON representation without quotes. Source Edit

    proc escapeJsonUnquoted(s: string; result: var string) {....raises: [], tags: [],
        forbids: [].}

  Converts a string `s` to its JSON representation without quotes. Appends to `result`. Source Edit

    proc getBiggestInt(n: JsonNode; default: BiggestInt = 0): BiggestInt {.
        ...raises: [], tags: [], forbids: [].}

  Retrieves the BiggestInt value of a `JInt JsonNode`.

  Returns `default` if `n` is not a `JInt`, or if `n` is nil.

  Source Edit

    proc getBool(n: JsonNode; default: bool = false): bool {....raises: [], tags: [],
        forbids: [].}

  Retrieves the bool value of a `JBool JsonNode`.

  Returns `default` if `n` is not a `JBool`, or if `n` is nil.

  Source Edit

    proc getElems(n: JsonNode; default: seq[JsonNode] = @[]): seq[JsonNode] {.
        ...raises: [], tags: [], forbids: [].}

  Retrieves the array of a `JArray JsonNode`.

  Returns `default` if `n` is not a `JArray`, or if `n` is nil.

  Source Edit

    proc getFields(n: JsonNode; default = initOrderedTable[string, JsonNode](2)): OrderedTable[
        string, JsonNode] {....raises: [], tags: [], forbids: [].}

  Retrieves the key, value pairs of a `JObject JsonNode`.

  Returns `default` if `n` is not a `JObject`, or if `n` is nil.

  Source Edit

    proc getFloat(n: JsonNode; default: float = 0.0): float {....raises: [], tags: [],
        forbids: [].}

  Retrieves the float value of a `JFloat JsonNode`.

  Returns `default` if `n` is not a `JFloat` or `JInt`, or if `n` is nil.

  Source Edit

    proc getInt(n: JsonNode; default: int = 0): int {....raises: [], tags: [],
        forbids: [].}

  Retrieves the int value of a `JInt JsonNode`.

  Returns `default` if `n` is not a `JInt`, or if `n` is nil.

  Source Edit

    proc getOrDefault(node: JsonNode; key: string): JsonNode {....raises: [], tags: [],
        forbids: [].}

  Gets a field from a `node`. If `node` is nil or not an object or value at `key` does not exist, returns nil Source Edit

    proc getStr(n: JsonNode; default: string = ""): string {....raises: [], tags: [],
        forbids: [].}

  Retrieves the string value of a `JString JsonNode`.

  Returns `default` if `n` is not a `JString`, or if `n` is nil.

  Source Edit

    proc hash(n: JsonNode): Hash {.noSideEffect, ...raises: [Exception],
                                   tags: [RootEffect], forbids: [].}

  Compute the hash for a JSON node Source Edit

    proc hash(n: OrderedTable[string, JsonNode]): Hash {.noSideEffect,
        ...raises: [Exception], tags: [RootEffect], forbids: [].}

  Source Edit

    proc hasKey(node: JsonNode; key: string): bool {....raises: [], tags: [],
        forbids: [].}

  Checks if `key` exists in `node`. Source Edit

    proc len(n: JsonNode): int {....raises: [], tags: [], forbids: [].}

  If `n` is a `JArray`, it returns the number of elements. If `n` is a `JObject`, it returns the number of pairs. Else it returns 0. Source Edit

    proc newJArray(): JsonNode {....raises: [], tags: [], forbids: [].}

  Creates a new `JArray JsonNode` Source Edit

    proc newJBool(b: bool): JsonNode {....raises: [], tags: [], forbids: [].}

  Creates a new `JBool JsonNode`. Source Edit

    proc newJFloat(n: float): JsonNode {....raises: [], tags: [], forbids: [].}

  Creates a new `JFloat JsonNode`. Source Edit

    proc newJInt(n: BiggestInt): JsonNode {....raises: [], tags: [], forbids: [].}

  Creates a new `JInt JsonNode`. Source Edit

    proc newJNull(): JsonNode {....raises: [], tags: [], forbids: [].}

  Creates a new `JNull JsonNode`. Source Edit

    proc newJObject(): JsonNode {....raises: [], tags: [], forbids: [].}

  Creates a new `JObject JsonNode` Source Edit

    proc newJString(s: string): JsonNode {....raises: [], tags: [], forbids: [].}

  Creates a new `JString JsonNode`. Source Edit

    proc parseFile(filename: string): JsonNode {.
        ...raises: [IOError, OSError, JsonParsingError, ValueError],
        tags: [ReadIOEffect, WriteIOEffect], forbids: [].}

  Parses `file` into a `JsonNode`. If `file` contains extra data, it will raise `JsonParsingError`. Source Edit

    proc parseJson(buffer: string; rawIntegers = false; rawFloats = false): JsonNode {.
        ...raises: [IOError, OSError, JsonParsingError, ValueError],
        tags: [ReadIOEffect, WriteIOEffect], forbids: [].}

  Parses JSON from `buffer`. If `buffer` contains extra data, it will raise `JsonParsingError`. If `rawIntegers` is true, integer literals will not be converted to a `JInt` field but kept as raw numbers via `JString`. If `rawFloats` is true, floating point literals will not be converted to a `JFloat` field but kept as raw numbers via `JString`. Source Edit

    proc parseJson(s: Stream; filename: string = ""; rawIntegers = false;
                   rawFloats = false): JsonNode {.
        ...raises: [IOError, OSError, IOError, OSError, JsonParsingError, ValueError],
        tags: [ReadIOEffect, WriteIOEffect], forbids: [].}

  Parses from a stream `s` into a `JsonNode`. `filename` is only needed for nice error messages. If `s` contains extra data, it will raise `JsonParsingError`. This closes the stream `s` after it's done. If `rawIntegers` is true, integer literals will not be converted to a `JInt` field but kept as raw numbers via `JString`. If `rawFloats` is true, floating point literals will not be converted to a `JFloat` field but kept as raw numbers via `JString`. Source Edit

    proc pretty(node: JsonNode; indent = 2): string {....raises: [], tags: [],
        forbids: [].}

  Returns a JSON Representation of `node`, with indentation and on multiple lines.

  Similar to prettyprint in Python.

  Example:

      let j = %* {"name": "Isaac", "books": ["Robot Dreams"],
                  "details": {"age": 35, "pi": 3.1415}}
      doAssert pretty(j) == """
      {
        "name": "Isaac",
        "books": [
          "Robot Dreams"
        ],
        "details": {
          "age": 35,
          "pi": 3.1415
        }
      }"""

  Source Edit

    proc to[T](node: JsonNode; t: typedesc[T]): T

  Unmarshals the specified node into the object type specified.

  Known limitations:

  - Heterogeneous arrays are not supported.
  - Sets in object variants are not supported.
  - Not nil annotations are not supported.

  Example:

      let jsonNode = parseJson("""
            {
              "person": {
                "name": "Nimmer",
                "age": 21
              },
              "list": [1, 2, 3, 4]
            }
          """)

      type
        Person = object
          name: string
          age: int

        Data = object
          person: Person
          list: seq[int]

      var data = to(jsonNode, Data)
      doAssert data.person.name == "Nimmer"
      doAssert data.person.age == 21
      doAssert data.list == @[1, 2, 3, 4]

  Source Edit

    proc toUgly(result: var string; node: JsonNode) {....raises: [], tags: [],
        forbids: [].}

  Converts `node` to its JSON Representation, without regard for human readability. Meant to improve `$` string conversion performance.

  JSON representation is stored in the passed `result`

  This provides higher efficiency than the `pretty` procedure as it does not attempt to format the resulting JSON to make it human readable.

  Source Edit

    proc `{}`(node: JsonNode; index: varargs[int]): JsonNode {....raises: [], tags: [],
        forbids: [].}

  Traverses the node and gets the given value. If any of the indexes do not exist, returns `nil`. Also returns `nil` if one of the intermediate data structures is not an array. Source Edit

    proc `{}`(node: JsonNode; key: string): JsonNode {....raises: [], tags: [],
        forbids: [].}

  Gets a field from a `node`. If `node` is nil or not an object or value at `key` does not exist, returns nil Source Edit

    proc `{}`(node: JsonNode; keys: varargs[string]): JsonNode {....raises: [],
        tags: [], forbids: [].}

  Traverses the node and gets the given value. If any of the keys do not exist, returns `nil`. Also returns `nil` if one of the intermediate data structures is not an object.

  This proc can be used to create tree structures on the fly (sometimes called autovivification):

  Example:

      var myjson = %* {"parent": {"child": {"grandchild": 1}}}
      doAssert myjson{"parent", "child", "grandchild"} == newJInt(1)

  Source Edit

    proc `{}=`(node: JsonNode; keys: varargs[string]; value: JsonNode) {.
        ...raises: [KeyError], tags: [], forbids: [].}

  Traverses the node and tries to set the value at the given location to `value`. If any of the keys are missing, they are added. Source Edit

# Iterators

    iterator items(node: JsonNode): JsonNode {....raises: [], tags: [], forbids: [].}

  Iterator for the items of `node`. `node` has to be a JArray. Source Edit

    iterator keys(node: JsonNode): string {....raises: [], tags: [], forbids: [].}

  Iterator for the keys in `node`. `node` has to be a JObject. Source Edit

    iterator mitems(node: var JsonNode): var JsonNode {....raises: [], tags: [],
        forbids: [].}

  Iterator for the items of `node`. `node` has to be a JArray. Items can be modified. Source Edit

    iterator mpairs(node: var JsonNode): tuple[key: string, val: var JsonNode] {.
        ...raises: [], tags: [], forbids: [].}

  Iterator for the child elements of `node`. `node` has to be a JObject. Values can be modified Source Edit

    iterator pairs(node: JsonNode): tuple[key: string, val: JsonNode] {....raises: [],
        tags: [], forbids: [].}

  Iterator for the child elements of `node`. `node` has to be a JObject. Source Edit

    iterator parseJsonFragments(s: Stream; filename: string = "";
                                rawIntegers = false; rawFloats = false): JsonNode {.
        ...raises: [IOError, OSError, IOError, OSError, JsonParsingError, ValueError],
        tags: [ReadIOEffect, WriteIOEffect], forbids: [].}

  Parses from a stream `s` into `JsonNodes`. `filename` is only needed for nice error messages. The JSON fragments are separated by whitespace. This can be substantially faster than the comparable loop `for x in splitWhitespace(s): yield parseJson(x)`. This closes the stream `s` after it's done. If `rawIntegers` is true, integer literals will not be converted to a `JInt` field but kept as raw numbers via `JString`. If `rawFloats` is true, floating point literals will not be converted to a `JFloat` field but kept as raw numbers via `JString`. Source Edit

# Macros

    macro `%*`(x: untyped): untyped

  Convert an expression to a JsonNode directly, without having to specify `%` for every element. Source Edit

    macro isRefSkipDistinct(arg: typed): untyped

  internal only, do not use Source Edit

# Templates

    template `%`(j: JsonNode): JsonNode

  Source Edit

# Exports

$, $, $, $, $, $, JsonEventKind, JsonError, JsonParser, JsonKindError, open, open, open, open, open, close, close, close, close, str, getInt, getFloat, kind, kind, getColumn, getLine, getFilename, errorMsg, errorMsgExpected, next, JsonParsingError, raiseParseErr, nimIdentNormalize