zwilias / json-decode-exploration / Json.Decode.Exploration

This package presents a somewhat experimental approach to JSON decoding. Its API looks very much like the core Json.Decode API. The major differences are the final decodeString and decodeValue functions, which return a DecodeResult a.

Decoding with this library can result in one of 4 possible outcomes:

Both the Errors and Warnings types are (mostly) machine readable: they are implemented as a recursive datastructure that points to the location of the error in the input json, producing information about what went wrong (i.e. "what was the expected type, and what did the actual value look like").

Further, this library also adds a few extra Decoders that help with making assertions about the structure of the JSON while decoding.

For convenience, this library also includes a Json.Decode.Exploration.Pipeline module which is largely a copy of NoRedInk/elm-decode-pipeline.

Running a Decoder

Runing a Decoder works largely the same way as it does in the familiar core library. There is one serious caveat, however:

This library does not allowing decoding non-serializable JS values.

This means that trying to use this library to decode a Value which contains non-serializable information like functions will not work. It will, however, result in a BadJson result.

Trying to use this library on cyclic values (like HTML events) is quite likely to blow up completely. Don't try this, except maybe at home.

decodeString : Decoder a -> String -> DecodeResult a

Decode a JSON string. If the string isn't valid JSON, this will fail with a BadJson result.

decodeValue : Decoder a -> Value -> DecodeResult a

Run a Decoder on a Value.

Note that this may still fail with a BadJson if there are non-JSON compatible values in the provided Value. In particular, don't attempt to use this library when decoding Events - it will blow up. Badly.

strict : DecodeResult a -> Result Errors a

Interpret a decode result in a strict way, lifting warnings to errors.

import List.Nonempty as Nonempty
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" ["foo"] """
    |> decodeString isArray
    |> strict
--> (Here <| Failure "Unused index: 0" Nothing)
-->   |> (Err << Nonempty.fromElement)


""" null """
    |> decodeString (null "cool")
    |> strict
--> Ok "cool"

""" { "foo": "bar" } """
    |> decodeString isObject
    |> strict
--> (Here <| Failure "Unused field: foo" Nothing)
-->   |> (Err << Nonempty.fromElement)

Bad JSON will also result in a Failure, with Nothing as the actual value:

""" foobar """
    |> decodeString string
    |> strict
--> (Here <| Failure "Invalid JSON" Nothing)
-->   |> (Err << Nonempty.fromElement)

Errors will still be errors, of course.

""" null """
    |> decodeString string
    |> strict
--> (Here <| Expected TString Encode.null)
-->   |> (Err << Nonempty.fromElement)


type DecodeResult a
    = BadJson
    | Errors Errors
    | WithWarnings Warnings a
    | Success a

Decoding can have 4 different outcomes:


type alias Value =
Json.Decode.Value

A simple type alias for Json.Decode.Value.

Dealing with warnings and errors


type alias Errors =
List.Nonempty.Nonempty (Located Error)

Decoding may fail with 1 or more errors, so Errors is a Nonempty of errors.


type Error
    = BadOneOf (List Errors)
    | Expected ExpectedType Value
    | Failure String (Maybe Value)

The most basic kind of an Error is Failure, which comes annotated with a string describing the failure, and the JSON Value that was encountered instead.

The other cases describe the "path" to where the error occurred.

errorsToString : Errors -> String

Stringifies errors to a human readable string.


type alias Warnings =
List.Nonempty.Nonempty (Located Warning)

Decoding may generate warnings. In case the result is a WithWarnings, you will have 1 or more warnings, as a Nonempty list.


type Warning
    = UnusedValue Value
    | UnusedField String
    | UnusedIndex Basics.Int
    | Warning String Value

Like with errors, the most basic warning is an unused value. The other cases describe the path to the warnings.

warningsToString : Warnings -> String

Stringifies warnings to a human readable string.


type ExpectedType
    = TString
    | TBool
    | TInt
    | TNumber
    | TArray
    | TObject
    | TArrayIndex Basics.Int
    | TObjectField String
    | TNull

An enumeration of the different types that could be expected by a decoder.

Primitives


type Decoder a

Kind of the core idea of this library. Think of it as a piece of data that describes how to read and transform JSON. You can use decodeString and decodeValue to actually execute a decoder on JSON.

string : Decoder String

Decode a string.

import List.Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" "hello world" """
    |> decodeString string
--> Success "hello world"


""" 123 """
    |> decodeString string
--> Errors (Nonempty (Here <| Expected TString (Encode.int 123)) [])

bool : Decoder Basics.Bool

Decode a boolean value.

""" [ true, false ] """
    |> decodeString (list bool)
--> Success [ True, False ]

int : Decoder Basics.Int

Decode a number into an Int.

import List.Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" 123 """
    |> decodeString int
--> Success 123


""" 0.1 """
    |> decodeString int
--> Errors <|
-->   Nonempty
-->     (Here <| Expected TInt (Encode.float 0.1))
-->     []

float : Decoder Basics.Float

Decode a number into a Float.

import List.Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" 12.34 """
    |> decodeString float
--> Success 12.34


""" 12 """
    |> decodeString float
--> Success 12


""" null """
    |> decodeString float
--> Errors (Nonempty (Here <| Expected TNumber Encode.null) [])

Data Structures

nullable : Decoder a -> Decoder (Maybe a)

Decodes successfully and wraps with a Just. If the values is null succeeds with Nothing.

""" [ { "foo": "bar" }, { "foo": null } ] """
    |> decodeString (list <| field "foo" <| nullable string)
--> Success [ Just "bar", Nothing ]

list : Decoder a -> Decoder (List a)

Decode a list of values, decoding each entry with the provided decoder.

import List.Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" [ "foo", "bar" ] """
    |> decodeString (list string)
--> Success [ "foo", "bar" ]


""" [ "foo", null ] """
    |> decodeString (list string)
--> Errors <|
-->   Nonempty
-->     (AtIndex 1 <|
-->       Nonempty (Here <| Expected TString Encode.null) []
-->     )
-->     []

array : Decoder a -> Decoder (Array a)

Convenience function. Decode a JSON array into an Elm Array.

import Array

""" [ 1, 2, 3 ] """
    |> decodeString (array int)
--> Success <| Array.fromList [ 1, 2, 3 ]

dict : Decoder v -> Decoder (Dict String v)

Convenience function. Decode a JSON object into an Elm Dict String.

import Dict


""" { "foo": "bar", "bar": "hi there" } """
    |> decodeString (dict string)
--> Success <| Dict.fromList
-->   [ ( "bar", "hi there" )
-->   , ( "foo", "bar" )
-->   ]

keyValuePairs : Decoder a -> Decoder (List ( String, a ))

Decode a JSON object into a list of key-value pairs. The decoder you provide will be used to decode the values.

""" { "foo": "bar", "hello": "world" } """
    |> decodeString (keyValuePairs string)
--> Success [ ( "foo", "bar" ), ( "hello", "world" ) ]

Structural ascertainments

isObject : Decoder ()

A Decoder to ascertain that a JSON value is in fact, a JSON object.

Using this decoder marks the object itself as used, without touching any of its children. It is, as such, fairly well behaved.

import List.Nonempty as Nonempty
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" { } """
    |> decodeString isObject
--> Success ()


""" [] """
    |> decodeString isObject
--> Errors <| Nonempty.fromElement <| Here <| Expected TObject (Encode.list identity [])

isArray : Decoder ()

Similar to isObject, a decoder to ascertain that a JSON value is a JSON array.

import List.Nonempty as Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" [] """
    |> decodeString isArray
--> Success ()


""" [ "foo" ] """
    |> decodeString isArray
--> WithWarnings (Nonempty (Here (UnusedIndex 0)) []) ()


""" null """
    |> decodeString isArray
--> Errors <| Nonempty.fromElement <| Here <| Expected TArray Encode.null

Object Primitives

field : String -> Decoder a -> Decoder a

Decode the content of a field using a provided decoder.

import List.Nonempty as Nonempty
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode

""" { "foo": "bar" } """
    |> decodeString (field "foo" string)
--> Success "bar"


""" [ { "foo": "bar" }, { "foo": "baz", "hello": "world" } ] """
    |> decodeString (list (field "foo" string))
--> WithWarnings expectedWarnings [ "bar", "baz" ]


expectedWarnings : Warnings
expectedWarnings =
    UnusedField "hello"
        |> Here
        |> Nonempty.fromElement
        |> AtIndex 1
        |> Nonempty.fromElement

at : List String -> Decoder a -> Decoder a

Decodes a value at a certain path, using a provided decoder. Essentially, writing at [ "a", "b", "c" ] string is sugar over writing field "a" (field "b" (field "c" string))}.

""" { "a": { "b": { "c": "hi there" } } } """
    |> decodeString (at [ "a", "b", "c" ] string)
--> Success "hi there"

index : Basics.Int -> Decoder a -> Decoder a

Decode a specific index using a specified Decoder.

import List.Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" [ "hello", 123 ] """
    |> decodeString (map2 Tuple.pair (index 0 string) (index 1 int))
--> Success ( "hello", 123 )


""" [ "hello", "there" ] """
    |> decodeString (index 1 string)
--> WithWarnings (Nonempty (AtIndex 0 (Nonempty (Here (UnusedValue (Encode.string "hello"))) [])) [])
-->   "there"

Inconsistent Structure

maybe : Decoder a -> Decoder (Maybe a)

Decodes successfully and wraps with a Just, handling failure by succeeding with Nothing.

import List.Nonempty as Nonempty
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" [ "foo", 12 ] """
    |> decodeString (list <| maybe string)
--> WithWarnings expectedWarnings [ Just "foo", Nothing ]


expectedWarnings : Warnings
expectedWarnings =
    UnusedValue (Encode.int 12)
        |> Here
        |> Nonempty.fromElement
        |> AtIndex 1
        |> Nonempty.fromElement

oneOf : List (Decoder a) -> Decoder a

Tries a bunch of decoders. The first one to not fail will be the one used.

If all fail, the errors are collected into a BadOneOf.

import List.Nonempty as Nonempty
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode

""" [ 12, "whatever" ] """
    |> decodeString (list <| oneOf [ map String.fromInt int, string ])
--> Success [ "12", "whatever" ]


""" null """
    |> decodeString (oneOf [ string, map String.fromInt int ])
--> Errors <| Nonempty.fromElement <| Here <| BadOneOf
-->   [ Nonempty.fromElement <| Here <| Expected TString Encode.null
-->   , Nonempty.fromElement <| Here <| Expected TInt Encode.null
-->   ]

Fancy Decoding

lazy : (() -> Decoder a) -> Decoder a

Required when using (mutually) recursive decoders.

value : Decoder Value

Extract a piece without actually decoding it.

If a structure is decoded as a value, everything in the structure will be considered as having been used and will not appear in UnusedValue warnings.

import Json.Encode as Encode


""" [ 123, "world" ] """
    |> decodeString value
--> Success (Encode.list identity [ Encode.int 123, Encode.string "world" ])

null : a -> Decoder a

Decode a null and succeed with some value.

""" null """
    |> decodeString (null "it was null")
--> Success "it was null"

Note that undefined and null are not the same thing. This cannot be used to verify that a field is missing, only that it is explicitly set to null.

""" { "foo": null } """
    |> decodeString (field "foo" (null ()))
--> Success ()


import List.Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" { } """
    |> decodeString (field "foo" (null ()))
--> Errors <|
-->   Nonempty
-->     (Here <| Expected (TObjectField "foo") (Encode.object []))
-->     []

check : Decoder a -> a -> Decoder b -> Decoder b

Useful for checking a value in the JSON matches the value you expect it to have. If it does, succeeds with the second decoder. If it doesn't it fails.

This can be used to decode union types:

type Pet = Cat | Dog | Rabbit

petDecoder : Decoder Pet
petDecoder =
    oneOf
        [ check string "cat" <| succeed Cat
        , check string "dog" <| succeed Dog
        , check string "rabbit" <| succeed Rabbit
        ]

""" [ "dog", "rabbit", "cat" ] """
    |> decodeString (list petDecoder)
--> Success [ Dog, Rabbit, Cat ]

succeed : a -> Decoder a

A decoder that will ignore the actual JSON and succeed with the provided value. Note that this may still fail when dealing with an invalid JSON string.

If a value in the JSON ends up being ignored because of this, this will cause a warning.

import List.Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode


""" null """
    |> decodeString (value |> andThen (\_ -> succeed "hello world"))
--> Success "hello world"


""" null """
    |> decodeString (succeed "hello world")
--> WithWarnings
-->     (Nonempty (Here <| UnusedValue Encode.null) [])
-->     "hello world"


""" foo """
    |> decodeString (succeed "hello world")
--> BadJson

fail : String -> Decoder a

Ignore the json and fail with a provided message.

import List.Nonempty exposing (Nonempty(..))
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode

""" "hello" """
    |> decodeString (fail "failure")
--> Errors (Nonempty (Here <| Failure "failure" (Just <| Encode.string "hello")) [])

warn : String -> Decoder a -> Decoder a

Add a warning to the result of a decoder.

For example, imagine we are upgrading some internal JSON format. We might add a temporary workaround for backwards compatibility. By adding a warning to the decoder, we can flag these or print them during development.

import List.Nonempty as Nonempty
import Json.Decode.Exploration.Located exposing (Located(..))
import Json.Encode as Encode

decoder : Decoder (List Int)
decoder =
    oneOf
        [ list int
        , int |> map List.singleton |> warn "Converted to list"
        ]

expectedWarnings : Warnings
expectedWarnings =
    Warning "Converted to list" (Encode.int 123)
        |> Here
        |> Nonempty.fromElement

""" 123 """
   |>  decodeString decoder
--> WithWarnings expectedWarnings [ 123 ]

Note that warnings added to a failing decoder won't show up.

""" null """
    |> decodeString (warn "this might be null" int)
--> Errors (Nonempty.fromElement (Here <| Expected TInt Encode.null))

andThen : (a -> Decoder b) -> Decoder a -> Decoder b

Chain decoders where one decoder depends on the value of another decoder.

Mapping

Note: If you run out of map functions, take a look at the pipeline module which makes it easier to handle large objects.

map : (a -> b) -> Decoder a -> Decoder b

Useful for transforming decoders.

""" "foo" """
    |> decodeString (map String.toUpper string)
--> Success "FOO"

map2 : (a -> b -> c) -> Decoder a -> Decoder b -> Decoder c

Combine 2 decoders.

map3 : (a -> b -> c -> d) -> Decoder a -> Decoder b -> Decoder c -> Decoder d

Combine 3 decoders.

map4 : (a -> b -> c -> d -> e) -> Decoder a -> Decoder b -> Decoder c -> Decoder d -> Decoder e

Combine 4 decoders.

map5 : (a -> b -> c -> d -> e -> f) -> Decoder a -> Decoder b -> Decoder c -> Decoder d -> Decoder e -> Decoder f

Combine 5 decoders.

map6 : (a -> b -> c -> d -> e -> f -> g) -> Decoder a -> Decoder b -> Decoder c -> Decoder d -> Decoder e -> Decoder f -> Decoder g

Combine 6 decoders.

map7 : (a -> b -> c -> d -> e -> f -> g -> h) -> Decoder a -> Decoder b -> Decoder c -> Decoder d -> Decoder e -> Decoder f -> Decoder g -> Decoder h

Combine 7 decoders.

map8 : (a -> b -> c -> d -> e -> f -> g -> h -> i) -> Decoder a -> Decoder b -> Decoder c -> Decoder d -> Decoder e -> Decoder f -> Decoder g -> Decoder h -> Decoder i

Combine 8 decoders.

andMap : Decoder a -> Decoder (a -> b) -> Decoder b

Decode an argument and provide it to a function in a decoder.

decoder : Decoder String
decoder =
    succeed (String.repeat)
        |> andMap (field "count" int)
        |> andMap (field "val" string)


""" { "val": "hi", "count": 3 } """
    |> decodeString decoder
--> Success "hihihi"

Stripping

stripString : Decoder a -> String -> Result Errors String

Reduce JSON down to what is needed to ensure decoding succeeds.

This is useful if you are doing a build step where you want to strip down your JSON data to the minimal amount needed by your Decoder. Because of the purity of Elm's functions, you can safely strip out unused data assuming that you use the exact same Decoder to strip the JSON as you use when you re-run it against the stripped JSON. Be sure your Decoder doesn't depend on any parameters which will vary between at build-time and run-time or you will lose this guarantee.

You can also take a look at the validateStrip test cases in this test module.

import Json.Decode.Exploration as Decode exposing (Decoder)


jsonValue : String
jsonValue =
    """
    {
      "topLevelUsed": 123,
      "partiallyUsed": [
        {"used": "Hi! This is read by Decode.index so this Object will be included.", "unused": "Please ignore me"},
        {"used": "This whole Object is stripped out because it isn't read by Decode.index!", "unused": "This field is always ignored"}
        ],
      "unused": 456,
      "nestedUnused": "This gets stripped out of the final JSON"
    }
    """

type alias Record = { num : Int, string : String }

decoder : Decoder Record
decoder =
    Decode.map2 Record
        (Decode.field "topLevelUsed" Decode.int)
        (Decode.field "partiallyUsed"
             (Decode.index 0 (Decode.field "used" Decode.string))
        )


Decode.stripString decoder jsonValue
--> Ok """{"topLevelUsed":123,"partiallyUsed":[{"used":"Hi! This is read by Decode.index so this Object will be included."}]}"""

stripValue : Decoder a -> Value -> Result Errors Value

Reduce JSON down to what is needed to ensure decoding succeeds.

See the stripString docs for more details and an example.