Janiczek / architecture-test / ArchitectureTest

A library for fuzz testing TEA models by simulating user interactions (using fuzzed lists of Msgs).

This means:

You get the nice property of fuzz tests that this kind of testing will show you the minimal Msg sequence to provoke a bug.

The app in doc examples below is:

{ model = ConstantModel model
, update = UpdateWithoutCmds update
, msgFuzzer =
    Fuzz.oneOf
        [ Fuzz.int 0 50 |> Fuzz.map AddCoins
        , Fuzz.constant Cancel
        , Fuzz.constant Buy
        , Fuzz.constant TakeProduct
        ]
}

For a complete code example, see the examples/ directory of the repo.

Tests

msgTest : String -> TestedApp model msg -> Fuzzer msg -> (model -> msg -> model -> Expectation) -> Test

Tests that a condition holds for a randomly generated Model after that specific Msg is applied.

The process is as follows:

  1. get an initial Model (based on TestedApp)

  2. stuff it and a list of random Msgs into update to get a random Model

  3. create a Msg we will test

  4. stuff it and the random Model into update to get the final Model

  5. run your test function on the three values (random Model, tested Msg, final Model)

cancelReturnsMoney : Test
cancelReturnsMoney =
    msgTest "Cancelling returns all input money"
        app
        (Fuzz.constant Cancel)
    <|
        \_ _ finalModel -> finalModel.currentCoins |> Expect.equal 0

The test function's arguments are:

random Model (before the tested Msg) -> tested Msg -> final Model

msgTestWithPrecondition : String -> TestedApp model msg -> Fuzzer msg -> (model -> Basics.Bool) -> (model -> msg -> model -> Expectation) -> Test

Similar to msgTest, but only gets run when a precondition holds.

buyingAbovePriceVendsProduct : Test
buyingAbovePriceVendsProduct =
    msgTestWithPrecondition "Buying above price vends the product"
        app
        (Fuzz.constant Buy)
        (\model -> model.currentCoins >= model.productPrice)
    <|
        \_ _ finalModel ->
            finalModel.isProductVended
                |> Expect.true "Product should be vended after trying to buy with enough money"

The precondition acts on the "model before specific Msg" (see msgTest docs).

invariantTest : String -> TestedApp model msg -> (model -> List msg -> model -> Expectation) -> Test

Tests that a property holds no matter what Msgs we applied.

priceConstant : Test
priceConstant =
    invariantTest "Price is constant"
        app
    <|
        \initModel _ finalModel ->
            finalModel.productPrice
                |> Expect.equal initModel.productPrice

The test function's arguments are:

init model -> random Msgs -> final model

Types


type alias TestedApp model msg =
{ model : TestedModel model
, update : TestedUpdate model msg
, msgFuzzer : Fuzzer msg
, msgToString : msg -> String
, modelToString : model -> String 
}

All of these "architecture tests" are going to have something in common: Model, update function and Msgs.

Note that for some tests you can eg. make the Msg fuzzer prefer certain Msgs more if you need to test them more extensively.


type TestedModel model
    = ConstantModel model
    | FuzzedModel (Fuzzer model)
    | OneOfModels (List model)

The strategy for choosing an init model to which the Msgs will be applied.


type TestedUpdate model msg
    = UpdateWithoutCmds (msg -> model -> model)
    | NormalUpdate (msg -> model -> ( model, Platform.Cmd.Cmd msg ))

Main applications can be of two types: those without Cmds and normal (Cmds present).

For custom update functions returning eg. triples etc., just use UpdateWithoutCmds with a function that returns just the model part of the result:

update : Msg -> Model -> {model : Model, cmd : Cmd Msg, outMsg : OutMsg}

UpdateWithoutCmds (\msg model -> update msg model |> .model)

Fuzzers

modelFuzzer : TestedApp model msg -> Fuzzer model

Fuzz the model, always starting with initial Models and then doing consecutive update calls with fuzzed Msgs.

Guarantees the final model is reachable using your Msgs and thus "makes sense."