This is an "experts only" module that exposes functions needed to run and
display tests. A typical user will use an existing runner library for Node or
the browser, which is implemented using this interface. A list of these runners
can be found in the README
.
{ run : () -> List Expectation
, labels : List String
}
A function which, when evaluated, produces a list of expectations. Also a list of labels which apply to this outcome.
Test Runners which have had seeds distributed to them, and which are now either invalid or are ready to run. Seeded runners include some metadata:
Invalid
runners had a problem (e.g. two sibling tests had the same description) making them un-runnable.Only
runners can be run, but Test.only
was used somewhere, so ultimately they will lead to a failed test run even if each test that gets run passes.Skipping
runners can be run, but Test.skip
was used somewhere, so ultimately they will lead to a failed test run even if each test that gets run passes.Plain
runners are ready to run, and have none of these issues.fromTest : Basics.Int -> Random.Seed -> Test -> SeededRunners
Convert a Test
into SeededRunners
.
In order to run any fuzz tests that the Test
may have, it requires a default run count as well
as an initial Random.Seed
. 100
is a good run count. To obtain a good random seed, pass a
random 32-bit integer to Random.initialSeed
. You can obtain such an integer by running
Math.floor(Math.random()*0xFFFFFFFF)
in Node. It's typically fine to hard-code this value into
your Elm code; it's easy and makes your tests reproducible.
getFailureReason : Expectation -> Maybe { given : Maybe String, description : String, reason : Failure.Reason }
Return Nothing
if the given Expectation
is a pass
.
If it is a fail
, return a record containing the expectation
description, the Reason
the test failed, and the given inputs if
it was a fuzz test. (If it was not a fuzz test, the record's given
field
will be Nothing
).
For example:
getFailureReason (Expect.equal 1 2)
-- Just { reason = Equal 1 2, description = "Expect.equal", given = Nothing }
getFailureReason (Expect.equal 1 1)
-- Nothing
isTodo : Expectation -> Basics.Bool
Determine if an expectation was created by a call to Test.todo
. Runners
may treat these tests differently in their output.
getDistributionReport : Expectation -> Test.Distribution.DistributionReport
Returns a DistributionReport
computed for a given test.
formatLabels : (String -> format) -> (String -> format) -> List String -> List format
A standard way to format descriptions and test labels, to keep things consistent across test runner implementations.
The HTML, Node, String, and Log runners all use this.
What it does:
Example:
[ "the actual test that failed"
, "nested description failure"
, "top-level description failure"
]
|> formatLabels ((++) "↓ ") ((++) "✗ ")
{-
[ "↓ top-level description failure"
, "↓ nested description failure"
, "✗ the actual test that failed"
]
-}
These functions give you the ability to run fuzzers separate of running fuzz tests.
A Simplifiable a
is an opaque type that allows you to obtain a value of type
a
that is simpler than the one you've previously obtained.
fuzz : Fuzzer a -> Random.Generator (Result String ( a, Simplifiable a ))
Given a fuzzer, return a random generator to produce a value and a Simplifiable. The value is what a fuzz test would have received as input.
Note that fuzzers aren't generated to succeed, which is why this function returns a Result. The String inside the Err case will contain a failure reason.
simplify : (a -> Expectation) -> ( a, Simplifiable a ) -> Maybe ( a, Simplifiable a )
Given a Simplifiable, simplify the value further. Pass your test function to drive the simplification process: if a simplified value passes the test, it will be discarded. In this sense, you will get the simplest value that still fails your test.