This is a library of fuzzers you can use to supply values to your fuzz tests. You can typically pick out which ones you need according to their types.
A Fuzzer a
knows how to create values of type a
. It can create them randomly,
so that your test's expectations are run against many values. Fuzzers will often
generate edge cases likely to find bugs. If the fuzzer can make your test fail,
the test runner also knows how to "simplify" that failing input into more minimal
examples, some of which might also cause the tests to fail. In this way, fuzzers
can usually find the simplest input that reproduces a bug.
Internal.Fuzzer a
The representation of fuzzers is opaque. Conceptually, a Fuzzer a
consists
of a way to randomly generate values of type a
in a way allowing the test runner
to simplify those values.
examples : Basics.Int -> Fuzzer a -> List a
Generate a few example values from the fuzzer.
Useful in REPL:
> import Fuzz
> Fuzz.examples 20 (Fuzz.intRange 20 50)
[42,45,32,26,33,29,41,45,23,45,34,23,22,42,29,27,41,43,30,50]
: List Int
Uses the first argument as the seed as well as the count of examples to generate.
Will return an empty list in case of rejection.
labelExamples : Basics.Int -> List ( String, a -> Basics.Bool ) -> Fuzzer a -> List ( List String, Maybe a )
Show examples of values satisfying given classification predicates (see
also Test.reportDistribution
and
Test.expectDistribution
).
Generates a given number of values and classifies them based on the predicates.
Uses the first argument as the seed as well as the count of examples to generate.
This function will always return all the given "base" labels, even if no examples of them could be found:
Fuzz.labelExamples 100
[ ( "Lower boundary (1)", \n -> n == 1 )
, ( "Upper boundary (20)", \n -> n == 20 )
, ( "In the middle (2..19)", \n -> n > 1 && n < 20 )
, ( "Outside boundaries??", \n -> n < 1 || n > 20 )
]
(Fuzz.intRange 1 20)
-->
[ ( [ "Lower boundary (1)" ], Just 1 )
, ( [ "Upper boundary (20)" ], Just 20 )
, ( [ "In the middle (2..19)" ], Just 5 )
, ( [ "Outside boundaries??" ], Nothing )
]
In case of predicate overlap (eg. something is both green and big) this function will also return all the found combinations:
Fuzz.labelExamples 100
[ ( "fizz", \n -> (n |> modBy 3) == 0 )
, ( "buzz", \n -> (n |> modBy 5) == 0 )
]
(Fuzz.intRange 1 20)
-->
[ ( [ "fizz" ], Just 3 )
, ( [ "buzz" ], Just 10 )
, ( [ "fizz, buzz" ], Just 15 )
]
int : Fuzzer Basics.Int
A fuzzer for int values. It will never produce NaN
, Infinity
, or
-Infinity
.
This fuzzer will generate values in the range Random.minInt .. Random.maxInt
.
intRange : Basics.Int -> Basics.Int -> Fuzzer Basics.Int
A fuzzer for int values between a given minimum and maximum value, inclusive. Shrunk values will also be within the range.
uniformInt : Basics.Int -> Fuzzer Basics.Int
Draw an integer between 0 and n inclusive.
Will simplify towards 0, but draws uniformly over the whole range.
Max supported value is 2^32 - 1.
intAtLeast : Basics.Int -> Fuzzer Basics.Int
A fuzzer that will generate values in range n..2^32-1.
intAtMost : Basics.Int -> Fuzzer Basics.Int
A fuzzer that will generate values in range -(2^32-1)..n.
float : Fuzzer Basics.Float
A fuzzer for float values.
Will prefer integer values, nice fractions and positive numbers over the rest.
Will occasionally try infinities and NaN. If you don't want to generate these,
use Fuzz.niceFloat
.
niceFloat : Fuzzer Basics.Float
A fuzzer for float values.
Will prefer integer values, nice fractions and positive numbers over the rest.
Will never try infinities or NaN.
percentage : Fuzzer Basics.Float
A fuzzer for percentage values. Generates random floats between 0.0
inclusive and 1.0
exclusive, in an uniform fashion.
Will occasionally try the boundaries.
Doesn't shrink to nice values like Fuzz.float
does; shrinks towards
zero.
floatRange : Basics.Float -> Basics.Float -> Fuzzer Basics.Float
A fuzzer for float values within between a given minimum and maximum (inclusive).
Shrunken values will also be within the range.
floatAtLeast : Basics.Float -> Fuzzer Basics.Float
Fuzzer generating floats in range n..Infinity
.
The positive part of the range will shrink nicely, the negative part will shrink uniformly.
The fuzzer will occasionally try the minimum, 0 (if in range) and Infinity.
floatAtMost : Basics.Float -> Fuzzer Basics.Float
Fuzzer generating floats in range -Infinity..n
.
The negative part of the range will shrink nicely, the positive part will shrink uniformly.
The fuzzer will occasionally try the maximum, 0 (if in range) and -Infinity.
char : Fuzzer Char
A fuzzer for arbitrary Unicode char values.
Avoids surrogate pairs or their components (0xD800..0xDFFF
).
Will prefer ASCII characters, whitespace, and some examples known to cause trouble, like combining diacritics marks and emojis.
asciiChar : Fuzzer Char
A fuzzer for simple ASCII char values (range 32..126). Skips control characters and the extended character set.
For more serious char fuzzing look at Fuzz.char
which generates the
whole Unicode range.
string : Fuzzer String
Generates random unicode strings of up to 10 characters.
stringOfLength : Basics.Int -> Fuzzer String
Generates random unicode strings of a given length.
Note that some unicode characters have String.length
of 2. This fuzzer will
make sure the String.length
of the returned string is equal to the wanted
length, even if it will mean there are less characters. If you instead want it
to give N characters even if their String.length
will be above N, you can use
Fuzz.listOfLength n Fuzz.char
|> Fuzz.map String.fromList
stringOfLengthBetween : Basics.Int -> Basics.Int -> Fuzzer String
Generates random unicode strings of length between the given limits.
Note that some unicode characters have String.length
of 2. This fuzzer will
make sure the String.length
of the returned string is equal to the wanted
length, even if it will mean there are less characters. If you instead want it
to give between MIN and MAX characters even if their String.length
will be
above MAX, you can use
Fuzz.listOfLengthBetween min max Fuzz.char
|> Fuzz.map String.fromList
asciiString : Fuzzer String
Generates random ASCII strings of up to 10 characters.
asciiStringOfLength : Basics.Int -> Fuzzer String
Generates random ASCII strings of a given length.
asciiStringOfLengthBetween : Basics.Int -> Basics.Int -> Fuzzer String
Generates random ASCII strings of length between the given limits.
pair : Fuzzer a -> Fuzzer b -> Fuzzer ( a, b )
Create a fuzzer of pairs from two fuzzers.
triple : Fuzzer a -> Fuzzer b -> Fuzzer c -> Fuzzer ( a, b, c )
Create a fuzzer of triples from three fuzzers.
list : Fuzzer a -> Fuzzer (List a)
Given a fuzzer of a type, create a fuzzer of a list of that type. Generates random lists of varying length, up to 32 elements.
listOfLength : Basics.Int -> Fuzzer a -> Fuzzer (List a)
Given a fuzzer of a type, create a fuzzer of a list of that type. Generates random lists of exactly the specified length.
listOfLengthBetween : Basics.Int -> Basics.Int -> Fuzzer a -> Fuzzer (List a)
Given a fuzzer of a type, create a fuzzer of a list of that type. Generates random lists of length between the two given integers.
shuffledList : List a -> Fuzzer (List a)
A fuzzer that shuffles the given list.
array : Fuzzer a -> Fuzzer (Array a)
Given a fuzzer of a type, create a fuzzer of an array of that type. Generates random arrays of varying length, favoring shorter arrays.
maybe : Fuzzer a -> Fuzzer (Maybe a)
Given a fuzzer of a type, create a fuzzer of a maybe for that type.
result : Fuzzer error -> Fuzzer value -> Fuzzer (Result error value)
Given fuzzers for an error type and a success type, create a fuzzer for a result.
bool : Fuzzer Basics.Bool
A fuzzer for boolean values. It's useful when building up fuzzers of complex types that contain a boolean somewhere.
We recommend against writing tests fuzzing over booleans. Write a unit test for the true and false cases explicitly.
Simplifies in order False < True
.
unit : Fuzzer ()
A fuzzer for the unit value. Unit is a type with only one value, commonly used as a placeholder.
order : Fuzzer Basics.Order
A fuzzer for order values.
Simplifies in order LT < EQ < GT
.
weightedBool : Basics.Float -> Fuzzer Basics.Bool
A fuzzer for boolean values, generating True with the given probability (0.0 = always False, 1.0 = always True).
Probabilities outside the 0..1
range will be clamped to 0..1
.
Simplifies towards False (if not prevented to do that by using probability >= 1).
oneOf : List (Fuzzer a) -> Fuzzer a
Choose one of the given fuzzers at random. Each fuzzer has an equal chance
of being chosen; to customize the probabilities, use Fuzz.frequency
.
This fuzzer will simplify towards the fuzzers earlier in the list (each of which will also apply its own way to simplify the values).
Fuzz.oneOf
[ Fuzz.intRange 0 3
, Fuzz.intRange 7 9
]
oneOfValues : List a -> Fuzzer a
Choose one of the given values at random. Each value has an equal chance
of being chosen; to customize the probabilities, use
Fuzz.frequencyValues
.
This fuzzer will simplify towards the values earlier in the list.
Fuzz.oneOfValues
[ 999
, -42
]
frequency : List ( Basics.Float, Fuzzer a ) -> Fuzzer a
Create a new Fuzzer
by providing a list of probabilistic weights to use
with other fuzzers.
For example, to create a Fuzzer
that has a 1/4 chance of generating an int
between -1 and -100, and a 3/4 chance of generating one between 1 and 100,
you could do this:
Fuzz.frequency
[ ( 1, Fuzz.intRange -100 -1 )
, ( 3, Fuzz.intRange 1 100 )
]
This fuzzer will simplify towards the fuzzers earlier in the list (each of which will also apply its own way to simplify the values).
There are a few circumstances in which this function will return an invalid fuzzer, which causes it to fail any test that uses it:
Be careful recursively using this fuzzer in its arguments. Often using
Fuzz.map
is a better way to do what you want. If you are fuzzing a
tree-like data structure, you should include a depth limit so to avoid infinite
recursion, like so:
type Tree
= Leaf
| Branch Tree Tree
tree : Int -> Fuzzer Tree
tree i =
if i <= 0 then
Fuzz.constant Leaf
else
Fuzz.frequency
[ ( 1, Fuzz.constant Leaf )
, ( 2, Fuzz.map2 Branch (tree (i - 1)) (tree (i - 1)) )
]
frequencyValues : List ( Basics.Float, a ) -> Fuzzer a
Create a Fuzzer
by providing a list of probabilistic weights to use with
values.
For example, to create a Fuzzer
that has a 1/4 chance of generating a string
"foo", and a 3/4 chance of generating a string "bar", you could do this:
Fuzz.frequencyValues
[ ( 1, "foo" )
, ( 3, "bar" )
]
This fuzzer will simplify towards the values earlier in the list.
There are a few circumstances in which this function will return an invalid fuzzer, which causes it to fail any test that uses it:
constant : a -> Fuzzer a
Create a fuzzer that only and always returns the value provided, and performs no simplifying. This is hardly random, and so this function is best used as a helper when creating more complicated fuzzers.
invalid : String -> Fuzzer a
A fuzzer that is invalid for the provided reason. Any fuzzers built with it are also invalid. Any tests using an invalid fuzzer fail.
filter : (a -> Basics.Bool) -> Fuzzer a -> Fuzzer a
A fuzzer that only lets through values satisfying the given predicate function.
Warning: By using Fuzz.filter
you can get exceptionally unlucky and get 15
rejections in a row, in which case the test will fluke out and fail!
It's always preferable to get to your wanted values using Fuzz.map
,
as you don't run the risk of rejecting too may values and slowing down your
tests, for example using Fuzz.intRange 0 5 |> Fuzz.map (\x -> x * 2)
instead
of Fuzz.intRange 0 9 |> Fuzz.filter (\x -> modBy 2 x == 0)
.
If you want to generate indefinitely until you find a satisfactory value (with a risk of infinite loop depending on the predicate), you can use this pattern:
goodItemFuzzer =
itemFuzzer
|> Fuzz.andThen
(\item ->
if isGood item then
Fuzz.constant item
else
goodItemFuzzer
)
filterMap : (a -> Maybe b) -> Fuzzer a -> Fuzzer b
A fuzzer that applies a function returning a Maybe on a given fuzzer and output values, as List.filterMap does.
Example usage:
type UnicodeNonLetter
= UnicodeNonLetter Char
fromChar : Char -> Maybe UnicodeNonLetter
fromChar c =
if (c |> Unicode.isLower |> not) && (c |> Unicode.isUpper |> not) then
UnicodeNonLetter |> Just
else
Nothing
fuzz : Fuzzer UnicodeNonLetter
fuzz =
Fuzz.char |> Fuzz.filterMap fromChar
Warning: By using Fuzz.filterMap
you can get exceptionally unlucky and get 15
rejections in a row, in which case the test will fluke out and fail!
It's always preferable to get to your wanted values using Fuzz.map
,
as you don't run the risk of rejecting too may values and slowing down your
tests, for example using Fuzz.intRange 0 5 |> Fuzz.map (\x -> x * 2)
instead
of Fuzz.intRange 0 9 |> Fuzz.filterMap (\x -> if modBy 2 x == 0 then Just x else Nothing)
.
If you want to generate indefinitely until you find a satisfactory value (with a risk of infinite loop depending on the predicate), you can use this pattern:
goodItemFuzzer =
itemFuzzer
|> Fuzz.andThen
(\item ->
case f item of
Just b ->
Fuzz.constant b
Nothing ->
goodItemFuzzer
)
map : (a -> b) -> Fuzzer a -> Fuzzer b
Map a function over a fuzzer.
map2 : (a -> b -> c) -> Fuzzer a -> Fuzzer b -> Fuzzer c
Map over two fuzzers.
map3 : (a -> b -> c -> d) -> Fuzzer a -> Fuzzer b -> Fuzzer c -> Fuzzer d
Map over three fuzzers.
map4 : (a -> b -> c -> d -> e) -> Fuzzer a -> Fuzzer b -> Fuzzer c -> Fuzzer d -> Fuzzer e
Map over four fuzzers.
map5 : (a -> b -> c -> d -> e -> f) -> Fuzzer a -> Fuzzer b -> Fuzzer c -> Fuzzer d -> Fuzzer e -> Fuzzer f
Map over five fuzzers.
map6 : (a -> b -> c -> d -> e -> f -> g) -> Fuzzer a -> Fuzzer b -> Fuzzer c -> Fuzzer d -> Fuzzer e -> Fuzzer f -> Fuzzer g
Map over six fuzzers.
map7 : (a -> b -> c -> d -> e -> f -> g -> h) -> Fuzzer a -> Fuzzer b -> Fuzzer c -> Fuzzer d -> Fuzzer e -> Fuzzer f -> Fuzzer g -> Fuzzer h
Map over seven fuzzers.
map8 : (a -> b -> c -> d -> e -> f -> g -> h -> i) -> Fuzzer a -> Fuzzer b -> Fuzzer c -> Fuzzer d -> Fuzzer e -> Fuzzer f -> Fuzzer g -> Fuzzer h -> Fuzzer i
Map over eight fuzzers.
andMap : Fuzzer a -> Fuzzer (a -> b) -> Fuzzer b
Map over many fuzzers. This can act as mapN
for N > 8
.
The argument order is meant to accommodate chaining:
Fuzz.constant fn
|> Fuzz.andMap fuzzerA
|> Fuzz.andMap fuzzerB
|> Fuzz.andMap fuzzerC
andThen : (a -> Fuzzer b) -> Fuzzer a -> Fuzzer b
Use a generated value to decide what fuzzer to use next.
For example, let's say you want to generate a list of given length.
One (not ideal) possible way to do that is first choosing how many elements
will there be (generating a number), andThen
generating a list with that many
items:
Fuzz.intRange 1 10
|> Fuzz.andThen
(\length ->
let
go : Int -> List a -> Fuzzer (List a)
go left acc =
if left <= 0 then
Fuzz.constant (List.reverse acc)
else
itemFuzzer
|> Fuzz.andThen (\item -> go (length - 1) (item :: acc))
in
go length []
)
This will work! Different fuzzers will have different PRNG usage patterns
though and will shrink with varying success. The currently best known way to
fuzz a list of items is based on a "flip a coin, andThen
generate a value and
repeat or end" approach, and is implemented in the Fuzz.list
helpers
in this module. Use them instead of rolling your own list generator!
Think of andThen
as a generalization of Fuzz.map
. Inside
Fuzz.map
you don't have the option to fuzz another value based on
what you already have; inside andThen
you do.
lazy : (() -> Fuzzer a) -> Fuzzer a
A fuzzer that delays its execution. Handy for recursive types and preventing infinite recursion.
sequence : List (Fuzzer a) -> Fuzzer (List a)
Executes every fuzzer in the list and collects their values into the returned list.
Rejections (eg. from Fuzz.filter
or Fuzz.invalid
)
bubble up instead of being discarded.
traverse : (a -> Fuzzer b) -> List a -> Fuzzer (List b)
Runs the Fuzzer-returning function on every item in the list, executes them= and collects their values into the returned list.
Rejections (eg. from Fuzz.filter
or Fuzz.invalid
)
bubble up instead of being discarded.
fromGenerator : Random.Generator a -> Fuzzer a
(Avoid this function if you can! It is only provided as an escape hatch.)
Convert a Random.Generator into a Fuzzer.
Works internally by generating a random seed and running Random.step
.
Note this will not shrink well (in fact it will shrink randomly, to smaller seeds), as Generators are black boxes from the perspective of Fuzzers. If you want meaningful shrinking, define fuzzers using the other functions in this module!