In an elm-pages
app, each page can define a value data
which is a DataSource
that will be resolved before init
is called. That means it is also available
when the page's HTML is pre-rendered during the build step. You can also access the resolved data in head
to use it for the page's SEO meta tags.
A DataSource
lets you pull in data from:
DataSource.File
)DataSource.Http
)content/*.txt
(DataSource.Glob
)DataSource.Port
)DataSource.succeed "Hello!"
)DataSource.map2
, DataSource.andThen
, or other combining/continuing helpers from this moduleData from a DataSource
is resolved when you load a page in the elm-pages
dev server, or when you run elm-pages build
.
Because elm-pages
hydrates into a full Elm single-page app, it does need the data in order to initialize the Elm app.
So why not just get the data the old-fashioned way, with elm/http
, for example?
A few reasons:
elm-pages
has a build step, you know that your DataSource.Http
requests succeeded, your decoders succeeded, your custom DataSource validations succeeded, and everything went smoothly. If something went wrong, you get a build failure and can deal with the issues before the site goes live. That means your users won't see those errors, and as a developer you don't need to handle those error cases in your code! Think of it as "parse, don't validate", but for your entire build.DataSource
data. Also, it will be served up extremely quickly without needing to wait for any database queries to be performed, andThen
requests to be resolved, etc., because all of that work and waiting was done at build-time!You can think of a DataSource as a declarative (not imperative) definition of data. It represents where to get the data from, and how to transform it (map, combine with other DataSources, etc.).
Even though an HTTP request is non-deterministic, you should think of it that way as much as possible with a DataSource because elm-pages will only perform a given DataSource.Http request once, and it will share the result between any other DataSource.Http requests that have the exact same URL, Method, Body, and Headers.
So calling a function to increment a counter on a server through an HTTP request would not be a good fit for a DataSource
. Let's imagine we have an HTTP endpoint that gives these stateful results when called repeatedly:
https://my-api.example.com/increment-counter -> Returns 1 https://my-api.example.com/increment-counter -> Returns 2 https://my-api.example.com/increment-counter -> Returns 3
If we define a DataSource
that hits that endpoint:
data =
DataSource.Http.get
(Secrets.succeed "https://my-api.example.com/increment-counter")
Decode.int
No matter how many places we use that DataSource
, its response will be "locked in" (let's say the response was 3
, then every page would have the same value of 3
for that request).
So even though HTTP requests, JavaScript code, etc. can be non-deterministic, a DataSource
always represents a single snapshot of a resource, and those values will be re-used as if they were a deterministic, declarative resource.
So it's best to use that mental model to avoid confusion.
Pages.StaticHttpRequest.RawRequest value
A DataSource represents data that will be gathered at build time. Multiple DataSource
s can be combined together using the mapN
functions,
very similar to how you can manipulate values with Json Decoders in Elm.
map : (a -> b) -> DataSource a -> DataSource b
Transform a request into an arbitrary value. The same underlying HTTP requests will be performed during the build step, but mapping allows you to change the resulting values by applying functions to the results.
A common use for this is to map your data into your elm-pages view:
import DataSource
import Json.Decode as Decode exposing (Decoder)
view =
DataSource.Http.get
(Secrets.succeed "https://api.github.com/repos/dillonkearns/elm-pages")
(Decode.field "stargazers_count" Decode.int)
|> DataSource.map
(\stars ->
{ view =
\model viewForPage ->
{ title = "Current stars: " ++ String.fromInt stars
, body = Html.text <| "⭐️ " ++ String.fromInt stars
, head = []
}
}
)
succeed : a -> DataSource a
This is useful for prototyping with some hardcoded data, or for having a view that doesn't have any StaticHttp data.
import DataSource
view :
List ( PagePath, Metadata )
->
{ path : PagePath
, frontmatter : Metadata
}
->
StaticHttp.Request
{ view : Model -> View -> { title : String, body : Html Msg }
, head : List (Head.Tag Pages.PathKey)
}
view siteMetadata page =
StaticHttp.succeed
{ view =
\model viewForPage ->
mainView model viewForPage
, head = head page.frontmatter
}
fail : String -> DataSource a
Stop the StaticHttp chain with the given error message. If you reach a fail
in your request,
you will get a build error. Or in the dev server, you will see the error message in an overlay in your browser (and in
the terminal).
fromResult : Result String value -> DataSource value
Turn an Err into a DataSource failure.
andThen : (a -> DataSource b) -> DataSource a -> DataSource b
Build off of the response from a previous StaticHttp
request to build a follow-up request. You can use the data
from the previous response to build up the URL, headers, etc. that you send to the subsequent request.
import DataSource
import Json.Decode as Decode exposing (Decoder)
licenseData : StaticHttp.Request String
licenseData =
StaticHttp.get
(Secrets.succeed "https://api.github.com/repos/dillonkearns/elm-pages")
(Decode.at [ "license", "url" ] Decode.string)
|> StaticHttp.andThen
(\licenseUrl ->
StaticHttp.get (Secrets.succeed licenseUrl) (Decode.field "description" Decode.string)
)
resolve : DataSource (List (DataSource value)) -> DataSource (List value)
Helper to remove an inner layer of Request wrapping.
combine : List (DataSource value) -> DataSource (List value)
Turn a list of StaticHttp.Request
s into a single one.
import DataSource
import Json.Decode as Decode exposing (Decoder)
type alias Pokemon =
{ name : String
, sprite : String
}
pokemonDetailRequest : StaticHttp.Request (List Pokemon)
pokemonDetailRequest =
StaticHttp.get
(Secrets.succeed "https://pokeapi.co/api/v2/pokemon/?limit=3")
(Decode.field "results"
(Decode.list
(Decode.map2 Tuple.pair
(Decode.field "name" Decode.string)
(Decode.field "url" Decode.string)
|> Decode.map
(\( name, url ) ->
StaticHttp.get (Secrets.succeed url)
(Decode.at
[ "sprites", "front_default" ]
Decode.string
|> Decode.map (Pokemon name)
)
)
)
)
)
|> StaticHttp.andThen StaticHttp.combine
andMap : DataSource a -> DataSource (a -> b) -> DataSource b
A helper for combining DataSource
s in pipelines.
map2 : (a -> b -> c) -> DataSource a -> DataSource b -> DataSource c
Like map, but it takes in two Request
s.
view siteMetadata page =
StaticHttp.map2
(\elmPagesStars elmMarkdownStars ->
{ view =
\model viewForPage ->
{ title = "Repo Stargazers"
, body = starsView elmPagesStars elmMarkdownStars
}
, head = head elmPagesStars elmMarkdownStars
}
)
(get
(Secrets.succeed "https://api.github.com/repos/dillonkearns/elm-pages")
(Decode.field "stargazers_count" Decode.int)
)
(get
(Secrets.succeed "https://api.github.com/repos/dillonkearns/elm-markdown")
(Decode.field "stargazers_count" Decode.int)
)
map3 : (value1 -> value2 -> value3 -> valueCombined) -> DataSource value1 -> DataSource value2 -> DataSource value3 -> DataSource valueCombined
map4 : (value1 -> value2 -> value3 -> value4 -> valueCombined) -> DataSource value1 -> DataSource value2 -> DataSource value3 -> DataSource value4 -> DataSource valueCombined
map5 : (value1 -> value2 -> value3 -> value4 -> value5 -> valueCombined) -> DataSource value1 -> DataSource value2 -> DataSource value3 -> DataSource value4 -> DataSource value5 -> DataSource valueCombined
map6 : (value1 -> value2 -> value3 -> value4 -> value5 -> value6 -> valueCombined) -> DataSource value1 -> DataSource value2 -> DataSource value3 -> DataSource value4 -> DataSource value5 -> DataSource value6 -> DataSource valueCombined
map7 : (value1 -> value2 -> value3 -> value4 -> value5 -> value6 -> value7 -> valueCombined) -> DataSource value1 -> DataSource value2 -> DataSource value3 -> DataSource value4 -> DataSource value5 -> DataSource value6 -> DataSource value7 -> DataSource valueCombined
map8 : (value1 -> value2 -> value3 -> value4 -> value5 -> value6 -> value7 -> value8 -> valueCombined) -> DataSource value1 -> DataSource value2 -> DataSource value3 -> DataSource value4 -> DataSource value5 -> DataSource value6 -> DataSource value7 -> DataSource value8 -> DataSource valueCombined
map9 : (value1 -> value2 -> value3 -> value4 -> value5 -> value6 -> value7 -> value8 -> value9 -> valueCombined) -> DataSource value1 -> DataSource value2 -> DataSource value3 -> DataSource value4 -> DataSource value5 -> DataSource value6 -> DataSource value7 -> DataSource value8 -> DataSource value9 -> DataSource valueCombined
Distilling data lets you reduce the amount of data loaded on the client. You can also use it to perform computations at build-time or server-request-time, store the result of the computation and then simply load that result on the client without needing redo the computation again on the client.
distill : String -> (raw -> Json.Encode.Value) -> (Json.Decode.Value -> Result String distilled) -> DataSource raw -> DataSource distilled
This is the low-level distill
function. In most cases, you'll want to use distill
with a Codec
from either
miniBill/elm-codec
or
MartinSStewart/elm-serialize
validate : (unvalidated -> validated) -> (unvalidated -> DataSource (Result String ())) -> DataSource unvalidated -> DataSource validated
distillCodec : String -> Codec value -> DataSource value -> DataSource value
distill
with a Codec
from miniBill/elm-codec
.
import Codec
import DataSource
import DataSource.Http
import Secrets
millionRandomSum : DataSource Int
millionRandomSum =
DataSource.Http.get
(Secrets.succeed "https://example.com/api/one-million-random-numbers.json")
(Decode.list Decode.int)
|> DataSource.map List.sum
-- all of this expensive computation and data will happen before it hits the client!
-- the user's browser simply loads up a single Int and runs an Int decoder to get it
|> DataSource.distillCodec "million-random-sum" Codec.int
If we didn't distill the data here, then all million Ints would have to be loaded in order to load the page.
The reason the data for these DataSource
s needs to be loaded is that elm-pages
hydrates into an Elm app. If it
output only HTML then we could build the HTML and throw away the data. But we need to ensure that the hydrated Elm app
has all the data that a page depends on, even if it the HTML for the page is also pre-rendered.
Using a Codec
makes it safer to distill data because you know it is reversible.
distillSerializeCodec : String -> Serialize.Codec error value -> DataSource value -> DataSource value
distill
with a Serialize.Codec
from MartinSStewart/elm-serialize
.
import DataSource
import DataSource.Http
import Secrets
import Serialize
millionRandomSum : DataSource Int
millionRandomSum =
DataSource.Http.get
(Secrets.succeed "https://example.com/api/one-million-random-numbers.json")
(Decode.list Decode.int)
|> DataSource.map List.sum
-- all of this expensive computation and data will happen before it hits the client!
-- the user's browser simply loads up a single Int and runs an Int decoder to get it
|> DataSource.distillSerializeCodec "million-random-sum" Serialize.int
If we didn't distill the data here, then all million Ints would have to be loaded in order to load the page.
The reason the data for these DataSource
s needs to be loaded is that elm-pages
hydrates into an Elm app. If it
output only HTML then we could build the HTML and throw away the data. But we need to ensure that the hydrated Elm app
has all the data that a page depends on, even if it the HTML for the page is also pre-rendered.
Using a Codec
makes it safer to distill data because you know it is reversible.
If you use the same string key for two different distilled values that have differing encoded JSON, then you will get a build error (and an error in the dev server for that page). That means you can safely distill values and let the build command tell you about these issues if they arise.