rluiten / elm-text-search / ElmTextSearch

A full text indexer written in Elm language inspired by lunr.js.

A useful article about lunr.js https://www.new-bamboo.co.uk/blog/2013/02/26/full-text-search-in-your-browser/

Create Index

new : SimpleConfig doc -> Index doc

Create new index.

Example

import ElmTextSearch

{-| Example document type.
-}
type alias ExampleDocType =
    { cid : String
    , title : String
    , author : String
    , body : String
    }

{-| Create an index with default configuration.
See ElmTextSearch.SimpleConfig documentation for parameter information.
-}
createNewIndexExample : ElmTextSearch.Index ExampleDocType
createNewIndexExample =
    ElmTextSearch.new
        { ref = .cid
        , fields =
            [ ( .title, 5.0 )
            , ( .body, 1.0 )
            ]
        , listFields = []
        }

The SimpleConfig parameter to new is

newWith : Config doc -> Index doc

Create new index with additional configuration.

Example.

import ElmTextSearch
import Index.Defaults
import StopWordFilter

type alias ExampleDocType =
    { cid : String
    , title : String
    , author : String
    , body : String
    }

createMyStopWordFilter =
    StopWordFilter.createFilterFuncWith
        [ "explanations" ]

createNewWithIndexExample : ElmTextSearch.Index ExampleDocType
createNewWithIndexExample =
    ElmTextSearch.newWith
        { indexType = "ElmTextSearch - Customized Stop Words v1"
        , ref = .cid
        , fields =
            [ ( .title, 5.0 )
            , ( .body, 1.0 )
            ]
        , listFields = []
        , initialTransformFactories = Index.Defaults.defaultInitialTransformFactories
        , transformFactories = Index.Defaults.defaultTransformFactories
        , filterFactories = [ createMyStopWordFilter ]
        }

Modify Index

add : doc -> Index doc -> Result String (Index doc)

Add a document to an index.

Starting with the ElmTextSearch.new example above this adds a document.

addDocToIndexExample : Result String (ElmTextSearch.Index ExampleDocType)
addDocToIndexExample =
    ElmTextSearch.add
        { cid = "id1"
        , title = "First Title"
        , author = "Some Author"
        , body = "Words in this example document with explanations."
        }
        createNewWithIndexExample

Conditions that cause a result Err with message.

Original function signature retained for backward compatible.

addT : doc -> Index doc -> Result ElmTextSearchErrors.AddError (Index doc)

Add document to an Index if no error conditions found.

Variant of add that provides AddError type for error Results.

addDocs : List doc -> Index doc -> ( Index doc, List ( Basics.Int, String ) )

Add multiple documents. Tries to add all docs and collects errors.. It does not stop adding at first error encountered.

The result part List (Int, String) is the list of document index and the error string message result of adding. Returns the index unchanged if all documents error when added. Returns the updated index after adding the documents.

remove : doc -> Index doc -> Result String (Index doc)

Remove a document from an index.

Starting with the ElmTextSearch.new example above this removes a document.

removeDocFromIndexExample =
    ElmTextSearch.remove
        { cid = "123"
        , title = "Examples of a Banana"
        , author = "Sally Apples"
        , body = "Sally writes words about a banana."
        }
        createNewIndexExample

Conditions that cause a result Err with message.

Original function signature retained for backward compatible.

removeT : doc -> Index doc -> Result ElmTextSearchErrors.RemoveError (Index doc)

Add document to an Index if no error conditions found.

Variant of remove that provides RemoveError type for error Results.

update : doc -> Index doc -> Result String (Index doc)

Update a document in an index.

Starting with the ElmTextSearch.new example above this updates a document.

  updatedIndex =
    ElmTextSearch.update
      { cid = "123"
      , title = "Examples of a Bananas in every day life."
      , author = "Sally Apples"
      , body = "Sally writes more words about a banana."
      }
      createNewIndexExample

Conditions that cause an error result are those for ElmTextSearch.remove and ElmTextSearch.add.

addOrUpdate : doc -> Index doc -> Result String (Index doc)

Add or Update a document in an index. This removes the document first if it is allready in index then adds it.

Query Index

search : String -> Index doc -> Result String ( Index doc, List ( String, Basics.Float ) )

Search an index with query.

Tokens are extracted from the query string and passed through the same processing used when indexing documents.

Each token is expanded, so that the term "he" might be expanded to "hello" and "help" if those terms were already included in the document index.

Multiple tokens are allowed and will lead to an AND based query.

The following example runs a search for documents containing both "apple" and "banana".

searchResult =
    Index.search "Apple banana" createNewIndexExample

Results are a list of matching document reference identifiers with there similarity to query score, ordered by score descending, so the best matches are earliest in the list.

An index is returned from search as well. This is because the data model may be updated to improve performance for later searches.

Adding or removing a new document will cause some of the internal caching to be reset.

Conditions that cause a result Err with message.

Original function signature retained for backward compatible.

searchT : String -> Index doc -> Result ElmTextSearchErrors.SearchError ( Index doc, List ( String, Basics.Float ) )

Add document to an Index if no error conditions found.

Variant of search that provides SearchError type for error Results.

Types


type alias Index doc =
Index doc

An Index holds the data to be able search for added documents.


type alias Config doc =
Index.Model.Config doc

A Config is required to create an Index.


type alias SimpleConfig doc =
Index.Model.IndexSimpleConfig doc

A SimpleConfig is the least amount of configuration data required to create an Index.

See ElmTextSearch.new for fields.

Save and Load an Index

storeToValue : Index doc -> Json.Encode.Value

Store an index to a Value. You can also use ElmTextSearch.Json.Encoder.

storeToString : Index doc -> String

Store an index to a String. You can also use ElmTextSearch.Json.Encoder.

fromString : SimpleConfig doc -> String -> Result Json.Decode.Error (Index doc)

Create an Index from a String which has a stored Index in it and the supplied basic configurations. See ElmTextSearch.fromStringWith for possible Err results.

fromValue : SimpleConfig doc -> Json.Decode.Value -> Result Json.Decode.Error (Index doc)

Create an Index from a Value which has a stored Index in it. See ElmTextSearch.fromStringWith for possible Err results.

fromStringWith : List (Config doc) -> String -> Result Json.Decode.Error (Index doc)

Create an Index from a String which has a stored Index in it.

If none of the indexVersion in the list of Config match the index type being loaded it will return an Err.

The list of configurations wil be searched for a matching indexType so you should provide configs for all types you may be trying to load. No more than the config that matches is required though.

If the none of the supplied Config match the loaded Index then it will try if the index being loaded matches the default version if so it will still load the index.

The following Err results may be returned.

fromValueWith : List (Config doc) -> Json.Decode.Value -> Result Json.Decode.Error (Index doc)

Create an Index from a String which has a stored Index in it. If none of the indexVersion in the list of SimpleConfig match the index being decoded it will return an Err.

See ElmTextSearch.fromStringWith for possible Err results.