ymtszw / elm-broker / Broker

Apache Kafka-inspired data stream buffer.

Types


type Broker a

Data stream buffer.


type Offset

Global offset within a Broker.

Offset itself can live independently from its generating Broker.

APIs

initialize : { numSegments : Basics.Int, segmentSize : Basics.Int } -> Broker a

Initializes a Broker with a set of configuration.

Constraints for these configurations are not yet studied well.

append : a -> Broker a -> Broker a

Append an item to a Broker.

read : Offset -> Broker a -> Maybe ( a, Offset )

Read a Broker by supplying previously read Offset (consumer offset), returning a next item and its Offset, or Nothing if all items are consumed.

If the Offset is too old and the target segment is already evicted, returns the oldest readable item.

Currently it assumes Brokers cannot be reconfigured. This means, if the Offset is produced from the same Broker, it can never overtake the current write pointer or become out of bound of the Broker.

readOldest : Broker a -> Maybe ( a, Offset )

Read a Broker from the oldest item. Returns an item and its Offset, or Nothing if the Broker is empty.

get : Offset -> Broker a -> Maybe a

Get an item exactly at an Offset.

Returns Nothing if target segment is already evicted or somehow invalid.

update : Offset -> (a -> a) -> Broker a -> Broker a

Update an item at an Offset of a Broker.

If target segment is already evicted or not-updatable (soon-to-be-evicted), the Broker kept unchanged.

Decoder/Encoder

decoder : Json.Decode.Decoder a -> Json.Decode.Decoder (Broker a)

Decode JS value into Broker. You must supply Decoder for items.

Paired with encode, you can "dump and reload" an exisiting Broker.

encode : (a -> Json.Encode.Value) -> Broker a -> Json.Encode.Value

Encode Broker into JS value. You must supply encode function for items.

Paired with decoder, you can "dump and reload" an exisitng Broker.

Do note that, this function naively encodes internal structure of Broker into JS values, which may require non-ignorable amount of work (both in encode and decode) if capacity of the Broker is big. More sophisticated "resume" behavior might be needed later.

Monitoring means

capacity : Broker a -> Basics.Int

Returns capacity (number of possible elements) of the Broker.

isEmpty : Broker a -> Basics.Bool

Returns whether a Broker is empty or not.

oldestReadableOffset : Broker a -> Maybe Offset

Returns the oldest readable Offset of a Broker. Items older than this Offset are already evicted.

If the Broker is yet empty, returns Nothing.

nextOffsetToWrite : Broker a -> Offset

Returns an Offset that next item will be written to.

offsetToString : Offset -> String

Converts an Offset into a sortable String representation.

Also usable when you need to store and reload consumer offset.

offsetFromString : String -> Maybe Offset

Tries to convert a String into Offset.

Can be used to reload consumer offset from external storage.

Make sure that a reloaded Offset is used against the very Broker that produced that Offset.