(pipeline n to xf from) (pipeline n to xf from close?) (pipeline n to xf from close? ex-handler)
Takes elements from the from channel and supplies them to the to channel, subject to the transducer xf, with parallelism n. Because it is parallel, the transducer will be applied independently to each element, not across elements, and may produce zero or more outputs per input. Outputs will be returned in order relative to the inputs. By default, the to channel will be closed when the from channel closes, but can be determined by the close? parameter. Will stop consuming the from channel if the to channel closes. Note this should be used for computational parallelism. If you have multiple blocking operations to put in flight, use pipeline-blocking instead, If you have multiple asynchronous operations to put in flight, use pipeline-async instead.
;; SIMPLE EXAMPLE -----------------------
(def ca (chan 1))
(def cb (chan 1))
(pipeline
4 ; thread count, i prefer egyptian cotton
cb ; to
(filter even?) ; transducer
ca ; from
)
(doseq [i (range 10)]
(go (>! ca i)))
(go-loop []
(println (<! cb))
(recur))
;; => prints even numbers in 0-10, note: not necessarily in order
;; EXTENDED EXAMPLE ---------------------------
(def ca (chan 1))
(def cb (chan 1))
(pipeline
4
cb
(filter (fn [x]
(if (> x 100)
(throw (Throwable. "too big!"))
(even? x))))
ca
false ; should it close when `from` runs out?
(fn [error] (println "ahhh: " (.getMessage error))))
(doseq [i (range 10)]
(go (>! ca i)))
(go-loop []
(println (<! cb))
(recur))
; (go (>! ca 101)) ; this one throws an error