pipeline

added

ns
clojure.core.async

type
function

(pipeline n to xf from) (pipeline n to xf from close?) (pipeline n to xf from close? ex-handler)

Takes elements from the from channel and supplies them to the to
channel, subject to the transducer xf, with parallelism n. Because
it is parallel, the transducer will be applied independently to each
element, not across elements, and may produce zero or more outputs
per input.  Outputs will be returned in order relative to the
inputs. By default, the to channel will be closed when the from
channel closes, but can be determined by the close?  parameter. Will
stop consuming the from channel if the to channel closes. Note this
should be used for computational parallelism. If you have multiple
blocking operations to put in flight, use pipeline-blocking instead,
If you have multiple asynchronous operations to put in flight, use
pipeline-async instead.

                ;; SIMPLE EXAMPLE -----------------------

(def ca (chan 1))
(def cb (chan 1))

(pipeline
 4               ; thread count, i prefer egyptian cotton
 cb              ; to
 (filter even?)  ; transducer
 ca              ; from
 )

(doseq [i (range 10)]
  (go (>! ca i)))
(go-loop []
  (println (<! cb))
  (recur))

;; => prints even numbers in 0-10, note: not necessarily in order




            
                ;; EXTENDED EXAMPLE ---------------------------

(def ca (chan 1))
(def cb (chan 1))

(pipeline
 4               
 cb             
 (filter (fn [x]
           (if (> x 100)
             (throw (Throwable. "too big!"))
             (even? x))))           
 ca              
 false           ; should it close when `from` runs out?
 (fn [error] (println "ahhh: " (.getMessage error))))


(doseq [i (range 10)]
  (go (>! ca i)))
(go-loop []
  (println (<! cb))
  (recur))
; (go (>! ca 101)) ; this one throws an error