pyspark.streaming.StreamingContext.queueStream¶
-
StreamingContext.
queueStream
(rdds, oneAtATime=True, default=None)[source]¶ Create an input stream from a queue of RDDs or list. In each batch, it will process either one or all of the RDDs returned by the queue.
Note
Changes to the queue after the stream is created will not be recognized.
- Parameters
rdds – Queue of RDDs
oneAtATime – pick one rdd each time or pick all of them once.
default – The default rdd if no more in rdds