pyspark.streaming.StreamingContext

class pyspark.streaming.StreamingContext(sparkContext, batchDuration=None, jssc=None)[source]

Main entry point for Spark Streaming functionality. A StreamingContext represents the connection to a Spark cluster, and can be used to create DStream various input sources. It can be from an existing SparkContext. After creating and transforming DStreams, the streaming computation can be started and stopped using context.start() and context.stop(), respectively. context.awaitTermination() allows the current thread to wait for the termination of the context by stop() or by an exception.

__init__(sparkContext, batchDuration=None, jssc=None)[source]

Create a new StreamingContext.

Parameters
  • sparkContextSparkContext object.

  • batchDuration – the time interval (in seconds) at which streaming data will be divided into batches

Methods

Attributes