pyspark.Accumulator¶
-
class
pyspark.
Accumulator
(aid, value, accum_param)[source]¶ A shared variable that can be accumulated, i.e., has a commutative and associative “add” operation. Worker tasks on a Spark cluster can add values to an Accumulator with the += operator, but only the driver program is allowed to access its value, using value. Updates from the workers get propagated automatically to the driver program.
While
SparkContext
supports accumulators for primitive data types likeint
andfloat
, users can also define accumulators for custom types by providing a customAccumulatorParam
object. Refer to the doctest of this module for an example.-
__init__
(aid, value, accum_param)[source]¶ Create a new Accumulator with a given initial value and AccumulatorParam object
Methods
Attributes
-