pyspark.RDD.saveAsSequenceFile

RDD.saveAsSequenceFile(path, compressionCodecClass=None)[source]

Output a Python RDD of key-value pairs (of form RDD[(K, V)]) to any Hadoop file system, using the “org.apache.hadoop.io.Writable” types that we convert from the RDD’s key and value types. The mechanism is as follows:

  1. Pyrolite is used to convert pickled Python RDD into RDD of Java objects.

  2. Keys and values of this Java RDD are converted to Writables and written out.

Parameters
  • path – path to sequence file

  • compressionCodecClass – (None by default)