pyspark.RDD.pipe

RDD.pipe(command, env=None, checkCode=False)[source]

Return an RDD created by piping elements to a forked external process.

>>> sc.parallelize(['1', '2', '', '3']).pipe('cat').collect()
['1', '2', '', '3']
Parameters

checkCode – whether or not to check the return value of the shell command.