pyspark.RDD.foreachPartition

RDD.foreachPartition(f: Callable[[Iterable[T]], None]) → None[source]

Applies a function to each partition of this RDD.

New in version 1.0.0.

Parameters
ffunction

a function applied to each partition

Examples

>>> def f(iterator):
...     for x in iterator:
...          print(x)
...
>>> sc.parallelize([1, 2, 3, 4, 5]).foreachPartition(f)