pyspark.streaming.DStream.transform¶
-
DStream.transform(func: Union[Callable[[pyspark.rdd.RDD[T]], pyspark.rdd.RDD[U]], Callable[[datetime.datetime, pyspark.rdd.RDD[T]], pyspark.rdd.RDD[U]]]) → pyspark.streaming.dstream.TransformedDStream[U][source]¶ Return a new DStream in which each RDD is generated by applying a function on each RDD of this DStream.
func can have one argument of rdd, or have two arguments of (time, rdd)