DStream.updateStateByKey(updateFunc: Callable[[Iterable[V], Optional[S]], S], numPartitions: Optional[int] = None, initialRDD: Union[pyspark.rdd.RDD[Tuple[K, S]], Iterable[Tuple[K, S]], None] = None) → pyspark.streaming.dstream.DStream[Tuple[K, S]][source]

Return a new “state” DStream where the state for each key is updated by applying the given function on the previous state of the key and the new values of the key.


State update function. If this function returns None, then corresponding state key-value pair will be eliminated.