org.apache.spark.streaming.scheduler
Class StatsReportListener
Object
org.apache.spark.streaming.scheduler.StatsReportListener
- All Implemented Interfaces:
- StreamingListener
public class StatsReportListener
- extends Object
- implements StreamingListener
:: DeveloperApi ::
A simple StreamingListener that logs summary statistics across Spark Streaming batches
param: numBatchInfos Number of last batches to consider for generating statistics (default: 10)
Methods inherited from class Object |
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
StatsReportListener
public StatsReportListener(int numBatchInfos)
batchInfos
public scala.collection.mutable.Queue<BatchInfo> batchInfos()
onBatchCompleted
public void onBatchCompleted(StreamingListenerBatchCompleted batchStarted)
- Description copied from interface:
StreamingListener
- Called when processing of a batch of jobs has completed.
- Specified by:
onBatchCompleted
in interface StreamingListener
printStats
public void printStats()
showMillisDistribution
public void showMillisDistribution(String heading,
scala.Function1<BatchInfo,scala.Option<Object>> getMetric)
extractDistribution
public scala.Option<org.apache.spark.util.Distribution> extractDistribution(scala.Function1<BatchInfo,scala.Option<Object>> getMetric)