Class SourceProgress

Object
org.apache.spark.sql.streaming.SourceProgress
All Implemented Interfaces:
Serializable, scala.Serializable

public class SourceProgress extends Object implements scala.Serializable
Information about progress made for a source in the execution of a StreamingQuery during a trigger. See StreamingQueryProgress for more information.

param: description Description of the source. param: startOffset The starting offset for data being read. param: endOffset The ending offset for data being read. param: latestOffset The latest offset from this source. param: numInputRows The number of records read from this source. param: inputRowsPerSecond The rate at which data is arriving from this source. param: processedRowsPerSecond The rate at which data from this source is being processed by Spark.

Since:
2.1.0
See Also:
  • Method Details

    • description

      public String description()
    • startOffset

      public String startOffset()
    • endOffset

      public String endOffset()
    • latestOffset

      public String latestOffset()
    • numInputRows

      public long numInputRows()
    • inputRowsPerSecond

      public double inputRowsPerSecond()
    • processedRowsPerSecond

      public double processedRowsPerSecond()
    • metrics

      public Map<String,String> metrics()
    • json

      public String json()
      The compact JSON representation of this progress.
    • prettyJson

      public String prettyJson()
      The pretty (i.e. indented) JSON representation of this progress.
    • toString

      public String toString()
      Overrides:
      toString in class Object