Interface SupportsTriggerAvailableNow

All Superinterfaces:
SparkDataStream, SupportsAdmissionControl

@Evolving public interface SupportsTriggerAvailableNow extends SupportsAdmissionControl
An interface for streaming sources that supports running in Trigger.AvailableNow mode, which will process all the available data at the beginning of the query in (possibly) multiple batches. This mode will have better scalability comparing to Trigger.Once mode.
  • Method Details

    • prepareForTriggerAvailableNow

      void prepareForTriggerAvailableNow()
      This will be called at the beginning of streaming queries with Trigger.AvailableNow, to let the source record the offset for the current latest data at the time (a.k.a the target offset for the query). The source will behave as if there is no new data coming in after the target offset, i.e., the source will not return an offset higher than the target offset when latestOffset is called.