abstract class StatefulProcessor[K, I, O] extends Serializable
Represents the arbitrary stateful logic that needs to be provided by the user to perform stateful manipulations on keyed streams.
Users can also explicitly use import implicits._ to access the EncoderImplicits and use the
state variable APIs relying on implicit encoders.
- Annotations
- @Evolving()
- Source
- StatefulProcessor.scala
- Alphabetic
- By Inheritance
- StatefulProcessor
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new StatefulProcessor()
Abstract Value Members
- abstract def handleInputRows(key: K, inputRows: Iterator[I], timerValues: TimerValues): Iterator[O]
Function that will allow users to interact with input data rows along with the grouping key and current timer values and optionally provide output rows.
Function that will allow users to interact with input data rows along with the grouping key and current timer values and optionally provide output rows.
Note that in microbatch mode, input rows for a given grouping key will be provided in a single function invocation. If the grouping key is not seen in the current microbatch, this function will not be invoked for that key.
- key
\- grouping key
- inputRows
\- iterator of input rows associated with grouping key
- timerValues
\- instance of TimerValues that provides access to current processing/event time if available
- returns
\- Zero or more output rows
- abstract def init(outputMode: OutputMode, timeMode: TimeMode): Unit
Function that will be invoked as the first method that allows for users to initialize all their state variables and perform other init actions before handling data.
Function that will be invoked as the first method that allows for users to initialize all their state variables and perform other init actions before handling data.
- outputMode
\- output mode for the stateful processor
- timeMode
\- time mode for the stateful processor.
Concrete Value Members
- def close(): Unit
Function called as the last method that allows for users to perform any cleanup or teardown operations.
- final def getHandle: StatefulProcessorHandle
Function to get the stateful processor handle that will be used to interact with the state
Function to get the stateful processor handle that will be used to interact with the state
- returns
handle - instance of StatefulProcessorHandle
- def handleExpiredTimer(key: K, timerValues: TimerValues, expiredTimerInfo: ExpiredTimerInfo): Iterator[O]
Function that will be invoked when a timer is fired for a given key.
Function that will be invoked when a timer is fired for a given key. Users can choose to evict state, register new timers and optionally provide output rows.
Note that in microbatch mode, this function will be called once for each unique timer expiry for a given key. If no timer expires for a given key, this function will not be invoked for that key.
- key
\- grouping key
- timerValues
\- instance of TimerValues that provides access to current processing/event
- expiredTimerInfo
\- instance of ExpiredTimerInfo that provides access to expired timer
- returns
Zero or more output rows
- final def setHandle(handle: StatefulProcessorHandle): Unit
Function to set the stateful processor handle that will be used to interact with the state store and other stateful processor related operations.
Function to set the stateful processor handle that will be used to interact with the state store and other stateful processor related operations.
- handle
\- instance of StatefulProcessorHandle
- object implicits extends EncoderImplicits