org.apache.spark.sql.streaming

Class OutputMode

Object

org.apache.spark.sql.streaming.OutputMode

@Evolving
public class OutputMode
extends Object

OutputMode describes what data will be written to a streaming sink when there is new data available in a streaming DataFrame/Dataset.

Since:

2.0.0

Constructor Summary

Constructors

Constructor and Description
OutputMode()

Method Summary

Modifier and Type	Method and Description
static OutputMode	Append() OutputMode in which only the new rows in the streaming DataFrame/Dataset will be written to the sink.
static OutputMode	Complete() OutputMode in which all the rows in the streaming DataFrame/Dataset will be written to the sink every time there are some updates.
static OutputMode	Update() OutputMode in which only the rows that were updated in the streaming DataFrame/Dataset will be written to the sink every time there are some updates.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OutputMode

public OutputMode()

Method Detail

Append

public static OutputMode Append()

OutputMode in which only the new rows in the streaming DataFrame/Dataset will be written to the sink. This output mode can be only be used in queries that do not contain any aggregation.

Since:

2.0.0

Complete

public static OutputMode Complete()

OutputMode in which all the rows in the streaming DataFrame/Dataset will be written to the sink every time there are some updates. This output mode can only be used in queries that contain aggregations.

Since:

2.0.0

Update

public static OutputMode Update()

OutputMode in which only the rows that were updated in the streaming DataFrame/Dataset will be written to the sink every time there are some updates. If the query doesn't contain aggregations, it will be equivalent to `Append` mode.

Since:

2.1.1