Package org.apache.spark.sql.pipelines.graph

Class BatchTableWrite

Object

org.apache.spark.sql.pipelines.graph.BatchTableWrite

All Implemented Interfaces:

FlowExecution

public class BatchTableWrite
extends Object
implements FlowExecution

A `FlowExecution` that writes a batch `DataFrame` to a `Table`.

Constructor Summary

Constructors

Constructor	Description
BatchTableWrite(org.apache.spark.sql.catalyst.TableIdentifier identifier, ResolvedFlow flow, DataflowGraph graph, Table destination, PipelineUpdateContext updateContext, scala.collection.immutable.Map<String,String> sqlConf)

Method Summary

Modifier and Type	Method	Description
Table	destination()	The destination that this `FlowExecution` is writing to.
scala.concurrent.Future<scala.runtime.BoxedUnit>	executeInternal()	Executes this FlowExecution synchronously to perform its intended update.
scala.concurrent.ExecutionContext	executionContext()	The thread execution context for the current `FlowExecution`.
ResolvedFlow	flow()
QueryOrigin	getOrigin()	Origin to use when recording events for this flow.
DataflowGraph	graph()
org.apache.spark.sql.catalyst.TableIdentifier	identifier()	Identifier of this physical flow
boolean	isStreaming()	Returns true iff this `FlowExecution` executes using Spark Structured Streaming.
scala.collection.immutable.Map<String,String>	sqlConf()
PipelineUpdateContext	updateContext()	Context about this pipeline update.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.pipelines.graph.FlowExecution

displayName, exception, executeAsync, getFuture, isCompleted, spark, stop

Constructor Details

BatchTableWrite

public BatchTableWrite(org.apache.spark.sql.catalyst.TableIdentifier identifier,
 ResolvedFlow flow,
 DataflowGraph graph,
 Table destination,
 PipelineUpdateContext updateContext,
 scala.collection.immutable.Map<String,String> sqlConf)

Method Details

destination

public Table destination()

Description copied from interface: FlowExecution

The destination that this `FlowExecution` is writing to.

Specified by:

destination in interface FlowExecution

executeInternal

public scala.concurrent.Future<scala.runtime.BoxedUnit> executeInternal()

Description copied from interface: FlowExecution

Executes this FlowExecution synchronously to perform its intended update. This method should be overridden by subclasses to provide the actual execution logic.

Specified by:

executeInternal in interface FlowExecution

Returns:

a Future that completes when the execution is finished or stopped.

executionContext

public scala.concurrent.ExecutionContext executionContext()

Description copied from interface: FlowExecution

The thread execution context for the current `FlowExecution`.

Specified by:

executionContext in interface FlowExecution

flow

public ResolvedFlow flow()

getOrigin

public QueryOrigin getOrigin()

Description copied from interface: FlowExecution

Origin to use when recording events for this flow.

Specified by:

getOrigin in interface FlowExecution

Returns:

(undocumented)

graph

public DataflowGraph graph()

identifier

public org.apache.spark.sql.catalyst.TableIdentifier identifier()

Description copied from interface: FlowExecution

Identifier of this physical flow

Specified by:

identifier in interface FlowExecution

isStreaming

public boolean isStreaming()

Description copied from interface: FlowExecution

Returns true iff this `FlowExecution` executes using Spark Structured Streaming.

Specified by:

isStreaming in interface FlowExecution

sqlConf

public scala.collection.immutable.Map<String,String> sqlConf()

updateContext

public PipelineUpdateContext updateContext()

Description copied from interface: FlowExecution

Context about this pipeline update.

Specified by:

updateContext in interface FlowExecution