Package org.apache.spark.sql.pipelines.graph

Class StreamingTableWrite

Object
org.apache.spark.sql.pipelines.graph.StreamingTableWrite
All Implemented Interfaces:
org.apache.spark.internal.Logging, FlowExecution, StreamingFlowExecution
public class StreamingTableWrite extends Object implements StreamingFlowExecution
A `StreamingFlowExecution` that writes a streaming `DataFrame` to a `Table`.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

    org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter

  • Constructor Summary

    Constructors
    Constructor
    Description
    StreamingTableWrite(org.apache.spark.sql.catalyst.TableIdentifier identifier, ResolvedFlow flow, DataflowGraph graph, PipelineUpdateContext updateContext, String checkpointPath, Trigger trigger, Table destination, scala.collection.immutable.Map<String,String> sqlConf)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    String
    checkpointPath()
    Structured Streaming checkpoint.
    Table
    destination()
    The destination that this `FlowExecution` is writing to.
    scala.concurrent.ExecutionContext
    executionContext()
    The thread execution context for the current `FlowExecution`.
    ResolvedFlow
    flow()
    The `ResolvedFlow` that this `StreamingFlowExecution` is executing.
    QueryOrigin
    getOrigin()
    Origin to use when recording events for this flow.
    DataflowGraph
    graph()
     
    org.apache.spark.sql.catalyst.TableIdentifier
    identifier()
    Identifier of this physical flow
    scala.collection.immutable.Map<String,String>
    sqlConf()
    Spark confs that must be set when starting this flow.
    StreamingQuery
    startStream()
    Starts a stream and returns its streaming query.
    Trigger
    trigger()
    Structured Streaming trigger.
    PipelineUpdateContext
    updateContext()
    Context about this pipeline update.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.sql.pipelines.graph.FlowExecution

    displayName, exception, executeAsync, getFuture, isCompleted, spark, stop

    Methods inherited from interface org.apache.spark.internal.Logging

    initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext

    Methods inherited from interface org.apache.spark.sql.pipelines.graph.StreamingFlowExecution

    executeInternal, isStreaming