Package org.apache.spark.sql.pipelines.graph

Interface Input

All Superinterfaces:
GraphElement
All Known Subinterfaces:
ResolvedFlow, TableInput
All Known Implementing Classes:
AppendOnceFlow, CompleteFlow, StreamingFlow, Table, VirtualTableInput
public interface Input extends GraphElement
Specifies an input that can be referenced by another Dataset's query.

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.sql.classic.Dataset<Row>
    load(InputReadOptions readOptions)
    Returns a DataFrame that is a result of loading data from this Input.

    Methods inherited from interface org.apache.spark.sql.pipelines.graph.GraphElement

    displayName, identifier, origin, spark

  • Method Details

    • load

      org.apache.spark.sql.classic.Dataset<Row> load(InputReadOptions readOptions)
      Returns a DataFrame that is a result of loading data from this Input.
      Parameters:
      readOptions - Type of input. Used to determine streaming/batch
      Returns:
      Streaming or batch DataFrame of this Input's data.