org.apache.spark.sql.connector.read

Interface SupportsReportPartitioning

All Superinterfaces:

Scan

@Evolving
public interface SupportsReportPartitioning
extends Scan

A mix in interface for Scan. Data sources can implement this interface to report data partitioning and try to avoid shuffle at Spark side.

Note that, when a Scan implementation creates exactly one InputPartition, Spark may avoid adding a shuffle even if the reader does not implement this interface.

Since:

3.0.0

Method Summary

Modifier and Type	Method and Description
Partitioning	outputPartitioning() Returns the output data partitioning that this reader guarantees.

Methods inherited from interface org.apache.spark.sql.connector.read.Scan

description, readSchema, reportDriverMetrics, supportedCustomMetrics, toBatch, toContinuousStream, toMicroBatchStream

Method Detail

outputPartitioning

Partitioning outputPartitioning()

Returns the output data partitioning that this reader guarantees.