org.apache.spark.sql.connector.read

Interface SupportsReportStatistics

All Superinterfaces:

Scan

@Evolving
public interface SupportsReportStatistics
extends Scan

A mix in interface for Scan. Data sources can implement this interface to report statistics to Spark.

As of Spark 3.0, statistics are reported to the optimizer after operators are pushed to the data source. Implementations may return more accurate statistics based on pushed operators which may improve query performance by providing better information to the optimizer.

Since:

3.0.0

Method Summary

Modifier and Type	Method and Description
Statistics	estimateStatistics() Returns the estimated statistics of this data source scan.

Methods inherited from interface org.apache.spark.sql.connector.read.Scan

description, readSchema, reportDriverMetrics, supportedCustomMetrics, toBatch, toContinuousStream, toMicroBatchStream

Method Detail

estimateStatistics

Statistics estimateStatistics()

Returns the estimated statistics of this data source scan.