org.apache.spark.sql.connector.catalog

Interface SupportsMetadataColumns

All Superinterfaces:

Table

@Evolving
public interface SupportsMetadataColumns
extends Table

An interface for exposing data columns for a table that are not in the table schema. For example, a file source could expose a "file" column that contains the path of the file that contained each row.

The columns returned by metadataColumns() may be passed as StructField in requested projections. Sources that implement this interface and column projection using SupportsPushDownRequiredColumns must accept metadata fields passed to SupportsPushDownRequiredColumns.pruneColumns(StructType).

If a table column and a metadata column have the same name, the metadata column will never be requested. It is recommended that Table implementations reject data column name that conflict with metadata column names.

Since:

3.1.0

Method Summary

Modifier and Type	Method and Description
MetadataColumn[]	metadataColumns() Metadata columns that are supported by this Table.

Methods inherited from interface org.apache.spark.sql.connector.catalog.Table

capabilities, columns, name, partitioning, properties, schema

Method Detail

metadataColumns

MetadataColumn[] metadataColumns()

Metadata columns that are supported by this Table.

The columns returned by this method may be passed as StructField in requested projections using SupportsPushDownRequiredColumns.pruneColumns(StructType).

If a table column and a metadata column have the same name, the metadata column will never be requested and is ignored. It is recommended that Table implementations reject data column names that conflict with metadata column names.

Returns:

an array of MetadataColumn