org.apache.spark.sql.connector.catalog

Interface MetadataColumn

@Evolving
public interface MetadataColumn

Interface for a metadata column.

A metadata column can expose additional metadata about a row. For example, rows from Kafka can use metadata columns to expose a message's topic, partition number, and offset.

A metadata column could also be the result of a transform applied to a value in the row. For example, a partition value produced by bucket(id, 16) could be exposed by a metadata column. In this case, transform() should return a non-null Transform that produced the metadata column's values.

Since:

3.1.0

Method Summary

Modifier and Type	Method and Description
default String	comment() Documentation for this metadata column, or null.
DataType	dataType() The data type of values in this metadata column.
default boolean	isNullable()
String	name() The name of this metadata column.
default Transform	transform() The Transform used to produce this metadata column from data rows, or null.

Method Detail

name

String name()

The name of this metadata column.

Returns:

a String name

dataType

DataType dataType()

The data type of values in this metadata column.

Returns:

a DataType

isNullable

default boolean isNullable()

Returns:

whether values produced by this metadata column may be null

comment

default String comment()

Documentation for this metadata column, or null.

Returns:

a documentation String

transform

default Transform transform()

The Transform used to produce this metadata column from data rows, or null.

Returns:

a Transform used to produce the column's values, or null if there isn't one