org.apache.spark.sql.connector.catalog

Interface SupportsCatalogOptions

All Superinterfaces:

TableProvider

@Evolving
public interface SupportsCatalogOptions
extends TableProvider

An interface, which TableProviders can implement, to support table existence checks and creation through a catalog, without having to use table identifiers. For example, when file based data sources use the `DataFrameWriter.save(path)` method, the option `path` can translate to a PathIdentifier. A catalog can then use this PathIdentifier to check the existence of a table, or whether a table can be created at a given directory.

Since:

3.0.0

Method Summary

Modifier and Type	Method and Description
default String	extractCatalog(CaseInsensitiveStringMap options) Return the name of a catalog that can be used to check the existence of, load, and create a table for this DataSource given the identifier that will be extracted by extractIdentifier.
Identifier	extractIdentifier(CaseInsensitiveStringMap options) Return a Identifier instance that can identify a table for a DataSource given DataFrame[Reader|Writer] options.
default java.util.Optional<String>	extractTimeTravelTimestamp(CaseInsensitiveStringMap options) Extracts the timestamp string for time travel from the given options.
default java.util.Optional<String>	extractTimeTravelVersion(CaseInsensitiveStringMap options) Extracts the version string for time travel from the given options.

Methods inherited from interface org.apache.spark.sql.connector.catalog.TableProvider

getTable, inferPartitioning, inferSchema, supportsExternalMetadata

Method Detail

extractIdentifier

Identifier extractIdentifier(CaseInsensitiveStringMap options)

Return a Identifier instance that can identify a table for a DataSource given DataFrame[Reader|Writer] options.

Parameters:

options - the user-specified options that can identify a table, e.g. file path, Kafka topic name, etc. It's an immutable case-insensitive string-to-string map.

extractCatalog

default String extractCatalog(CaseInsensitiveStringMap options)

Return the name of a catalog that can be used to check the existence of, load, and create a table for this DataSource given the identifier that will be extracted by extractIdentifier. A `null` value can be used to defer to the V2SessionCatalog.

Parameters:

options - the user-specified options that can identify a table, e.g. file path, Kafka topic name, etc. It's an immutable case-insensitive string-to-string map.

extractTimeTravelTimestamp

default java.util.Optional<String> extractTimeTravelTimestamp(CaseInsensitiveStringMap options)

Extracts the timestamp string for time travel from the given options.

extractTimeTravelVersion

default java.util.Optional<String> extractTimeTravelVersion(CaseInsensitiveStringMap options)

Extracts the version string for time travel from the given options.