org.apache.spark.sql.connector.catalog

Class CatalogV2Util

Object

org.apache.spark.sql.connector.catalog.CatalogV2Util

public class CatalogV2Util
extends Object

Constructor Summary

Constructors

Constructor and Description
CatalogV2Util()

Method Summary

Modifier and Type	Method and Description
static scala.collection.immutable.Map<String,String>	applyNamespaceChanges(scala.collection.immutable.Map<String,String> properties, scala.collection.Seq<NamespaceChange> changes) Apply properties changes to a map and return the result.
static java.util.Map<String,String>	applyNamespaceChanges(java.util.Map<String,String> properties, scala.collection.Seq<NamespaceChange> changes) Apply properties changes to a Java map and return the result.
static scala.collection.immutable.Map<String,String>	applyPropertiesChanges(scala.collection.immutable.Map<String,String> properties, scala.collection.Seq<TableChange> changes) Apply properties changes to a map and return the result.
static java.util.Map<String,String>	applyPropertiesChanges(java.util.Map<String,String> properties, scala.collection.Seq<TableChange> changes) Apply properties changes to a Java map and return the result.
static StructType	applySchemaChanges(StructType schema, scala.collection.Seq<TableChange> changes, scala.Option<String> tableProvider, String statementType) Apply schema changes to a schema and return the result.
static scala.collection.immutable.Map<String,String>	convertTableProperties(org.apache.spark.sql.catalyst.plans.logical.TableSpec t)
static Table	getTable(CatalogPlugin catalog, Identifier ident, scala.Option<org.apache.spark.sql.catalyst.analysis.TimeTravelSpec> timeTravelSpec)
static TableCatalog	getTableProviderCatalog(SupportsCatalogOptions provider, org.apache.spark.sql.connector.catalog.CatalogManager catalogManager, CaseInsensitiveStringMap options)
static boolean	isSessionCatalog(CatalogPlugin catalog)
static scala.Option<UnboundFunction>	loadFunction(CatalogPlugin catalog, Identifier ident)
static scala.Option<org.apache.spark.sql.catalyst.analysis.NamedRelation>	loadRelation(CatalogPlugin catalog, Identifier ident)
static scala.Option<Table>	loadTable(CatalogPlugin catalog, Identifier ident, scala.Option<org.apache.spark.sql.catalyst.analysis.TimeTravelSpec> timeTravelSpec)
static scala.collection.Seq<String>	NAMESPACE_RESERVED_PROPERTIES() The list of reserved namespace properties, which can not be removed or changed directly by the syntax: {{ ALTER NAMESPACE ...
static Column[]	structTypeToV2Columns(StructType schema) Converts a StructType to DS v2 columns, which decodes the StructField metadata to v2 column comment and default value or generation expression.
static scala.collection.Seq<String>	TABLE_RESERVED_PROPERTIES() The list of reserved table properties, which can not be removed or changed directly by the syntax: {{ ALTER TABLE ...
static StructType	v2ColumnsToStructType(Column[] columns) Converts DS v2 columns to StructType, which encodes column comment and default value to StructField metadata.
static scala.collection.immutable.Map<String,String>	withDefaultOwnership(scala.collection.immutable.Map<String,String> properties)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CatalogV2Util

public CatalogV2Util()

Method Detail

TABLE_RESERVED_PROPERTIES

public static scala.collection.Seq<String> TABLE_RESERVED_PROPERTIES()

The list of reserved table properties, which can not be removed or changed directly by the syntax: {{ ALTER TABLE ... SET TBLPROPERTIES ... }}

They need specific syntax to modify

Returns:

(undocumented)

NAMESPACE_RESERVED_PROPERTIES

public static scala.collection.Seq<String> NAMESPACE_RESERVED_PROPERTIES()

The list of reserved namespace properties, which can not be removed or changed directly by the syntax: {{ ALTER NAMESPACE ... SET PROPERTIES ... }}

They need specific syntax to modify

Returns:

(undocumented)

applyNamespaceChanges

public static scala.collection.immutable.Map<String,String> applyNamespaceChanges(scala.collection.immutable.Map<String,String> properties,
                                                                                  scala.collection.Seq<NamespaceChange> changes)

Apply properties changes to a map and return the result.

Parameters:

properties - (undocumented)

changes - (undocumented)

Returns:

(undocumented)

applyNamespaceChanges

public static java.util.Map<String,String> applyNamespaceChanges(java.util.Map<String,String> properties,
                                                                 scala.collection.Seq<NamespaceChange> changes)

Apply properties changes to a Java map and return the result.

Parameters:

properties - (undocumented)

changes - (undocumented)

Returns:

(undocumented)

applyPropertiesChanges

public static scala.collection.immutable.Map<String,String> applyPropertiesChanges(scala.collection.immutable.Map<String,String> properties,
                                                                                   scala.collection.Seq<TableChange> changes)

Apply properties changes to a map and return the result.

Parameters:

properties - (undocumented)

changes - (undocumented)

Returns:

(undocumented)

applyPropertiesChanges

public static java.util.Map<String,String> applyPropertiesChanges(java.util.Map<String,String> properties,
                                                                  scala.collection.Seq<TableChange> changes)

Apply properties changes to a Java map and return the result.

Parameters:

properties - (undocumented)

changes - (undocumented)

Returns:

(undocumented)

applySchemaChanges

public static StructType applySchemaChanges(StructType schema,
                                            scala.collection.Seq<TableChange> changes,
                                            scala.Option<String> tableProvider,
                                            String statementType)

Apply schema changes to a schema and return the result.

Parameters:

schema - (undocumented)

changes - (undocumented)

tableProvider - (undocumented)

statementType - (undocumented)

Returns:

(undocumented)

loadTable

public static scala.Option<Table> loadTable(CatalogPlugin catalog,
                                            Identifier ident,
                                            scala.Option<org.apache.spark.sql.catalyst.analysis.TimeTravelSpec> timeTravelSpec)

getTable

public static Table getTable(CatalogPlugin catalog,
                             Identifier ident,
                             scala.Option<org.apache.spark.sql.catalyst.analysis.TimeTravelSpec> timeTravelSpec)

loadFunction

public static scala.Option<UnboundFunction> loadFunction(CatalogPlugin catalog,
                                                         Identifier ident)

loadRelation

public static scala.Option<org.apache.spark.sql.catalyst.analysis.NamedRelation> loadRelation(CatalogPlugin catalog,
                                                                                              Identifier ident)

isSessionCatalog

public static boolean isSessionCatalog(CatalogPlugin catalog)

convertTableProperties

public static scala.collection.immutable.Map<String,String> convertTableProperties(org.apache.spark.sql.catalyst.plans.logical.TableSpec t)

withDefaultOwnership

public static scala.collection.immutable.Map<String,String> withDefaultOwnership(scala.collection.immutable.Map<String,String> properties)

getTableProviderCatalog

public static TableCatalog getTableProviderCatalog(SupportsCatalogOptions provider,
                                                   org.apache.spark.sql.connector.catalog.CatalogManager catalogManager,
                                                   CaseInsensitiveStringMap options)

v2ColumnsToStructType

public static StructType v2ColumnsToStructType(Column[] columns)

Converts DS v2 columns to StructType, which encodes column comment and default value to StructField metadata. This is mainly used to define the schema of v2 scan, w.r.t. the columns of the v2 table.

Parameters:

columns - (undocumented)

Returns:

(undocumented)

structTypeToV2Columns

public static Column[] structTypeToV2Columns(StructType schema)

Converts a StructType to DS v2 columns, which decodes the StructField metadata to v2 column comment and default value or generation expression. This is mainly used to generate DS v2 columns from table schema in DDL commands, so that Spark can pass DS v2 columns to DS v2 createTable and related APIs.

Parameters:

schema - (undocumented)

Returns:

(undocumented)