Package org.apache.spark.sql

Class TableValuedFunction

Object

org.apache.spark.sql.TableValuedFunction

public abstract class TableValuedFunction
extends Object

Interface for invoking table-valued functions in Spark SQL.

Since:

4.0.0

Constructor Summary

Constructors

Constructor	Description
TableValuedFunction()

Method Summary

Modifier and Type	Method	Description
abstract Dataset<Row>	collations()	Gets all of the Spark SQL string collations.
abstract Dataset<Row>	explode(Column collection)	Creates a DataFrame containing a new row for each element in the given array or map column.
abstract Dataset<Row>	explode_outer(Column collection)	Creates a DataFrame containing a new row for each element in the given array or map column.
abstract Dataset<Row>	inline(Column input)	Creates a DataFrame containing a new row for each element in the given array of structs.
abstract Dataset<Row>	inline_outer(Column input)	Creates a DataFrame containing a new row for each element in the given array of structs.
Dataset<Row>	json_tuple(Column input, Column... fields)	Creates a DataFrame containing a new row for a json column according to the given field names.
abstract Dataset<Row>	json_tuple(Column input, scala.collection.immutable.Seq<Column> fields)	Creates a DataFrame containing a new row for a json column according to the given field names.
abstract Dataset<Row>	posexplode(Column collection)	Creates a DataFrame containing a new row for each element with position in the given array or map column.
abstract Dataset<Row>	posexplode_outer(Column collection)	Creates a DataFrame containing a new row for each element with position in the given array or map column.
abstract Dataset<Long>	range(long end)	Creates a Dataset with a single LongType column named id, containing elements in a range from 0 to end (exclusive) with step value 1.
abstract Dataset<Long>	range(long start, long end)	Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with step value 1.
abstract Dataset<Long>	range(long start, long end, long step)	Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with a step value.
abstract Dataset<Long>	range(long start, long end, long step, int numPartitions)	Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with a step value, with partition number specified.
abstract Dataset<Row>	sql_keywords()	Gets Spark SQL keywords.
Dataset<Row>	stack(Column n, Column... fields)	Separates col1, ..., colk into n rows.
abstract Dataset<Row>	stack(Column n, scala.collection.immutable.Seq<Column> fields)	Separates col1, ..., colk into n rows.
abstract Dataset<Row>	variant_explode(Column input)	Separates a variant object/array into multiple rows containing its fields/elements.
abstract Dataset<Row>	variant_explode_outer(Column input)	Separates a variant object/array into multiple rows containing its fields/elements.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TableValuedFunction

public TableValuedFunction()

Method Details

collations

public abstract Dataset<Row> collations()

Gets all of the Spark SQL string collations.

Returns:

(undocumented)

Since:

4.0.0

explode

public abstract Dataset<Row> explode(Column collection)

Creates a DataFrame containing a new row for each element in the given array or map column. Uses the default column name col for elements in the array and key and value for elements in the map unless specified otherwise.

Parameters:

collection - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

explode_outer

public abstract Dataset<Row> explode_outer(Column collection)

Creates a DataFrame containing a new row for each element in the given array or map column. Uses the default column name col for elements in the array and key and value for elements in the map unless specified otherwise. Unlike explode, if the array/map is null or empty then null is produced.

Parameters:

collection - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

inline

public abstract Dataset<Row> inline(Column input)

Creates a DataFrame containing a new row for each element in the given array of structs.

Parameters:

input - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

inline_outer

public abstract Dataset<Row> inline_outer(Column input)

Creates a DataFrame containing a new row for each element in the given array of structs. Unlike inline, if the array is null or empty then null is produced for each nested column.

Parameters:

input - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

json_tuple

public Dataset<Row> json_tuple(Column input,
 Column... fields)

Creates a DataFrame containing a new row for a json column according to the given field names.

Parameters:

input - (undocumented)

fields - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

json_tuple

public abstract Dataset<Row> json_tuple(Column input,
 scala.collection.immutable.Seq<Column> fields)

Creates a DataFrame containing a new row for a json column according to the given field names.

Parameters:

input - (undocumented)

fields - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

posexplode

public abstract Dataset<Row> posexplode(Column collection)

Creates a DataFrame containing a new row for each element with position in the given array or map column. Uses the default column name pos for position, and col for elements in the array and key and value for elements in the map unless specified otherwise.

Parameters:

collection - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

posexplode_outer

public abstract Dataset<Row> posexplode_outer(Column collection)

Creates a DataFrame containing a new row for each element with position in the given array or map column. Uses the default column name pos for position, and col for elements in the array and key and value for elements in the map unless specified otherwise. Unlike posexplode, if the array/map is null or empty then the row (null, null) is produced.

Parameters:

collection - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

range

public abstract Dataset<Long> range(long end)

Creates a Dataset with a single LongType column named id, containing elements in a range from 0 to end (exclusive) with step value 1.

Parameters:

end - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

range

public abstract Dataset<Long> range(long start,
 long end)

Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with step value 1.

Parameters:

start - (undocumented)

end - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

range

public abstract Dataset<Long> range(long start,
 long end,
 long step)

Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with a step value.

Parameters:

start - (undocumented)

end - (undocumented)

step - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

range

public abstract Dataset<Long> range(long start,
 long end,
 long step,
 int numPartitions)

Creates a Dataset with a single LongType column named id, containing elements in a range from start to end (exclusive) with a step value, with partition number specified.

Parameters:

start - (undocumented)

end - (undocumented)

step - (undocumented)

numPartitions - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

sql_keywords

public abstract Dataset<Row> sql_keywords()

Gets Spark SQL keywords.

Returns:

(undocumented)

Since:

4.0.0

stack

public Dataset<Row> stack(Column n,
 Column... fields)

Separates col1, ..., colk into n rows. Uses column names col0, col1, etc. by default unless specified otherwise.

Parameters:

n - (undocumented)

fields - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

stack

public abstract Dataset<Row> stack(Column n,
 scala.collection.immutable.Seq<Column> fields)

Separates col1, ..., colk into n rows. Uses column names col0, col1, etc. by default unless specified otherwise.

Parameters:

n - (undocumented)

fields - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

variant_explode

public abstract Dataset<Row> variant_explode(Column input)

Separates a variant object/array into multiple rows containing its fields/elements. Its result schema is struct<pos int, key string, value variant>. pos is the position of the field/element in its parent object/array, and value is the field/element value. key is the field name when exploding a variant object, or is NULL when exploding a variant array. It ignores any input that is not a variant array/object, including SQL NULL, variant null, and any other variant values.

Parameters:

input - (undocumented)

Returns:

(undocumented)

Since:

4.0.0

variant_explode_outer

public abstract Dataset<Row> variant_explode_outer(Column input)

Separates a variant object/array into multiple rows containing its fields/elements. Its result schema is struct<pos int, key string, value variant>. pos is the position of the field/element in its parent object/array, and value is the field/element value. key is the field name when exploding a variant object, or is NULL when exploding a variant array. Unlike variant_explode, if the given variant is not a variant array/object, including SQL NULL, variant null, and any other variant values, then NULL is produced.

Parameters:

input - (undocumented)

Returns:

(undocumented)

Since:

4.0.0